JUnit version 4.13.2
.20:07:20.191 [DEBUG - main] (RandomUtils.java:49) Using random seed: 1779134840180
20:07:20.249 [INFO - main] (RetryRule.java:138) Creating RetryStatement with result reporter and retry count of 0 (no retries)
20:07:20.322 [INFO - main] (KuduTestHarness.java:155) Creating a new MiniKuduCluster...
20:07:20.338 [INFO - main] (TempDirUtils.java:71) Using the temp directory defined by TEST_TMPDIR: /tmp/dist-test-taskx5o66C/test-tmp
20:07:20.345 [INFO - main] (KuduBinaryLocator.java:83) Using Kudu binary directory specified by system property 'kuduBinDir': ../../../build/release/bin
20:07:20.348 [INFO - main] (MiniKuduCluster.java:217) Starting process: [/tmp/dist-test-taskx5o66C/java/build/dist-test/../../../build/release/bin/kudu, test, mini_cluster, --serialization=pb]
20:07:20.552 [DEBUG - main] (MiniKuduCluster.java:185) Request: create_cluster {
  num_masters: 3
  num_tservers: 3
  enable_kerberos: false
  cluster_root: "/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717"
  hms_mode: NONE
  mini_kdc_options {
  }
  principal: "kudu"
  mini_oidc_options {
  }
}

20:07:20.570 [DEBUG - main] (MiniKuduCluster.java:195) Response: 
20:07:20.572 [DEBUG - main] (MiniKuduCluster.java:185) Request: start_cluster {
}

20:07:20.574 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.573663  9114 mini_oidc.cc:150] Starting JWKS server
20:07:20.575 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.574512  9114 webserver.cc:533] Webserver started at http://127.0.0.1:45815/ using document root <none> and password file <none>
20:07:20.575 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.574610  9114 mini_oidc.cc:186] Starting OIDC Discovery server
20:07:20.575 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.574787  9114 webserver.cc:533] Webserver started at http://127.0.0.1:35685/ using document root <none> and password file <none>
20:07:20.584 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-05-18T20:07:20Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
20:07:20.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-05-18T20:07:20Z Disabled control of system clock
20:07:20.589 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.588521  9114 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:20.589 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:20.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-0/wal
20:07:20.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-0/data
20:07:20.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
20:07:20.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
20:07:20.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
20:07:20.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:20.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:20.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:20.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-0/logs
20:07:20.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-0/data/info.pb
20:07:20.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:20.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
20:07:20.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
20:07:20.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
20:07:20.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
20:07:20.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:20.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:20.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:20.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
20:07:20.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
20:07:20.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
20:07:20.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
20:07:20.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.190:34269
20:07:20.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.190
20:07:20.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
20:07:20.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:20.597 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:20.597 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:20.597 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
20:07:20.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
20:07:20.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:20.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
20:07:20.599 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:45815/jwks.json with env {}
20:07:20.677 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.676419  9125 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
20:07:20.677 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.676693  9125 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
20:07:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.676730  9125 flags.cc:432] Enabled unsafe flag: --never_fsync=true
20:07:20.679 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.679009  9125 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
20:07:20.679 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.679116  9125 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
20:07:20.680 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.679139  9125 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
20:07:20.680 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.679158  9125 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
20:07:20.680 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.679179  9125 flags.cc:432] Enabled experimental flag: --enable_jwt_token_auth=true
20:07:20.680 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.679206  9125 flags.cc:432] Enabled experimental flag: --jwks_url=http://localhost:45815/jwks.json
20:07:20.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.681710  9125 master_runner.cc:387] Master server non-default flags:
20:07:20.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:20.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:20.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:20.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
20:07:20.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-0/data
20:07:20.684 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-0/wal
20:07:20.685 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
20:07:20.685 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:20.685 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:20.686 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:20.686 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
20:07:20.686 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.190:34269
20:07:20.686 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
20:07:20.687 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
20:07:20.687 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
20:07:20.687 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:45815/jwks.json
20:07:20.687 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:20.687 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:20.688 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-0/data/info.pb
20:07:20.688 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.190
20:07:20.688 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
20:07:20.688 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
20:07:20.689 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.9125
20:07:20.689 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:20.689 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
20:07:20.689 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
20:07:20.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:20.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-0/logs
20:07:20.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:20.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
20:07:20.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
20:07:20.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
20:07:20.692 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
20:07:20.692 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 66827b15b158c37363c37baa160f29a0ca1f1e63
20:07:20.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type RELEASE
20:07:20.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 18 May 2026 19:43:16 UTC on e7f111948823
20:07:20.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 11961
20:07:20.694 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.682081  9125 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
20:07:20.694 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.682410  9125 file_cache.cc:492] Constructed file cache file cache with capacity 419430
20:07:20.694 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.685649  9133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:20.695 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.685823  9125 server_base.cc:1061] running on GCE node
20:07:20.695 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.685880  9131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:20.695 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.686096  9130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:20.695 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.686371  9125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
20:07:20.696 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.686715  9125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
20:07:20.696 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.687896  9125 hybrid_clock.cc:648] HybridClock initialized: now 1779134840687873 us; error 53 us; skew 500 ppm
20:07:20.696 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.689453  9125 webserver.cc:533] Webserver started at http://127.8.230.190:41449/ using document root <none> and password file <none>
20:07:20.696 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.689688  9125 fs_manager.cc:362] Metadata directory not provided
20:07:20.696 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.689764  9125 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
20:07:20.697 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.689877  9125 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
20:07:20.697 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.691143  9125 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-0/data/instance:
20:07:20.697 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "bac0a7cfdbfb4fb2b6f13da2ed73c097"
20:07:20.697 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:20 on dist-test-slave-mzrs"
20:07:20.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.691560  9125 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-0/wal/instance:
20:07:20.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "bac0a7cfdbfb4fb2b6f13da2ed73c097"
20:07:20.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:20 on dist-test-slave-mzrs"
20:07:20.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.693243  9125 fs_manager.cc:696] Time spent creating directory manager: real 0.001s	user 0.003s	sys 0.000s
20:07:20.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.694216  9139 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
20:07:20.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.694402  9125 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
20:07:20.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.694471  9125 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-0/data,/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-0/wal
20:07:20.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "bac0a7cfdbfb4fb2b6f13da2ed73c097"
20:07:20.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:20 on dist-test-slave-mzrs"
20:07:20.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.694523  9125 fs_report.cc:389] FS layout report
20:07:20.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
20:07:20.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-0/wal
20:07:20.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-0/wal
20:07:20.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-0/data/data
20:07:20.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
20:07:20.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
20:07:20.701 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
20:07:20.701 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
20:07:20.701 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
20:07:20.701 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
20:07:20.701 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
20:07:20.701 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
20:07:20.701 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
20:07:20.701 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
20:07:20.701 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
20:07:20.716 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.715508  9125 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
20:07:20.716 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.715833  9125 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
20:07:20.716 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.715927  9125 kserver.cc:163] Server-wide thread pool size limit: 3276
20:07:20.720 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.720217  9125 rpc_server.cc:307] RPC server started. Bound to: 127.8.230.190:34269
20:07:20.721 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.720252  9191 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.230.190:34269 every 8 connection(s)
20:07:20.721 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.720654  9125 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-0/data/info.pb
20:07:20.721 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.721731  9114 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskx5o66C/build/release/bin/kudu as pid 9125
20:07:20.722 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.721832  9114 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-0/wal/instance
20:07:20.722 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.722060  9192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
20:07:20.723 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.722842  9114 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:20.723 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:20.723 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-1/wal
20:07:20.723 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-1/data
20:07:20.723 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
20:07:20.724 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
20:07:20.724 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
20:07:20.724 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:20.724 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:20.724 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:20.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-1/logs
20:07:20.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-1/data/info.pb
20:07:20.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:20.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
20:07:20.726 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
20:07:20.726 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
20:07:20.726 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
20:07:20.726 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:20.726 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:20.726 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:20.727 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
20:07:20.727 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
20:07:20.727 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
20:07:20.727 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
20:07:20.727 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.189:44351
20:07:20.727 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.189
20:07:20.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
20:07:20.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:20.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:20.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:20.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
20:07:20.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
20:07:20.729 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:20.729 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
20:07:20.729 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:45815/jwks.json with env {}
20:07:20.730 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.724825  9192 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.230.190" port: 34269 } has no permanent_uuid. Determining permanent_uuid...
20:07:20.734 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.734118  9192 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.230.189" port: 44351 } has no permanent_uuid. Determining permanent_uuid...
20:07:20.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.734697  9143 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.230.189:44351: connect: Connection refused (error 111)
20:07:20.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.735320  9192 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.8.230.189:44351: Network error: Client connection negotiation failed: client connection to 127.8.230.189:44351: connect: Connection refused (error 111)
20:07:20.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.784574  9192 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.230.189" port: 44351 } attempt: 1
20:07:20.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.785641  9192 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.8.230.189:44351: Network error: Client connection negotiation failed: client connection to 127.8.230.189:44351: connect: Connection refused (error 111)
20:07:20.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.807654  9194 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
20:07:20.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.807910  9194 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
20:07:20.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.807933  9194 flags.cc:432] Enabled unsafe flag: --never_fsync=true
20:07:20.810 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.810149  9194 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
20:07:20.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.810221  9194 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
20:07:20.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.810237  9194 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
20:07:20.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.810250  9194 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
20:07:20.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.810333  9194 flags.cc:432] Enabled experimental flag: --enable_jwt_token_auth=true
20:07:20.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.810346  9194 flags.cc:432] Enabled experimental flag: --jwks_url=http://localhost:45815/jwks.json
20:07:20.812 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.812431  9194 master_runner.cc:387] Master server non-default flags:
20:07:20.813 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:20.813 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:20.813 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:20.813 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
20:07:20.813 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-1/data
20:07:20.813 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-1/wal
20:07:20.814 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
20:07:20.814 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:20.814 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:20.815 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:20.815 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
20:07:20.815 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.189:44351
20:07:20.815 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
20:07:20.815 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
20:07:20.815 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
20:07:20.816 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:45815/jwks.json
20:07:20.816 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:20.816 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:20.816 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-1/data/info.pb
20:07:20.816 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.189
20:07:20.817 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
20:07:20.817 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
20:07:20.817 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.9194
20:07:20.817 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:20.817 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
20:07:20.817 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
20:07:20.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:20.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-1/logs
20:07:20.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:20.819 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
20:07:20.819 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
20:07:20.819 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
20:07:20.819 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
20:07:20.819 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 66827b15b158c37363c37baa160f29a0ca1f1e63
20:07:20.820 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type RELEASE
20:07:20.820 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 18 May 2026 19:43:16 UTC on e7f111948823
20:07:20.820 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 11961
20:07:20.820 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.812762  9194 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
20:07:20.820 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.812973  9194 file_cache.cc:492] Constructed file cache file cache with capacity 419430
20:07:20.821 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.815214  9202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:20.821 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.815214  9205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:20.821 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.815416  9203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:20.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.816125  9194 server_base.cc:1061] running on GCE node
20:07:20.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.816387  9194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
20:07:20.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.816736  9194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
20:07:20.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.817914  9194 hybrid_clock.cc:648] HybridClock initialized: now 1779134840817879 us; error 48 us; skew 500 ppm
20:07:20.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.819450  9194 webserver.cc:533] Webserver started at http://127.8.230.189:34941/ using document root <none> and password file <none>
20:07:20.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.819701  9194 fs_manager.cc:362] Metadata directory not provided
20:07:20.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.819761  9194 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
20:07:20.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.819865  9194 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
20:07:20.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.821063  9194 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-1/data/instance:
20:07:20.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "d32e19f129bd45eead36995cd0715d70"
20:07:20.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:20 on dist-test-slave-mzrs"
20:07:20.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.821483  9194 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-1/wal/instance:
20:07:20.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "d32e19f129bd45eead36995cd0715d70"
20:07:20.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:20 on dist-test-slave-mzrs"
20:07:20.827 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.826718  9194 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.002s
20:07:20.828 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.827818  9211 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
20:07:20.828 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.828050  9194 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
20:07:20.828 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.828111  9194 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-1/data,/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-1/wal
20:07:20.829 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "d32e19f129bd45eead36995cd0715d70"
20:07:20.829 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:20 on dist-test-slave-mzrs"
20:07:20.829 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.828186  9194 fs_report.cc:389] FS layout report
20:07:20.829 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
20:07:20.830 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-1/wal
20:07:20.830 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-1/wal
20:07:20.830 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-1/data/data
20:07:20.830 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
20:07:20.830 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
20:07:20.830 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
20:07:20.831 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
20:07:20.831 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
20:07:20.831 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
20:07:20.831 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
20:07:20.831 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
20:07:20.831 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
20:07:20.832 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
20:07:20.832 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
20:07:20.839 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.838649  9194 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
20:07:20.839 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.839046  9194 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
20:07:20.839 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.839172  9194 kserver.cc:163] Server-wide thread pool size limit: 3276
20:07:20.844 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.843791  9194 rpc_server.cc:307] RPC server started. Bound to: 127.8.230.189:44351
20:07:20.844 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.843883  9263 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.230.189:44351 every 8 connection(s)
20:07:20.844 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.844134  9194 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-1/data/info.pb
20:07:20.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.844794  9264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
20:07:20.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.846182  9264 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.230.190" port: 34269 } has no permanent_uuid. Determining permanent_uuid...
20:07:20.851 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.850441  9114 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskx5o66C/build/release/bin/kudu as pid 9194
20:07:20.852 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.850529  9114 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-1/wal/instance
20:07:20.853 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.850912  9114 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:20.853 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:20.853 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-2/wal
20:07:20.853 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-2/data
20:07:20.853 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
20:07:20.853 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
20:07:20.853 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
20:07:20.853 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:20.854 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:20.854 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:20.854 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-2/logs
20:07:20.854 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-2/data/info.pb
20:07:20.855 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:20.855 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
20:07:20.855 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
20:07:20.855 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
20:07:20.855 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
20:07:20.855 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:20.855 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:20.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:20.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
20:07:20.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
20:07:20.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
20:07:20.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
20:07:20.857 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.188:39477
20:07:20.857 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.188
20:07:20.857 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
20:07:20.857 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:20.857 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:20.857 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:20.858 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
20:07:20.858 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
20:07:20.858 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:20.858 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
20:07:20.858 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:45815/jwks.json with env {}
20:07:20.858 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.851677  9264 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.230.189" port: 44351 } has no permanent_uuid. Determining permanent_uuid...
20:07:20.859 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.853837  9192 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.230.189" port: 44351 } attempt: 2
20:07:20.859 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.857978  9264 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } has no permanent_uuid. Determining permanent_uuid...
20:07:20.859 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.858461  9212 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.230.188:39477: connect: Connection refused (error 111)
20:07:20.860 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.860307  9264 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.8.230.188:39477: Network error: Client connection negotiation failed: client connection to 127.8.230.188:39477: connect: Connection refused (error 111)
20:07:20.862 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.862426  9192 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } has no permanent_uuid. Determining permanent_uuid...
20:07:20.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.863153  9192 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.8.230.188:39477: Network error: Client connection negotiation failed: client connection to 127.8.230.188:39477: connect: Connection refused (error 111)
20:07:20.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.906396  9192 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } attempt: 1
20:07:20.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.907423  9192 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.8.230.188:39477: Network error: Client connection negotiation failed: client connection to 127.8.230.188:39477: connect: Connection refused (error 111)
20:07:20.909 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.909533  9264 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } attempt: 1
20:07:20.910 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.910570  9264 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.8.230.188:39477: Network error: Client connection negotiation failed: client connection to 127.8.230.188:39477: connect: Connection refused (error 111)
20:07:20.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.946672  9267 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
20:07:20.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.946897  9267 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
20:07:20.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.946918  9267 flags.cc:432] Enabled unsafe flag: --never_fsync=true
20:07:20.948 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.948606  9267 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
20:07:20.949 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.948665  9267 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
20:07:20.949 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.948679  9267 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
20:07:20.949 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.948693  9267 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
20:07:20.949 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.948704  9267 flags.cc:432] Enabled experimental flag: --enable_jwt_token_auth=true
20:07:20.949 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.948716  9267 flags.cc:432] Enabled experimental flag: --jwks_url=http://localhost:45815/jwks.json
20:07:20.951 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.950871  9267 master_runner.cc:387] Master server non-default flags:
20:07:20.951 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:20.951 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:20.951 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:20.951 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
20:07:20.951 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-2/data
20:07:20.952 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-2/wal
20:07:20.952 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
20:07:20.952 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:20.952 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:20.952 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:20.952 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
20:07:20.952 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.188:39477
20:07:20.952 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
20:07:20.953 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
20:07:20.953 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
20:07:20.953 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:45815/jwks.json
20:07:20.953 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:20.953 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:20.953 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-2/data/info.pb
20:07:20.953 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.188
20:07:20.954 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
20:07:20.954 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
20:07:20.954 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.9267
20:07:20.954 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:20.954 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
20:07:20.954 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
20:07:20.955 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:20.955 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-2/logs
20:07:20.955 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:20.955 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
20:07:20.956 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
20:07:20.956 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
20:07:20.956 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
20:07:20.956 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 66827b15b158c37363c37baa160f29a0ca1f1e63
20:07:20.956 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type RELEASE
20:07:20.956 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 18 May 2026 19:43:16 UTC on e7f111948823
20:07:20.956 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 11961
20:07:20.957 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.951136  9267 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
20:07:20.957 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.951325  9267 file_cache.cc:492] Constructed file cache file cache with capacity 419430
20:07:20.957 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.953608  9278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:20.957 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.954597  9192 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } attempt: 2
20:07:20.957 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.954655  9267 server_base.cc:1061] running on GCE node
20:07:20.957 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.954846  9275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:20.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.955809  9192 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.8.230.188:39477: Network error: Client connection negotiation failed: client connection to 127.8.230.188:39477: connect: Connection refused (error 111)
20:07:20.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.955895  9276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:20.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.956189  9267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
20:07:20.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.956511  9267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
20:07:20.959 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.957690  9267 hybrid_clock.cc:648] HybridClock initialized: now 1779134840957654 us; error 43 us; skew 500 ppm
20:07:20.959 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.959342  9267 webserver.cc:533] Webserver started at http://127.8.230.188:34055/ using document root <none> and password file <none>
20:07:20.959 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.959594  9267 fs_manager.cc:362] Metadata directory not provided
20:07:20.959 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.959663  9267 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
20:07:20.959 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.959787  9267 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
20:07:20.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.960980  9267 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-2/data/instance:
20:07:20.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "1c7ed9528119451c982d1bb6eb68e442"
20:07:20.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:20 on dist-test-slave-mzrs"
20:07:20.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.961423  9267 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-2/wal/instance:
20:07:20.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "1c7ed9528119451c982d1bb6eb68e442"
20:07:20.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:20 on dist-test-slave-mzrs"
20:07:20.963 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.963368  9267 fs_manager.cc:696] Time spent creating directory manager: real 0.002s	user 0.000s	sys 0.004s
20:07:20.964 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.964452  9284 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
20:07:20.964 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.964679  9267 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
20:07:20.965 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.964756  9267 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-2/data,/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-2/wal
20:07:20.965 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "1c7ed9528119451c982d1bb6eb68e442"
20:07:20.965 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:20 on dist-test-slave-mzrs"
20:07:20.965 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.964819  9267 fs_report.cc:389] FS layout report
20:07:20.965 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
20:07:20.965 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-2/wal
20:07:20.965 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-2/wal
20:07:20.965 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-2/data/data
20:07:20.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
20:07:20.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
20:07:20.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
20:07:20.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
20:07:20.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
20:07:20.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
20:07:20.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
20:07:20.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
20:07:20.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
20:07:20.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
20:07:20.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
20:07:20.979 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:20.978771  9264 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } attempt: 2
20:07:20.979 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:20.979717  9264 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.8.230.188:39477: Network error: Client connection negotiation failed: client connection to 127.8.230.188:39477: connect: Connection refused (error 111)
20:07:21.002 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.001763  9267 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
20:07:21.002 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.002099  9267 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
20:07:21.002 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.002199  9267 kserver.cc:163] Server-wide thread pool size limit: 3276
20:07:21.006 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.006482  9267 rpc_server.cc:307] RPC server started. Bound to: 127.8.230.188:39477
20:07:21.007 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.006546  9336 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.230.188:39477 every 8 connection(s)
20:07:21.007 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.006844  9267 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-2/data/info.pb
20:07:21.007 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.007534  9337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
20:07:21.009 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.009024  9337 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.230.190" port: 34269 } has no permanent_uuid. Determining permanent_uuid...
20:07:21.010 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.010228  9114 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskx5o66C/build/release/bin/kudu as pid 9267
20:07:21.010 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.010363  9114 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/master-2/wal/instance
20:07:21.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.014922  9337 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.230.189" port: 44351 } has no permanent_uuid. Determining permanent_uuid...
20:07:21.019 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.018725  9337 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } has no permanent_uuid. Determining permanent_uuid...
20:07:21.024 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.024344  9337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442: Bootstrap starting.
20:07:21.025 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.025120  9337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442: Neither blocks nor log segments found. Creating new log.
20:07:21.025 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.025353  9337 log.cc:826] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442: Log is configured to *not* fsync() on all Append() calls
20:07:21.026 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.026103  9337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442: No bootstrap required, opened a new log
20:07:21.028 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.027794  9337 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bac0a7cfdbfb4fb2b6f13da2ed73c097" member_type: VOTER last_known_addr { host: "127.8.230.190" port: 34269 } } peers { permanent_uuid: "d32e19f129bd45eead36995cd0715d70" member_type: VOTER last_known_addr { host: "127.8.230.189" port: 44351 } } peers { permanent_uuid: "1c7ed9528119451c982d1bb6eb68e442" member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } }
20:07:21.028 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.027983  9337 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
20:07:21.028 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.028012  9337 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c7ed9528119451c982d1bb6eb68e442, State: Initialized, Role: FOLLOWER
20:07:21.028 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.028136  9337 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bac0a7cfdbfb4fb2b6f13da2ed73c097" member_type: VOTER last_known_addr { host: "127.8.230.190" port: 34269 } } peers { permanent_uuid: "d32e19f129bd45eead36995cd0715d70" member_type: VOTER last_known_addr { host: "127.8.230.189" port: 44351 } } peers { permanent_uuid: "1c7ed9528119451c982d1bb6eb68e442" member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } }
20:07:21.028 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.028434  9337 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [sys.catalog]: configured and running, proceeding with master startup.
20:07:21.029 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.028458  9345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bac0a7cfdbfb4fb2b6f13da2ed73c097" member_type: VOTER last_known_addr { host: "127.8.230.190" port: 34269 } } peers { permanent_uuid: "d32e19f129bd45eead36995cd0715d70" member_type: VOTER last_known_addr { host: "127.8.230.189" port: 44351 } } peers { permanent_uuid: "1c7ed9528119451c982d1bb6eb68e442" member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } } }
20:07:21.029 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.028556  9345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [sys.catalog]: This master's current role is: FOLLOWER
20:07:21.030 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.030678  9356 catalog_manager.cc:1593] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
20:07:21.031 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.030756  9356 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
20:07:21.032 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.030750  9267 master_runner.cc:187] Error getting master registration for 127.8.230.190:34269: OK, instance_id { permanent_uuid: "bac0a7cfdbfb4fb2b6f13da2ed73c097" instance_seqno: 1779134840719010 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
20:07:21.032 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.032279  9267 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
20:07:21.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.062979  9192 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } attempt: 3
20:07:21.068 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.067444  9192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bac0a7cfdbfb4fb2b6f13da2ed73c097: Bootstrap starting.
20:07:21.068 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.068341  9192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bac0a7cfdbfb4fb2b6f13da2ed73c097: Neither blocks nor log segments found. Creating new log.
20:07:21.069 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.069029  9192 log.cc:826] T 00000000000000000000000000000000 P bac0a7cfdbfb4fb2b6f13da2ed73c097: Log is configured to *not* fsync() on all Append() calls
20:07:21.070 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.069919  9192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bac0a7cfdbfb4fb2b6f13da2ed73c097: No bootstrap required, opened a new log
20:07:21.071 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.070896  9264 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } attempt: 3
20:07:21.072 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.071750  9192 raft_consensus.cc:359] T 00000000000000000000000000000000 P bac0a7cfdbfb4fb2b6f13da2ed73c097 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bac0a7cfdbfb4fb2b6f13da2ed73c097" member_type: VOTER last_known_addr { host: "127.8.230.190" port: 34269 } } peers { permanent_uuid: "d32e19f129bd45eead36995cd0715d70" member_type: VOTER last_known_addr { host: "127.8.230.189" port: 44351 } } peers { permanent_uuid: "1c7ed9528119451c982d1bb6eb68e442" member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } }
20:07:21.073 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.071899  9192 raft_consensus.cc:385] T 00000000000000000000000000000000 P bac0a7cfdbfb4fb2b6f13da2ed73c097 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
20:07:21.073 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.071933  9192 raft_consensus.cc:740] T 00000000000000000000000000000000 P bac0a7cfdbfb4fb2b6f13da2ed73c097 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bac0a7cfdbfb4fb2b6f13da2ed73c097, State: Initialized, Role: FOLLOWER
20:07:21.073 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.072044  9192 consensus_queue.cc:260] T 00000000000000000000000000000000 P bac0a7cfdbfb4fb2b6f13da2ed73c097 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bac0a7cfdbfb4fb2b6f13da2ed73c097" member_type: VOTER last_known_addr { host: "127.8.230.190" port: 34269 } } peers { permanent_uuid: "d32e19f129bd45eead36995cd0715d70" member_type: VOTER last_known_addr { host: "127.8.230.189" port: 44351 } } peers { permanent_uuid: "1c7ed9528119451c982d1bb6eb68e442" member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } }
20:07:21.073 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.072396  9192 sys_catalog.cc:565] T 00000000000000000000000000000000 P bac0a7cfdbfb4fb2b6f13da2ed73c097 [sys.catalog]: configured and running, proceeding with master startup.
20:07:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.072352  9359 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac0a7cfdbfb4fb2b6f13da2ed73c097 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bac0a7cfdbfb4fb2b6f13da2ed73c097" member_type: VOTER last_known_addr { host: "127.8.230.190" port: 34269 } } peers { permanent_uuid: "d32e19f129bd45eead36995cd0715d70" member_type: VOTER last_known_addr { host: "127.8.230.189" port: 44351 } } peers { permanent_uuid: "1c7ed9528119451c982d1bb6eb68e442" member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } } }
20:07:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.072448  9359 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac0a7cfdbfb4fb2b6f13da2ed73c097 [sys.catalog]: This master's current role is: FOLLOWER
20:07:21.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.075593  9264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d32e19f129bd45eead36995cd0715d70: Bootstrap starting.
20:07:21.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.076282  9264 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d32e19f129bd45eead36995cd0715d70: Neither blocks nor log segments found. Creating new log.
20:07:21.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.076603  9264 log.cc:826] T 00000000000000000000000000000000 P d32e19f129bd45eead36995cd0715d70: Log is configured to *not* fsync() on all Append() calls
20:07:21.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.077773  9264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d32e19f129bd45eead36995cd0715d70: No bootstrap required, opened a new log
20:07:21.079 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.077147  9125 master_runner.cc:187] Error getting master registration for 127.8.230.189:44351: OK, instance_id { permanent_uuid: "d32e19f129bd45eead36995cd0715d70" instance_seqno: 1779134840842696 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
20:07:21.079 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.078959  9125 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
20:07:21.079 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.079107  9370 catalog_manager.cc:1593] T 00000000000000000000000000000000 P bac0a7cfdbfb4fb2b6f13da2ed73c097: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
20:07:21.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.079157  9370 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
20:07:21.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.079852  9264 raft_consensus.cc:359] T 00000000000000000000000000000000 P d32e19f129bd45eead36995cd0715d70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bac0a7cfdbfb4fb2b6f13da2ed73c097" member_type: VOTER last_known_addr { host: "127.8.230.190" port: 34269 } } peers { permanent_uuid: "d32e19f129bd45eead36995cd0715d70" member_type: VOTER last_known_addr { host: "127.8.230.189" port: 44351 } } peers { permanent_uuid: "1c7ed9528119451c982d1bb6eb68e442" member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } }
20:07:21.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.079962  9264 raft_consensus.cc:385] T 00000000000000000000000000000000 P d32e19f129bd45eead36995cd0715d70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
20:07:21.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.079983  9264 raft_consensus.cc:740] T 00000000000000000000000000000000 P d32e19f129bd45eead36995cd0715d70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d32e19f129bd45eead36995cd0715d70, State: Initialized, Role: FOLLOWER
20:07:21.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.080103  9264 consensus_queue.cc:260] T 00000000000000000000000000000000 P d32e19f129bd45eead36995cd0715d70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bac0a7cfdbfb4fb2b6f13da2ed73c097" member_type: VOTER last_known_addr { host: "127.8.230.190" port: 34269 } } peers { permanent_uuid: "d32e19f129bd45eead36995cd0715d70" member_type: VOTER last_known_addr { host: "127.8.230.189" port: 44351 } } peers { permanent_uuid: "1c7ed9528119451c982d1bb6eb68e442" member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } }
20:07:21.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.080363  9373 sys_catalog.cc:455] T 00000000000000000000000000000000 P d32e19f129bd45eead36995cd0715d70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bac0a7cfdbfb4fb2b6f13da2ed73c097" member_type: VOTER last_known_addr { host: "127.8.230.190" port: 34269 } } peers { permanent_uuid: "d32e19f129bd45eead36995cd0715d70" member_type: VOTER last_known_addr { host: "127.8.230.189" port: 44351 } } peers { permanent_uuid: "1c7ed9528119451c982d1bb6eb68e442" member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } } }
20:07:21.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.080420  9264 sys_catalog.cc:565] T 00000000000000000000000000000000 P d32e19f129bd45eead36995cd0715d70 [sys.catalog]: configured and running, proceeding with master startup.
20:07:21.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.080423  9373 sys_catalog.cc:458] T 00000000000000000000000000000000 P d32e19f129bd45eead36995cd0715d70 [sys.catalog]: This master's current role is: FOLLOWER
20:07:21.082 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.082420  9384 catalog_manager.cc:1593] T 00000000000000000000000000000000 P d32e19f129bd45eead36995cd0715d70: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
20:07:21.083 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.082873  9384 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
20:07:21.084 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.084681  9194 master_runner.cc:264] No leader master found from master d32e19f129bd45eead36995cd0715d70
20:07:21.085 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.084749  9194 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
20:07:21.091 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.090657  9217 tablet.cc:2404] T 00000000000000000000000000000000 P d32e19f129bd45eead36995cd0715d70: Can't schedule compaction. Clean time has not been advanced past its initial value.
20:07:21.218 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.216693  9145 tablet.cc:2404] T 00000000000000000000000000000000 P bac0a7cfdbfb4fb2b6f13da2ed73c097: Can't schedule compaction. Clean time has not been advanced past its initial value.
20:07:21.253 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.252971  9290 tablet.cc:2404] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442: Can't schedule compaction. Clean time has not been advanced past its initial value.
20:07:21.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.302942  9345 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
20:07:21.304 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.303104  9345 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bac0a7cfdbfb4fb2b6f13da2ed73c097" member_type: VOTER last_known_addr { host: "127.8.230.190" port: 34269 } } peers { permanent_uuid: "d32e19f129bd45eead36995cd0715d70" member_type: VOTER last_known_addr { host: "127.8.230.189" port: 44351 } } peers { permanent_uuid: "1c7ed9528119451c982d1bb6eb68e442" member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } }
20:07:21.305 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.303398  9345 leader_election.cc:290] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bac0a7cfdbfb4fb2b6f13da2ed73c097 (127.8.230.190:34269), d32e19f129bd45eead36995cd0715d70 (127.8.230.189:44351)
20:07:21.305 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.303771  9238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1c7ed9528119451c982d1bb6eb68e442" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d32e19f129bd45eead36995cd0715d70" is_pre_election: true
20:07:21.305 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.303758  9166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1c7ed9528119451c982d1bb6eb68e442" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bac0a7cfdbfb4fb2b6f13da2ed73c097" is_pre_election: true
20:07:21.307 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.303912  9166 raft_consensus.cc:2468] T 00000000000000000000000000000000 P bac0a7cfdbfb4fb2b6f13da2ed73c097 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c7ed9528119451c982d1bb6eb68e442 in term 0.
20:07:21.307 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.303912  9238 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d32e19f129bd45eead36995cd0715d70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c7ed9528119451c982d1bb6eb68e442 in term 0.
20:07:21.307 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.304152  9285 leader_election.cc:304] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c7ed9528119451c982d1bb6eb68e442, bac0a7cfdbfb4fb2b6f13da2ed73c097; no voters: 
20:07:21.307 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.304370  9345 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [term 0 FOLLOWER]: Leader pre-election won for term 1
20:07:21.308 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.304435  9345 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
20:07:21.308 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.304462  9345 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [term 0 FOLLOWER]: Advancing to term 1
20:07:21.309 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.305506  9345 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bac0a7cfdbfb4fb2b6f13da2ed73c097" member_type: VOTER last_known_addr { host: "127.8.230.190" port: 34269 } } peers { permanent_uuid: "d32e19f129bd45eead36995cd0715d70" member_type: VOTER last_known_addr { host: "127.8.230.189" port: 44351 } } peers { permanent_uuid: "1c7ed9528119451c982d1bb6eb68e442" member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } }
20:07:21.309 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.305707  9345 leader_election.cc:290] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [CANDIDATE]: Term 1 election: Requested vote from peers bac0a7cfdbfb4fb2b6f13da2ed73c097 (127.8.230.190:34269), d32e19f129bd45eead36995cd0715d70 (127.8.230.189:44351)
20:07:21.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.305928  9166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1c7ed9528119451c982d1bb6eb68e442" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bac0a7cfdbfb4fb2b6f13da2ed73c097"
20:07:21.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.305982  9238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1c7ed9528119451c982d1bb6eb68e442" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d32e19f129bd45eead36995cd0715d70"
20:07:21.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.306020  9166 raft_consensus.cc:3060] T 00000000000000000000000000000000 P bac0a7cfdbfb4fb2b6f13da2ed73c097 [term 0 FOLLOWER]: Advancing to term 1
20:07:21.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.306072  9238 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d32e19f129bd45eead36995cd0715d70 [term 0 FOLLOWER]: Advancing to term 1
20:07:21.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.306936  9166 raft_consensus.cc:2468] T 00000000000000000000000000000000 P bac0a7cfdbfb4fb2b6f13da2ed73c097 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c7ed9528119451c982d1bb6eb68e442 in term 1.
20:07:21.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.306980  9238 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d32e19f129bd45eead36995cd0715d70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c7ed9528119451c982d1bb6eb68e442 in term 1.
20:07:21.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.307170  9285 leader_election.cc:304] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c7ed9528119451c982d1bb6eb68e442, bac0a7cfdbfb4fb2b6f13da2ed73c097; no voters: 
20:07:21.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.307271  9345 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [term 1 FOLLOWER]: Leader election won for term 1
20:07:21.313 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.307442  9345 raft_consensus.cc:697] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [term 1 LEADER]: Becoming Leader. State: Replica: 1c7ed9528119451c982d1bb6eb68e442, State: Running, Role: LEADER
20:07:21.313 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.307535  9345 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bac0a7cfdbfb4fb2b6f13da2ed73c097" member_type: VOTER last_known_addr { host: "127.8.230.190" port: 34269 } } peers { permanent_uuid: "d32e19f129bd45eead36995cd0715d70" member_type: VOTER last_known_addr { host: "127.8.230.189" port: 44351 } } peers { permanent_uuid: "1c7ed9528119451c982d1bb6eb68e442" member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } }
20:07:21.313 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.307883  9345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c7ed9528119451c982d1bb6eb68e442. Latest consensus state: current_term: 1 leader_uuid: "1c7ed9528119451c982d1bb6eb68e442" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bac0a7cfdbfb4fb2b6f13da2ed73c097" member_type: VOTER last_known_addr { host: "127.8.230.190" port: 34269 } } peers { permanent_uuid: "d32e19f129bd45eead36995cd0715d70" member_type: VOTER last_known_addr { host: "127.8.230.189" port: 44351 } } peers { permanent_uuid: "1c7ed9528119451c982d1bb6eb68e442" member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } } }
20:07:21.313 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.307943  9345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [sys.catalog]: This master's current role is: LEADER
20:07:21.313 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.308591  9388 catalog_manager.cc:1510] Loading table and tablet metadata into memory...
20:07:21.314 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.308763  9388 catalog_manager.cc:1519] Initializing Kudu cluster ID...
20:07:21.314 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.310189  9238 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d32e19f129bd45eead36995cd0715d70 [term 1 FOLLOWER]: Refusing update from remote peer 1c7ed9528119451c982d1bb6eb68e442: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
20:07:21.314 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.310513  9386 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d32e19f129bd45eead36995cd0715d70" member_type: VOTER last_known_addr { host: "127.8.230.189" port: 44351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
20:07:21.314 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.311689  9373 sys_catalog.cc:455] T 00000000000000000000000000000000 P d32e19f129bd45eead36995cd0715d70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c7ed9528119451c982d1bb6eb68e442. Latest consensus state: current_term: 1 leader_uuid: "1c7ed9528119451c982d1bb6eb68e442" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bac0a7cfdbfb4fb2b6f13da2ed73c097" member_type: VOTER last_known_addr { host: "127.8.230.190" port: 34269 } } peers { permanent_uuid: "d32e19f129bd45eead36995cd0715d70" member_type: VOTER last_known_addr { host: "127.8.230.189" port: 44351 } } peers { permanent_uuid: "1c7ed9528119451c982d1bb6eb68e442" member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } } }
20:07:21.314 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.311779  9373 sys_catalog.cc:458] T 00000000000000000000000000000000 P d32e19f129bd45eead36995cd0715d70 [sys.catalog]: This master's current role is: FOLLOWER
20:07:21.314 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.312214  9166 raft_consensus.cc:1275] T 00000000000000000000000000000000 P bac0a7cfdbfb4fb2b6f13da2ed73c097 [term 1 FOLLOWER]: Refusing update from remote peer 1c7ed9528119451c982d1bb6eb68e442: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
20:07:21.315 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.312461  9386 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bac0a7cfdbfb4fb2b6f13da2ed73c097" member_type: VOTER last_known_addr { host: "127.8.230.190" port: 34269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
20:07:21.315 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.313587  9359 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac0a7cfdbfb4fb2b6f13da2ed73c097 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c7ed9528119451c982d1bb6eb68e442. Latest consensus state: current_term: 1 leader_uuid: "1c7ed9528119451c982d1bb6eb68e442" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bac0a7cfdbfb4fb2b6f13da2ed73c097" member_type: VOTER last_known_addr { host: "127.8.230.190" port: 34269 } } peers { permanent_uuid: "d32e19f129bd45eead36995cd0715d70" member_type: VOTER last_known_addr { host: "127.8.230.189" port: 44351 } } peers { permanent_uuid: "1c7ed9528119451c982d1bb6eb68e442" member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } } }
20:07:21.315 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.313674  9359 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac0a7cfdbfb4fb2b6f13da2ed73c097 [sys.catalog]: This master's current role is: FOLLOWER
20:07:21.316 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.316412  9345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1c7ed9528119451c982d1bb6eb68e442" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bac0a7cfdbfb4fb2b6f13da2ed73c097" member_type: VOTER last_known_addr { host: "127.8.230.190" port: 34269 } } peers { permanent_uuid: "d32e19f129bd45eead36995cd0715d70" member_type: VOTER last_known_addr { host: "127.8.230.189" port: 44351 } } peers { permanent_uuid: "1c7ed9528119451c982d1bb6eb68e442" member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } } }
20:07:21.317 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.316522  9345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [sys.catalog]: This master's current role is: LEADER
20:07:21.317 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.316569  9389 mvcc.cc:204] Tried to move back new op lower bound from 7287336310003892224 to 7287336309996175360. Current Snapshot: MvccSnapshot[applied={T|T < 7287336310003892224}]
20:07:21.317 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.316655  9373 sys_catalog.cc:455] T 00000000000000000000000000000000 P d32e19f129bd45eead36995cd0715d70 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1c7ed9528119451c982d1bb6eb68e442" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bac0a7cfdbfb4fb2b6f13da2ed73c097" member_type: VOTER last_known_addr { host: "127.8.230.190" port: 34269 } } peers { permanent_uuid: "d32e19f129bd45eead36995cd0715d70" member_type: VOTER last_known_addr { host: "127.8.230.189" port: 44351 } } peers { permanent_uuid: "1c7ed9528119451c982d1bb6eb68e442" member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } } }
20:07:21.317 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.316759  9373 sys_catalog.cc:458] T 00000000000000000000000000000000 P d32e19f129bd45eead36995cd0715d70 [sys.catalog]: This master's current role is: FOLLOWER
20:07:21.318 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.316818  9390 mvcc.cc:204] Tried to move back new op lower bound from 7287336310003892224 to 7287336309996175360. Current Snapshot: MvccSnapshot[applied={T|T < 7287336310003892224}]
20:07:21.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.318972  9388 catalog_manager.cc:1382] Generated new cluster ID: 0b06460bc3584fb6b186d8a3926e1792
20:07:21.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.319064  9388 catalog_manager.cc:1530] Initializing Kudu internal certificate authority...
20:07:21.320 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.320640  9345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1c7ed9528119451c982d1bb6eb68e442" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bac0a7cfdbfb4fb2b6f13da2ed73c097" member_type: VOTER last_known_addr { host: "127.8.230.190" port: 34269 } } peers { permanent_uuid: "d32e19f129bd45eead36995cd0715d70" member_type: VOTER last_known_addr { host: "127.8.230.189" port: 44351 } } peers { permanent_uuid: "1c7ed9528119451c982d1bb6eb68e442" member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } } }
20:07:21.321 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.320765  9345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442 [sys.catalog]: This master's current role is: LEADER
20:07:21.321 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.321229  9359 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac0a7cfdbfb4fb2b6f13da2ed73c097 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1c7ed9528119451c982d1bb6eb68e442" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bac0a7cfdbfb4fb2b6f13da2ed73c097" member_type: VOTER last_known_addr { host: "127.8.230.190" port: 34269 } } peers { permanent_uuid: "d32e19f129bd45eead36995cd0715d70" member_type: VOTER last_known_addr { host: "127.8.230.189" port: 44351 } } peers { permanent_uuid: "1c7ed9528119451c982d1bb6eb68e442" member_type: VOTER last_known_addr { host: "127.8.230.188" port: 39477 } } }
20:07:21.321 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.321328  9359 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac0a7cfdbfb4fb2b6f13da2ed73c097 [sys.catalog]: This master's current role is: FOLLOWER
20:07:21.321 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.321400  9392 mvcc.cc:204] Tried to move back new op lower bound from 7287336310003892224 to 7287336309996175360. Current Snapshot: MvccSnapshot[applied={T|T < 7287336310003892224}]
20:07:21.333 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.332800  9388 catalog_manager.cc:1405] Generated new certificate authority record
20:07:21.333 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.333590  9388 catalog_manager.cc:1539] Loading token signing keys...
20:07:21.353 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.353437  9388 catalog_manager.cc:6069] T 00000000000000000000000000000000 P 1c7ed9528119451c982d1bb6eb68e442: Generated new TSK 0
20:07:21.354 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.354223  9388 catalog_manager.cc:1549] Initializing in-progress tserver states...
20:07:21.364 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.363721  9114 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:21.364 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:21.364 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal
20:07:21.364 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data
20:07:21.364 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
20:07:21.364 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
20:07:21.364 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
20:07:21.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:21.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:21.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:21.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/logs
20:07:21.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data/info.pb
20:07:21.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:21.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
20:07:21.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
20:07:21.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
20:07:21.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
20:07:21.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:21.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:21.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:21.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
20:07:21.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
20:07:21.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.129:0
20:07:21.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.8.230.129
20:07:21.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.129
20:07:21.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
20:07:21.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:21.367 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:21.367 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:21.367 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:21.367 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
20:07:21.445 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.444559  9398 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
20:07:21.445 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.444828  9398 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
20:07:21.445 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.444861  9398 flags.cc:432] Enabled unsafe flag: --never_fsync=true
20:07:21.447 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.446908  9398 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
20:07:21.447 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.447026  9398 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.230.129
20:07:21.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.449011  9398 tablet_server_runner.cc:78] Tablet server non-default flags:
20:07:21.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:21.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:21.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:21.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
20:07:21.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data
20:07:21.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal
20:07:21.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:21.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:21.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.129:0
20:07:21.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
20:07:21.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:21.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:21.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data/info.pb
20:07:21.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.129
20:07:21.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
20:07:21.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:21.451 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
20:07:21.451 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.9398
20:07:21.451 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:21.451 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
20:07:21.451 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
20:07:21.451 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:21.451 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.8.230.129
20:07:21.451 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/logs
20:07:21.452 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:21.452 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
20:07:21.452 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
20:07:21.452 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
20:07:21.452 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
20:07:21.452 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 66827b15b158c37363c37baa160f29a0ca1f1e63
20:07:21.452 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type RELEASE
20:07:21.452 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 18 May 2026 19:43:16 UTC on e7f111948823
20:07:21.453 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 11961
20:07:21.453 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.449357  9398 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
20:07:21.453 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.449659  9398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
20:07:21.453 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.452610  9403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:21.453 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.452823  9404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:21.453 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.452914  9406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:21.453 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.453351  9398 server_base.cc:1061] running on GCE node
20:07:21.453 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.453531  9398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
20:07:21.454 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.453819  9398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
20:07:21.455 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.454969  9398 hybrid_clock.cc:648] HybridClock initialized: now 1779134841454937 us; error 47 us; skew 500 ppm
20:07:21.456 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.456303  9398 webserver.cc:533] Webserver started at http://127.8.230.129:44243/ using document root <none> and password file <none>
20:07:21.456 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.456547  9398 fs_manager.cc:362] Metadata directory not provided
20:07:21.456 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.456609  9398 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
20:07:21.456 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.456719  9398 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
20:07:21.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.457696  9398 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data/instance:
20:07:21.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "3ab092e11f74445fbbe7a4c29511b76b"
20:07:21.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:21 on dist-test-slave-mzrs"
20:07:21.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.458106  9398 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal/instance:
20:07:21.459 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "3ab092e11f74445fbbe7a4c29511b76b"
20:07:21.459 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:21 on dist-test-slave-mzrs"
20:07:21.459 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.459770  9398 fs_manager.cc:696] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
20:07:21.460 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.460623  9412 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
20:07:21.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.460880  9398 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
20:07:21.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.460958  9398 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data,/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal
20:07:21.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "3ab092e11f74445fbbe7a4c29511b76b"
20:07:21.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:21 on dist-test-slave-mzrs"
20:07:21.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.461023  9398 fs_report.cc:389] FS layout report
20:07:21.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
20:07:21.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal
20:07:21.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal
20:07:21.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data/data
20:07:21.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
20:07:21.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
20:07:21.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
20:07:21.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
20:07:21.462 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
20:07:21.462 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
20:07:21.462 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
20:07:21.462 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
20:07:21.462 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
20:07:21.462 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
20:07:21.462 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
20:07:21.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.472591  9398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
20:07:21.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.473014  9398 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
20:07:21.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.473177  9398 kserver.cc:163] Server-wide thread pool size limit: 3276
20:07:21.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.473410  9398 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
20:07:21.474 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.473863  9398 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
20:07:21.474 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.473913  9398 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
20:07:21.474 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.473946  9398 ts_tablet_manager.cc:616] Registered 0 tablets
20:07:21.474 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.473964  9398 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
20:07:21.481 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.481622  9398 rpc_server.cc:307] RPC server started. Bound to: 127.8.230.129:42709
20:07:21.482 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.481739  9525 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.230.129:42709 every 8 connection(s)
20:07:21.483 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.482872  9398 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data/info.pb
20:07:21.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.489977  9114 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskx5o66C/build/release/bin/kudu as pid 9398
20:07:21.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.490125  9114 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal/instance
20:07:21.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.491525  9114 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:21.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:21.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/wal
20:07:21.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data
20:07:21.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
20:07:21.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
20:07:21.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
20:07:21.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:21.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:21.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:21.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/logs
20:07:21.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data/info.pb
20:07:21.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:21.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
20:07:21.493 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
20:07:21.493 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
20:07:21.493 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
20:07:21.493 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:21.493 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:21.493 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:21.494 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
20:07:21.494 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
20:07:21.494 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.130:0
20:07:21.494 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.8.230.130
20:07:21.494 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.130
20:07:21.494 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
20:07:21.494 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:21.494 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:21.494 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:21.494 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:21.494 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
20:07:21.494 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.493557  9526 heartbeater.cc:344] Connected to a master server at 127.8.230.188:39477
20:07:21.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.493692  9526 heartbeater.cc:461] Registering TS with master...
20:07:21.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.493994  9526 heartbeater.cc:507] Master 127.8.230.188:39477 requested a full tablet report, sending...
20:07:21.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.494166  9528 heartbeater.cc:344] Connected to a master server at 127.8.230.189:44351
20:07:21.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.494252  9528 heartbeater.cc:461] Registering TS with master...
20:07:21.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.494391  9528 heartbeater.cc:507] Master 127.8.230.189:44351 requested a full tablet report, sending...
20:07:21.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.494587  9301 ts_manager.cc:194] Registered new tserver with Master: 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709)
20:07:21.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.495214  9301 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.8.230.129:60757
20:07:21.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.495843  9228 ts_manager.cc:194] Registered new tserver with Master: 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709)
20:07:21.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.496254  9527 heartbeater.cc:344] Connected to a master server at 127.8.230.190:34269
20:07:21.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.496332  9527 heartbeater.cc:461] Registering TS with master...
20:07:21.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.496394  9527 heartbeater.cc:507] Master 127.8.230.190:34269 requested a full tablet report, sending...
20:07:21.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.496734  9154 ts_manager.cc:194] Registered new tserver with Master: 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709)
20:07:21.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.577191  9533 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
20:07:21.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.577446  9533 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
20:07:21.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.577481  9533 flags.cc:432] Enabled unsafe flag: --never_fsync=true
20:07:21.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.579545  9533 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
20:07:21.580 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.579679  9533 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.230.130
20:07:21.582 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.581856  9533 tablet_server_runner.cc:78] Tablet server non-default flags:
20:07:21.582 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:21.582 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:21.582 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:21.582 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
20:07:21.582 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data
20:07:21.582 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/wal
20:07:21.582 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:21.582 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:21.582 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.130:0
20:07:21.583 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
20:07:21.583 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:21.583 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:21.583 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data/info.pb
20:07:21.583 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.130
20:07:21.583 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
20:07:21.583 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:21.583 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
20:07:21.583 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.9533
20:07:21.583 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:21.583 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
20:07:21.583 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
20:07:21.583 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:21.584 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.8.230.130
20:07:21.584 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/logs
20:07:21.584 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:21.584 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
20:07:21.585 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
20:07:21.585 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
20:07:21.585 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
20:07:21.585 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 66827b15b158c37363c37baa160f29a0ca1f1e63
20:07:21.585 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type RELEASE
20:07:21.585 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 18 May 2026 19:43:16 UTC on e7f111948823
20:07:21.585 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 11961
20:07:21.585 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.582213  9533 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
20:07:21.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.582595  9533 file_cache.cc:492] Constructed file cache file cache with capacity 419430
20:07:21.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.585984  9541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:21.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.586051  9538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:21.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.585984  9539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:21.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.586637  9533 server_base.cc:1061] running on GCE node
20:07:21.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.586869  9533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
20:07:21.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.587123  9533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
20:07:21.588 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.588374  9533 hybrid_clock.cc:648] HybridClock initialized: now 1779134841588268 us; error 119 us; skew 500 ppm
20:07:21.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.589908  9533 webserver.cc:533] Webserver started at http://127.8.230.130:43383/ using document root <none> and password file <none>
20:07:21.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.590106  9533 fs_manager.cc:362] Metadata directory not provided
20:07:21.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.590144  9533 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
20:07:21.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.590237  9533 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
20:07:21.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.591323  9533 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data/instance:
20:07:21.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "57af19dfdfce4211824a07be0443b43a"
20:07:21.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:21 on dist-test-slave-mzrs"
20:07:21.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.591658  9533 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/wal/instance:
20:07:21.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "57af19dfdfce4211824a07be0443b43a"
20:07:21.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:21 on dist-test-slave-mzrs"
20:07:21.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.592906  9533 fs_manager.cc:696] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
20:07:21.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.593776  9547 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
20:07:21.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.594013  9533 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
20:07:21.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.594080  9533 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data,/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/wal
20:07:21.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "57af19dfdfce4211824a07be0443b43a"
20:07:21.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:21 on dist-test-slave-mzrs"
20:07:21.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.594132  9533 fs_report.cc:389] FS layout report
20:07:21.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
20:07:21.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/wal
20:07:21.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/wal
20:07:21.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data/data
20:07:21.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
20:07:21.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
20:07:21.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
20:07:21.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
20:07:21.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
20:07:21.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
20:07:21.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
20:07:21.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
20:07:21.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
20:07:21.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
20:07:21.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
20:07:21.609 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.608971  9533 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
20:07:21.610 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.609303  9533 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
20:07:21.610 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.609403  9533 kserver.cc:163] Server-wide thread pool size limit: 3276
20:07:21.610 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.609607  9533 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
20:07:21.610 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.609926  9533 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
20:07:21.611 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.609953  9533 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
20:07:21.611 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.609978  9533 ts_tablet_manager.cc:616] Registered 0 tablets
20:07:21.611 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.609992  9533 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
20:07:21.616 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.616243  9533 rpc_server.cc:307] RPC server started. Bound to: 127.8.230.130:37897
20:07:21.616 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.616333  9660 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.230.130:37897 every 8 connection(s)
20:07:21.616 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.616716  9533 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data/info.pb
20:07:21.618 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.617615  9114 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskx5o66C/build/release/bin/kudu as pid 9533
20:07:21.618 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.617703  9114 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/wal/instance
20:07:21.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.619544  9114 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:21.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:21.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/wal
20:07:21.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data
20:07:21.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
20:07:21.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
20:07:21.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
20:07:21.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:21.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:21.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:21.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/logs
20:07:21.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data/info.pb
20:07:21.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:21.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
20:07:21.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
20:07:21.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
20:07:21.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
20:07:21.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:21.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:21.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:21.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
20:07:21.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
20:07:21.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.131:0
20:07:21.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.8.230.131
20:07:21.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.131
20:07:21.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
20:07:21.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:21.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:21.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:21.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:21.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
20:07:21.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.623266  9663 heartbeater.cc:344] Connected to a master server at 127.8.230.189:44351
20:07:21.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.623387  9663 heartbeater.cc:461] Registering TS with master...
20:07:21.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.623553  9663 heartbeater.cc:507] Master 127.8.230.189:44351 requested a full tablet report, sending...
20:07:21.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.623927  9661 heartbeater.cc:344] Connected to a master server at 127.8.230.188:39477
20:07:21.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.624007  9661 heartbeater.cc:461] Registering TS with master...
20:07:21.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.624059  9228 ts_manager.cc:194] Registered new tserver with Master: 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897)
20:07:21.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.624121  9661 heartbeater.cc:507] Master 127.8.230.188:39477 requested a full tablet report, sending...
20:07:21.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.624472  9301 ts_manager.cc:194] Registered new tserver with Master: 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897)
20:07:21.625 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.624905  9301 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.8.230.130:47177
20:07:21.625 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.625753  9662 heartbeater.cc:344] Connected to a master server at 127.8.230.190:34269
20:07:21.626 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.625869  9662 heartbeater.cc:461] Registering TS with master...
20:07:21.626 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.625936  9662 heartbeater.cc:507] Master 127.8.230.190:34269 requested a full tablet report, sending...
20:07:21.626 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.626389  9154 ts_manager.cc:194] Registered new tserver with Master: 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897)
20:07:21.702 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.702378  9668 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
20:07:21.703 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.702594  9668 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
20:07:21.703 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.702617  9668 flags.cc:432] Enabled unsafe flag: --never_fsync=true
20:07:21.704 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.704370  9668 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
20:07:21.704 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.704453  9668 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.230.131
20:07:21.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.706367  9668 tablet_server_runner.cc:78] Tablet server non-default flags:
20:07:21.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:21.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:21.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:21.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
20:07:21.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data
20:07:21.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/wal
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.131:0
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data/info.pb
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.131
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.9668
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.8.230.131
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/logs
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 66827b15b158c37363c37baa160f29a0ca1f1e63
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type RELEASE
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 18 May 2026 19:43:16 UTC on e7f111948823
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 11961
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.706627  9668 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
20:07:21.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.706832  9668 file_cache.cc:492] Constructed file cache file cache with capacity 419430
20:07:21.710 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.709888  9674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:21.710 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.709860  9673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:21.710 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.709859  9668 server_base.cc:1061] running on GCE node
20:07:21.710 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:21.709900  9676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:21.710 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.710347  9668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
20:07:21.710 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.710645  9668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
20:07:21.712 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.711819  9668 hybrid_clock.cc:648] HybridClock initialized: now 1779134841711789 us; error 30 us; skew 500 ppm
20:07:21.713 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.713156  9668 webserver.cc:533] Webserver started at http://127.8.230.131:45551/ using document root <none> and password file <none>
20:07:21.713 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.713423  9668 fs_manager.cc:362] Metadata directory not provided
20:07:21.713 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.713492  9668 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
20:07:21.713 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.713613  9668 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
20:07:21.714 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.714665  9668 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data/instance:
20:07:21.715 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "95c1f61bee2d40849c44869e11134c4a"
20:07:21.715 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:21 on dist-test-slave-mzrs"
20:07:21.715 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.715044  9668 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/wal/instance:
20:07:21.715 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "95c1f61bee2d40849c44869e11134c4a"
20:07:21.715 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:21 on dist-test-slave-mzrs"
20:07:21.716 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.716747  9668 fs_manager.cc:696] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
20:07:21.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.717705  9682 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
20:07:21.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.717917  9668 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
20:07:21.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.717994  9668 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data,/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/wal
20:07:21.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "95c1f61bee2d40849c44869e11134c4a"
20:07:21.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:21 on dist-test-slave-mzrs"
20:07:21.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.718070  9668 fs_report.cc:389] FS layout report
20:07:21.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
20:07:21.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/wal
20:07:21.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/wal
20:07:21.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data/data
20:07:21.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
20:07:21.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
20:07:21.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
20:07:21.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
20:07:21.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
20:07:21.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
20:07:21.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
20:07:21.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
20:07:21.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
20:07:21.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
20:07:21.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
20:07:21.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.737893  9668 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
20:07:21.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.738337  9668 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
20:07:21.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.738476  9668 kserver.cc:163] Server-wide thread pool size limit: 3276
20:07:21.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.738754  9668 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
20:07:21.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.739161  9668 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
20:07:21.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.739228  9668 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
20:07:21.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.739264  9668 ts_tablet_manager.cc:616] Registered 0 tablets
20:07:21.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.739281  9668 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
20:07:21.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.745507  9668 rpc_server.cc:307] RPC server started. Bound to: 127.8.230.131:37545
20:07:21.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.745606  9795 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.230.131:37545 every 8 connection(s)
20:07:21.746 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.745975  9668 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data/info.pb
20:07:21.747 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.746969  9114 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskx5o66C/build/release/bin/kudu as pid 9668
20:07:21.747 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.747071  9114 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/wal/instance
20:07:21.752 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.752192  9796 heartbeater.cc:344] Connected to a master server at 127.8.230.188:39477
20:07:21.752 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.752327  9796 heartbeater.cc:461] Registering TS with master...
20:07:21.752 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.752344  9798 heartbeater.cc:344] Connected to a master server at 127.8.230.189:44351
20:07:21.752 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.752417  9798 heartbeater.cc:461] Registering TS with master...
20:07:21.753 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.752463  9797 heartbeater.cc:344] Connected to a master server at 127.8.230.190:34269
20:07:21.753 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.752504  9797 heartbeater.cc:461] Registering TS with master...
20:07:21.753 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.752576  9797 heartbeater.cc:507] Master 127.8.230.190:34269 requested a full tablet report, sending...
20:07:21.753 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.752552  9796 heartbeater.cc:507] Master 127.8.230.188:39477 requested a full tablet report, sending...
20:07:21.753 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.752552  9798 heartbeater.cc:507] Master 127.8.230.189:44351 requested a full tablet report, sending...
20:07:21.753 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.753113  9228 ts_manager.cc:194] Registered new tserver with Master: 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:21.753 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.753803  9301 ts_manager.cc:194] Registered new tserver with Master: 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:21.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.753824  9154 ts_manager.cc:194] Registered new tserver with Master: 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:21.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.754376  9301 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.8.230.131:58077
20:07:21.762 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:21.762189  9114 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
20:07:21.763 [DEBUG - main] (MiniKuduCluster.java:195) Response: 
20:07:21.764 [DEBUG - main] (MiniKuduCluster.java:185) Request: get_masters {
}

20:07:21.773 [DEBUG - main] (MiniKuduCluster.java:195) Response: get_masters {
  masters {
    id {
      type: MASTER
      index: 0
    }
    bound_rpc_address {
      host: "127.8.230.190"
      port: 34269
    }
    bound_http_address {
      host: "127.8.230.190"
      port: 41449
    }
  }
  masters {
    id {
      type: MASTER
      index: 1
    }
    bound_rpc_address {
      host: "127.8.230.189"
      port: 44351
    }
    bound_http_address {
      host: "127.8.230.189"
      port: 34941
    }
  }
  masters {
    id {
      type: MASTER
      index: 2
    }
    bound_rpc_address {
      host: "127.8.230.188"
      port: 39477
    }
    bound_http_address {
      host: "127.8.230.188"
      port: 34055
    }
  }
}

20:07:21.777 [DEBUG - main] (MiniKuduCluster.java:185) Request: get_tservers {
}

20:07:21.779 [DEBUG - main] (MiniKuduCluster.java:195) Response: get_tservers {
  tservers {
    id {
      type: TSERVER
      index: 0
    }
    bound_rpc_address {
      host: "127.8.230.129"
      port: 42709
    }
    bound_http_address {
      host: "127.8.230.129"
      port: 44243
    }
  }
  tservers {
    id {
      type: TSERVER
      index: 1
    }
    bound_rpc_address {
      host: "127.8.230.130"
      port: 37897
    }
    bound_http_address {
      host: "127.8.230.130"
      port: 43383
    }
  }
  tservers {
    id {
      type: TSERVER
      index: 2
    }
    bound_rpc_address {
      host: "127.8.230.131"
      port: 37545
    }
    bound_http_address {
      host: "127.8.230.131"
      port: 45551
    }
  }
}

20:07:21.780 [INFO - main] (KuduTestHarness.java:157) Creating a new Kudu client...
20:07:22.002 [DEBUG - main] (CompositeStrategy.java:59) Unable to load modern Subject methods, relying only on legacy methods
java.lang.NoSuchMethodException: javax.security.auth.Subject.current()
	at java.base/java.lang.Class.getDeclaredMethod(Class.java:2677)
	at org.apache.kudu.client.internals.ModernStrategy.<init>(ModernStrategy.java:51)
	at org.apache.kudu.client.internals.CompositeStrategy.<init>(CompositeStrategy.java:52)
	at org.apache.kudu.client.internals.CompositeStrategy.<clinit>(CompositeStrategy.java:41)
	at org.apache.kudu.client.internals.SecurityManagerCompatibility.get(SecurityManagerCompatibility.java:43)
	at org.apache.kudu.client.SecurityContext.setupSubject(SecurityContext.java:155)
	at org.apache.kudu.client.SecurityContext.<init>(SecurityContext.java:138)
	at org.apache.kudu.client.AsyncKuduClient.<init>(AsyncKuduClient.java:431)
	at org.apache.kudu.client.AsyncKuduClient$AsyncKuduClientBuilder.build(AsyncKuduClient.java:3109)
	at org.apache.kudu.test.KuduTestHarness.before(KuduTestHarness.java:161)
	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:50)
	at org.apache.kudu.test.junit.RetryRule$RetryStatement.doOneAttemptAndReport(RetryRule.java:184)
	at org.apache.kudu.test.junit.RetryRule$RetryStatement.evaluate(RetryRule.java:244)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:27)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
	at org.junit.runner.JUnitCore.runMain(JUnitCore.java:77)
	at org.junit.runner.JUnitCore.main(JUnitCore.java:36)
20:07:22.066 [DEBUG - main] (SecurityUtil.java:122) Could not login via JAAS. Using no credentials: Unable to obtain Principal Name for authentication 
20:07:22.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.079643  9370 catalog_manager.cc:1294] Loaded cluster ID: 0b06460bc3584fb6b186d8a3926e1792
20:07:22.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.079734  9370 catalog_manager.cc:1587] T 00000000000000000000000000000000 P bac0a7cfdbfb4fb2b6f13da2ed73c097: loading cluster ID for follower catalog manager: success
20:07:22.082 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.081071  9370 catalog_manager.cc:1609] T 00000000000000000000000000000000 P bac0a7cfdbfb4fb2b6f13da2ed73c097: acquiring CA information for follower catalog manager: success
20:07:22.082 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.081444  9370 catalog_manager.cc:1637] T 00000000000000000000000000000000 P bac0a7cfdbfb4fb2b6f13da2ed73c097: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
20:07:22.085 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.083585  9384 catalog_manager.cc:1294] Loaded cluster ID: 0b06460bc3584fb6b186d8a3926e1792
20:07:22.085 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.083678  9384 catalog_manager.cc:1587] T 00000000000000000000000000000000 P d32e19f129bd45eead36995cd0715d70: loading cluster ID for follower catalog manager: success
20:07:22.085 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.084968  9384 catalog_manager.cc:1609] T 00000000000000000000000000000000 P d32e19f129bd45eead36995cd0715d70: acquiring CA information for follower catalog manager: success
20:07:22.085 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.085297  9384 catalog_manager.cc:1637] T 00000000000000000000000000000000 P d32e19f129bd45eead36995cd0715d70: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
20:07:22.231 [DEBUG - Time-limited test] (Connection.java:748) [peer master-127.8.230.189:44351(127.8.230.189:44351)] connecting to peer
20:07:22.285 [DEBUG - Time-limited test] (Connection.java:748) [peer master-127.8.230.190:34269(127.8.230.190:34269)] connecting to peer
20:07:22.287 [DEBUG - Time-limited test] (Connection.java:748) [peer master-127.8.230.188:39477(127.8.230.188:39477)] connecting to peer
20:07:22.312 [DEBUG - kudu-nio-2] (Connection.java:757) [peer master-127.8.230.188:39477(127.8.230.188:39477)] Successfully connected to peer
20:07:22.312 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.8.230.190:34269(127.8.230.190:34269)] Successfully connected to peer
20:07:22.312 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.230.189:44351(127.8.230.189:44351)] Successfully connected to peer
20:07:22.412 [DEBUG - kudu-nio-1] (Negotiator.java:529) SASL mechanism PLAIN chosen for peer 127.8.230.190
20:07:22.414 [DEBUG - kudu-nio-0] (Negotiator.java:529) SASL mechanism PLAIN chosen for peer 127.8.230.189
20:07:22.416 [DEBUG - kudu-nio-2] (Negotiator.java:529) SASL mechanism PLAIN chosen for peer 127.8.230.188
20:07:22.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.496318  9526 heartbeater.cc:499] Master 127.8.230.188:39477 was elected leader, sending a full tablet report...
20:07:22.626 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.625880  9661 heartbeater.cc:499] Master 127.8.230.188:39477 was elected leader, sending a full tablet report...
20:07:22.645 [DEBUG - kudu-nio-1] (Negotiator.java:919) Authenticated connection [id: 0x43f701cc, L:/127.0.0.1:40136 - R:/127.8.230.190:34269] using SASL/PLAIN
20:07:22.645 [DEBUG - kudu-nio-0] (Negotiator.java:919) Authenticated connection [id: 0x035de965, L:/127.0.0.1:57754 - R:/127.8.230.189:44351] using SASL/PLAIN
20:07:22.647 [DEBUG - kudu-nio-2] (Negotiator.java:919) Authenticated connection [id: 0x897d9332, L:/127.0.0.1:44124 - R:/127.8.230.188:39477] using SASL/PLAIN
20:07:22.687 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet Kudu Master for table 'Kudu Master' with partition [<start>, <end>)
20:07:22.688 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=CreateTable, tablet=null, attempt=1, TimeoutTracker(timeout=50000, elapsed=494), Traces: [0ms] refreshing cache from master, [23ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.230.189:44351, [87ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.230.190:34269, [89ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.230.188:39477, [464ms] Sub RPC ConnectToMaster: received response from server master-127.8.230.189:44351: OK, [464ms] Sub RPC ConnectToMaster: received response from server master-127.8.230.188:39477: OK, [464ms] Sub RPC ConnectToMaster: received response from server master-127.8.230.190:34269: OK, deferred=null) after lookup
20:07:22.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.741422  9301 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44124:
20:07:22.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) name: "org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181"
20:07:22.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) schema {
20:07:22.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   columns {
20:07:22.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     name: "key"
20:07:22.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     type: INT32
20:07:22.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_key: true
20:07:22.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_nullable: false
20:07:22.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     cfile_block_size: 0
20:07:22.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     immutable: false
20:07:22.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_auto_incrementing: false
20:07:22.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
20:07:22.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   columns {
20:07:22.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     name: "column1_i"
20:07:22.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     type: INT32
20:07:22.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_key: false
20:07:22.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_nullable: false
20:07:22.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     cfile_block_size: 0
20:07:22.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     immutable: false
20:07:22.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_auto_incrementing: false
20:07:22.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
20:07:22.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   columns {
20:07:22.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     name: "column2_i"
20:07:22.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     type: INT32
20:07:22.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_key: false
20:07:22.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_nullable: false
20:07:22.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     cfile_block_size: 0
20:07:22.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     immutable: false
20:07:22.744 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_auto_incrementing: false
20:07:22.744 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
20:07:22.744 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   columns {
20:07:22.744 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     name: "column3_s"
20:07:22.744 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     type: STRING
20:07:22.744 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_key: false
20:07:22.744 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_nullable: true
20:07:22.744 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     encoding: DICT_ENCODING
20:07:22.744 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     compression: LZ4
20:07:22.744 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     cfile_block_size: 4096
20:07:22.744 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     immutable: false
20:07:22.744 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_auto_incrementing: false
20:07:22.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
20:07:22.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   columns {
20:07:22.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     name: "column4_b"
20:07:22.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     type: BOOL
20:07:22.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_key: false
20:07:22.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_nullable: false
20:07:22.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     cfile_block_size: 0
20:07:22.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     immutable: false
20:07:22.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_auto_incrementing: false
20:07:22.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
20:07:22.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) }
20:07:22.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) partition_schema {
20:07:22.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   range_schema {
20:07:22.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     columns {
20:07:22.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)       name: "key"
20:07:22.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     }
20:07:22.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
20:07:22.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) }
20:07:22.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:22.741797  9301 catalog_manager.cc:7058] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
20:07:22.758 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.758332  9796 heartbeater.cc:499] Master 127.8.230.188:39477 was elected leader, sending a full tablet report...
20:07:22.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.760449  9460 tablet_service.cc:1511] Processing CreateTablet for tablet a4035636acf142eeb16e984e2f510768 (DEFAULT_TABLE table=org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181 [id=69a0f1ecb12842e4ba783598072c0715]), partition=RANGE (key) PARTITION UNBOUNDED
20:07:22.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.760811  9460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4035636acf142eeb16e984e2f510768. 1 dirs total, 0 dirs full, 0 dirs failed
20:07:22.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.764165  9831 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Bootstrap starting.
20:07:22.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.764595  9595 tablet_service.cc:1511] Processing CreateTablet for tablet a4035636acf142eeb16e984e2f510768 (DEFAULT_TABLE table=org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181 [id=69a0f1ecb12842e4ba783598072c0715]), partition=RANGE (key) PARTITION UNBOUNDED
20:07:22.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.764886  9831 tablet_bootstrap.cc:654] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Neither blocks nor log segments found. Creating new log.
20:07:22.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.764892  9595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4035636acf142eeb16e984e2f510768. 1 dirs total, 0 dirs full, 0 dirs failed
20:07:22.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.765180  9831 log.cc:826] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Log is configured to *not* fsync() on all Append() calls
20:07:22.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.765823  9831 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: No bootstrap required, opened a new log
20:07:22.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.765887  9831 ts_tablet_manager.cc:1403] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
20:07:22.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.767705  9831 raft_consensus.cc:359] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:22.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.767886  9831 raft_consensus.cc:385] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
20:07:22.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.767925  9831 raft_consensus.cc:740] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ab092e11f74445fbbe7a4c29511b76b, State: Initialized, Role: FOLLOWER
20:07:22.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.768008  9831 consensus_queue.cc:260] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:22.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.768270  9831 ts_tablet_manager.cc:1434] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.000s
20:07:22.769 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.769778  9833 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a: Bootstrap starting.
20:07:22.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.770780  9833 tablet_bootstrap.cc:654] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a: Neither blocks nor log segments found. Creating new log.
20:07:22.771 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.771341  9833 log.cc:826] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a: Log is configured to *not* fsync() on all Append() calls
20:07:22.775 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.772696  9833 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a: No bootstrap required, opened a new log
20:07:22.775 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.772807  9833 ts_tablet_manager.cc:1403] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a: Time spent bootstrapping tablet: real 0.003s	user 0.002s	sys 0.000s
20:07:22.775 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.774595  9833 raft_consensus.cc:359] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:22.775 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.774783  9833 raft_consensus.cc:385] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
20:07:22.775 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.774818  9833 raft_consensus.cc:740] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57af19dfdfce4211824a07be0443b43a, State: Initialized, Role: FOLLOWER
20:07:22.775 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.774928  9833 consensus_queue.cc:260] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:22.775 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.775192  9833 ts_tablet_manager.cc:1434] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.000s
20:07:22.777 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.776752  9730 tablet_service.cc:1511] Processing CreateTablet for tablet a4035636acf142eeb16e984e2f510768 (DEFAULT_TABLE table=org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181 [id=69a0f1ecb12842e4ba783598072c0715]), partition=RANGE (key) PARTITION UNBOUNDED
20:07:22.777 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.777091  9730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4035636acf142eeb16e984e2f510768. 1 dirs total, 0 dirs full, 0 dirs failed
20:07:22.783 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.783097  9837 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a: Bootstrap starting.
20:07:22.784 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.784359  9837 tablet_bootstrap.cc:654] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a: Neither blocks nor log segments found. Creating new log.
20:07:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.784998  9837 log.cc:826] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a: Log is configured to *not* fsync() on all Append() calls
20:07:22.788 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:894) Opened table 69a0f1ecb12842e4ba783598072c0715
20:07:22.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.795156  9837 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a: No bootstrap required, opened a new log
20:07:22.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.795629  9837 ts_tablet_manager.cc:1403] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a: Time spent bootstrapping tablet: real 0.013s	user 0.002s	sys 0.000s
20:07:22.798 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.798040  9837 raft_consensus.cc:359] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:22.798 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.798600  9837 raft_consensus.cc:385] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
20:07:22.798 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.798749  9837 raft_consensus.cc:740] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95c1f61bee2d40849c44869e11134c4a, State: Initialized, Role: FOLLOWER
20:07:22.799 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.798931  9837 consensus_queue.cc:260] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:22.799 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.799350  9837 ts_tablet_manager.cc:1434] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a: Time spent starting tablet: real 0.003s	user 0.003s	sys 0.000s
20:07:22.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.800252  9834 raft_consensus.cc:493] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
20:07:22.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.800371  9834 raft_consensus.cc:515] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:22.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.800704  9834 leader_election.cc:290] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897), 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:22.806 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.805835  9750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "3ab092e11f74445fbbe7a4c29511b76b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95c1f61bee2d40849c44869e11134c4a" is_pre_election: true
20:07:22.806 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.806033  9750 raft_consensus.cc:2468] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ab092e11f74445fbbe7a4c29511b76b in term 0.
20:07:22.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.806627  9615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "3ab092e11f74445fbbe7a4c29511b76b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57af19dfdfce4211824a07be0443b43a" is_pre_election: true
20:07:22.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.806790  9615 raft_consensus.cc:2468] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ab092e11f74445fbbe7a4c29511b76b in term 0.
20:07:22.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.807147  9414 leader_election.cc:304] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ab092e11f74445fbbe7a4c29511b76b, 57af19dfdfce4211824a07be0443b43a; no voters: 
20:07:22.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.807305  9834 raft_consensus.cc:2804] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 0 FOLLOWER]: Leader pre-election won for term 1
20:07:22.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.807353  9834 raft_consensus.cc:493] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
20:07:22.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.807376  9834 raft_consensus.cc:3060] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 0 FOLLOWER]: Advancing to term 1
20:07:22.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.808297  9834 raft_consensus.cc:515] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:22.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.808460  9834 leader_election.cc:290] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [CANDIDATE]: Term 1 election: Requested vote from peers 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897), 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:22.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.808730  9615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "3ab092e11f74445fbbe7a4c29511b76b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57af19dfdfce4211824a07be0443b43a"
20:07:22.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.808815  9615 raft_consensus.cc:3060] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 0 FOLLOWER]: Advancing to term 1
20:07:22.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.809719  9615 raft_consensus.cc:2468] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab092e11f74445fbbe7a4c29511b76b in term 1.
20:07:22.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.809953  9750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "3ab092e11f74445fbbe7a4c29511b76b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95c1f61bee2d40849c44869e11134c4a"
20:07:22.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.810002  9414 leader_election.cc:304] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ab092e11f74445fbbe7a4c29511b76b, 57af19dfdfce4211824a07be0443b43a; no voters: 
20:07:22.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.810035  9750 raft_consensus.cc:3060] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 0 FOLLOWER]: Advancing to term 1
20:07:22.812 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.810982  9750 raft_consensus.cc:2468] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab092e11f74445fbbe7a4c29511b76b in term 1.
20:07:22.813 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.812160  9834 raft_consensus.cc:2804] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 1 FOLLOWER]: Leader election won for term 1
20:07:22.813 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.812384  9834 raft_consensus.cc:697] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 1 LEADER]: Becoming Leader. State: Replica: 3ab092e11f74445fbbe7a4c29511b76b, State: Running, Role: LEADER
20:07:22.813 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.812515  9834 consensus_queue.cc:237] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:22.814 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.813640  9301 catalog_manager.cc:5696] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129). New cstate: current_term: 1 leader_uuid: "3ab092e11f74445fbbe7a4c29511b76b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } health_report { overall_health: UNKNOWN } } }
20:07:22.824 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:894) Opened table 69a0f1ecb12842e4ba783598072c0715
20:07:22.847 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:22.854 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299993, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)[L],57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:22.856 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=null, attempt=1, sequence_id=-1, TimeoutTracker(timeout=30000, elapsed=20), Traces: [0ms] refreshing cache from master, [0ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [10ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=null) after lookup
20:07:22.866 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:22.869 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:22.868803  9664 tablet.cc:2404] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a: Can't schedule compaction. Clean time has not been advanced past its initial value.
20:07:22.869 [DEBUG - kudu-nio-3] (Connection.java:757) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] Successfully connected to peer
20:07:22.940 [DEBUG - kudu-nio-3] (Negotiator.java:919) Authenticated connection [id: 0x7d1598ca, L:/127.0.0.1:47820 - R:/127.8.230.129:42709] using TOKEN/null
20:07:22.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.944056  9615 raft_consensus.cc:1275] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 1 FOLLOWER]: Refusing update from remote peer 3ab092e11f74445fbbe7a4c29511b76b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
20:07:22.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.944070  9750 raft_consensus.cc:1275] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 1 FOLLOWER]: Refusing update from remote peer 3ab092e11f74445fbbe7a4c29511b76b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
20:07:22.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.944540  9842 consensus_queue.cc:1048] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [LEADER]: Connected to new peer: Peer: permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
20:07:22.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.944687  9834 consensus_queue.cc:1048] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [LEADER]: Connected to new peer: Peer: permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
20:07:22.949 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:22.948920  9847 mvcc.cc:204] Tried to move back new op lower bound from 7287336316695486464 to 7287336316160786432. Current Snapshot: MvccSnapshot[applied={T|T < 7287336316695486464}]
20:07:23.028 [DEBUG - kudu-nio-3] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 3 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181, tablet=a4035636acf142eeb16e984e2f510768, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
20:07:23.030 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.8.230.129:42709
20:07:23.031 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

20:07:23.032 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:23.032016  9114 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskx5o66C/build/release/bin/kudu with pid 9398
20:07:23.038 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
20:07:23.040 [DEBUG - kudu-nio-3] (Connection.java:239) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] handling channelInactive
20:07:23.041 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state READY due to: connection closed
20:07:23.041 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709) for tablet a4035636acf142eeb16e984e2f510768: connection closed
20:07:23.058 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:23.058 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)[L],57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:23.059 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=2, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=21), Traces: [0ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [3ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [17ms] refreshing cache from master, [17ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [19ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@800692891(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@ca4c44f -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2d12ba37 -> wakeup thread Time-limited test)) after lookup
20:07:23.060 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:23.063 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:23.063 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:23.063 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:23.077 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:23.078 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)[L],57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:23.079 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=4, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=40), Traces: [0ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [3ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [17ms] refreshing cache from master, [17ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [19ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [21ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [25ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [25ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [36ms] refreshing cache from master, [37ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [39ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@800692891(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@ca4c44f -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@497638860 after retry RPC@1966733216), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2d12ba37 -> wakeup thread Time-limited test -> (continuation of Deferred@497638860 after retry RPC@1966733216))) after lookup
20:07:23.079 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:23.082 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:23.082 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:23.082 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:23.099 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:23.100 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)[L],57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:23.101 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=6, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=62), Traces: [0ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [3ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [17ms] refreshing cache from master, [17ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [19ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [21ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [25ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [25ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [36ms] refreshing cache from master, [37ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [39ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [41ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [44ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [44ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [57ms] refreshing cache from master, [57ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [61ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@800692891(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@ca4c44f -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@497638860 after retry RPC@1966733216) -> (continuation of Deferred@1395147872 after retry RPC@1644823384), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2d12ba37 -> wakeup thread Time-limited test -> (continuation of Deferred@497638860 after retry RPC@1966733216) -> (continuation of Deferred@1395147872 after retry RPC@1644823384))) after lookup
20:07:23.102 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:23.104 [INFO - kudu-nio-6] (Connection.java:429) Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:23.104 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:23.104 [INFO - kudu-nio-6] (AsyncKuduClient.java:2355) Invalidating location 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:23.218 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:23.219 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)[L],57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:23.219 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=8, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=181), Traces: [0ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [3ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [17ms] refreshing cache from master, [17ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [19ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [21ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [25ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [25ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [36ms] refreshing cache from master, [37ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [39ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [41ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [44ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [44ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [57ms] refreshing cache from master, [57ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [61ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [63ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [66ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [67ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [177ms] refreshing cache from master, [177ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [180ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@800692891(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@ca4c44f -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@497638860 after retry RPC@1966733216) -> (continuation of Deferred@1395147872 after retry RPC@1644823384) -> (continuation of Deferred@362745862 after retry RPC@893163256), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2d12ba37 -> wakeup thread Time-limited test -> (continuation of Deferred@497638860 after retry RPC@1966733216) -> (continuation of Deferred@1395147872 after retry RPC@1644823384) -> (continuation of Deferred@362745862 after retry RPC@893163256))) after lookup
20:07:23.220 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:23.222 [INFO - kudu-nio-7] (Connection.java:429) Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:23.223 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:23.223 [INFO - kudu-nio-7] (AsyncKuduClient.java:2355) Invalidating location 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:23.557 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:23.558 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)[L],57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:23.558 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=10, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=520), Traces: [0ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [3ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [17ms] refreshing cache from master, [17ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [19ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [21ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [25ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [25ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [36ms] refreshing cache from master, [37ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [39ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [41ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [44ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [44ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [57ms] refreshing cache from master, [57ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [61ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [63ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [66ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [67ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [177ms] refreshing cache from master, [177ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [180ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [182ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [185ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [185ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [516ms] refreshing cache from master, [517ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [519ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@800692891(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@ca4c44f -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@497638860 after retry RPC@1966733216) -> (continuation of Deferred@1395147872 after retry RPC@1644823384) -> (continuation of Deferred@362745862 after retry RPC@893163256) -> (continuation of Deferred@1241670716 after retry RPC@1503146418), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2d12ba37 -> wakeup thread Time-limited test -> (continuation of Deferred@497638860 after retry RPC@1966733216) -> (continuation of Deferred@1395147872 after retry RPC@1644823384) -> (continuation of Deferred@362745862 after retry RPC@893163256) -> (continuation of Deferred@1241670716 after retry RPC@1503146418))) after lookup
20:07:23.560 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:23.562 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:23.562 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:23.562 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:24.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.479318  9859 raft_consensus.cc:493] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ab092e11f74445fbbe7a4c29511b76b)
20:07:24.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.479449  9859 raft_consensus.cc:515] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:24.480 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.479518  9860 raft_consensus.cc:493] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ab092e11f74445fbbe7a4c29511b76b)
20:07:24.480 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.479602  9860 raft_consensus.cc:515] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:24.480 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.479777  9859 leader_election.cc:290] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709), 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897)
20:07:24.480 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:24.480206  9683 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.230.129:42709: connect: Connection refused (error 111)
20:07:24.480 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.480417  9860 leader_election.cc:290] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709), 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:24.480 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:24.480703  9548 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.230.129:42709: connect: Connection refused (error 111)
20:07:24.481 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:24.481002  9683 leader_election.cc:336] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709): Network error: Client connection negotiation failed: client connection to 127.8.230.129:42709: connect: Connection refused (error 111)
20:07:24.481 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:24.481433  9548 leader_election.cc:336] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709): Network error: Client connection negotiation failed: client connection to 127.8.230.129:42709: connect: Connection refused (error 111)
20:07:24.483 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.483333  9615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "95c1f61bee2d40849c44869e11134c4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57af19dfdfce4211824a07be0443b43a" is_pre_election: true
20:07:24.483 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.483424  9615 raft_consensus.cc:2468] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95c1f61bee2d40849c44869e11134c4a in term 1.
20:07:24.483 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.483601  9684 leader_election.cc:304] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57af19dfdfce4211824a07be0443b43a, 95c1f61bee2d40849c44869e11134c4a; no voters: 3ab092e11f74445fbbe7a4c29511b76b
20:07:24.483 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.483784  9859 raft_consensus.cc:2804] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 1 FOLLOWER]: Leader pre-election won for term 2
20:07:24.483 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.483822  9859 raft_consensus.cc:493] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3ab092e11f74445fbbe7a4c29511b76b)
20:07:24.484 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.483841  9859 raft_consensus.cc:3060] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 1 FOLLOWER]: Advancing to term 2
20:07:24.484 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.483871  9750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "57af19dfdfce4211824a07be0443b43a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95c1f61bee2d40849c44869e11134c4a" is_pre_election: true
20:07:24.484 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.484452  9859 raft_consensus.cc:515] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:24.484 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.484609  9859 leader_election.cc:290] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [CANDIDATE]: Term 2 election: Requested vote from peers 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709), 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897)
20:07:24.484 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.484604  9750 raft_consensus.cc:2393] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57af19dfdfce4211824a07be0443b43a in current term 2: Already voted for candidate 95c1f61bee2d40849c44869e11134c4a in this term.
20:07:24.485 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.484903  9551 leader_election.cc:304] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57af19dfdfce4211824a07be0443b43a; no voters: 3ab092e11f74445fbbe7a4c29511b76b, 95c1f61bee2d40849c44869e11134c4a
20:07:24.485 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.484963  9615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "95c1f61bee2d40849c44869e11134c4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57af19dfdfce4211824a07be0443b43a"
20:07:24.485 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.485026  9615 raft_consensus.cc:3060] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 1 FOLLOWER]: Advancing to term 2
20:07:24.485 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:24.485648  9683 leader_election.cc:336] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709): Network error: Client connection negotiation failed: client connection to 127.8.230.129:42709: connect: Connection refused (error 111)
20:07:24.485 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.485743  9615 raft_consensus.cc:2468] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95c1f61bee2d40849c44869e11134c4a in term 2.
20:07:24.486 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.485792  9860 raft_consensus.cc:2749] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
20:07:24.486 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.485924  9684 leader_election.cc:304] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57af19dfdfce4211824a07be0443b43a, 95c1f61bee2d40849c44869e11134c4a; no voters: 3ab092e11f74445fbbe7a4c29511b76b
20:07:24.486 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.486035  9859 raft_consensus.cc:2804] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 2 FOLLOWER]: Leader election won for term 2
20:07:24.486 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.486218  9859 raft_consensus.cc:697] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 2 LEADER]: Becoming Leader. State: Replica: 95c1f61bee2d40849c44869e11134c4a, State: Running, Role: LEADER
20:07:24.486 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.486356  9859 consensus_queue.cc:237] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:24.487 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.487098  9301 catalog_manager.cc:5696] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a reported cstate change: term changed from 1 to 2, leader changed from 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129) to 95c1f61bee2d40849c44869e11134c4a (127.8.230.131). New cstate: current_term: 2 leader_uuid: "95c1f61bee2d40849c44869e11134c4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } health_report { overall_health: HEALTHY } } }
20:07:24.899 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:24.898883  9683 consensus_peers.cc:597] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a -> Peer 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709): Couldn't send request to peer 3ab092e11f74445fbbe7a4c29511b76b. Status: Network error: Client connection negotiation failed: client connection to 127.8.230.129:42709: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
20:07:24.923 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.923156  9615 raft_consensus.cc:1275] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 2 FOLLOWER]: Refusing update from remote peer 95c1f61bee2d40849c44869e11134c4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
20:07:24.924 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:24.923516  9859 consensus_queue.cc:1048] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
20:07:25.018 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:25.018 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)[L]]}]
20:07:25.019 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=12, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=1980), Traces: [0ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [3ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [17ms] refreshing cache from master, [17ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [19ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [21ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [25ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [25ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [36ms] refreshing cache from master, [37ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [39ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [41ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [44ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [44ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [57ms] refreshing cache from master, [57ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [61ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [63ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [66ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [67ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [177ms] refreshing cache from master, [177ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [180ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [182ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [185ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [185ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [516ms] refreshing cache from master, [517ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [519ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [522ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [524ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [524ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [1977ms] refreshing cache from master, [1977ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [1979ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@800692891(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@ca4c44f -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@497638860 after retry RPC@1966733216) -> (continuation of Deferred@1395147872 after retry RPC@1644823384) -> (continuation of Deferred@362745862 after retry RPC@893163256) -> (continuation of Deferred@1241670716 after retry RPC@1503146418) -> (continuation of Deferred@2095301305 after retry RPC@1693350198), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2d12ba37 -> wakeup thread Time-limited test -> (continuation of Deferred@497638860 after retry RPC@1966733216) -> (continuation of Deferred@1395147872 after retry RPC@1644823384) -> (continuation of Deferred@362745862 after retry RPC@893163256) -> (continuation of Deferred@1241670716 after retry RPC@1503146418) -> (continuation of Deferred@2095301305 after retry RPC@1693350198))) after lookup
20:07:25.020 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] connecting to peer
20:07:25.021 [DEBUG - kudu-nio-1] (Connection.java:757) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] Successfully connected to peer
20:07:25.040 [DEBUG - kudu-nio-1] (Negotiator.java:919) Authenticated connection [id: 0x53c32acd, L:/127.0.0.1:36690 - R:/127.8.230.131:37545] using TOKEN/null
20:07:25.052 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.8.230.129:42709
20:07:25.053 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

20:07:25.054 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.054225  9114 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:25.054 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:25.054 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal
20:07:25.055 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data
20:07:25.055 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
20:07:25.055 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
20:07:25.055 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
20:07:25.055 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:25.055 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:25.055 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:25.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/logs
20:07:25.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data/info.pb
20:07:25.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:25.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
20:07:25.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
20:07:25.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
20:07:25.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
20:07:25.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:25.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:25.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:25.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
20:07:25.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
20:07:25.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.129:42709
20:07:25.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.8.230.129
20:07:25.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:25.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=44243
20:07:25.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.129
20:07:25.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:25.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:25.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:25.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
20:07:25.143 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:25.142716  9889 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
20:07:25.143 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:25.142935  9889 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
20:07:25.143 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:25.142958  9889 flags.cc:432] Enabled unsafe flag: --never_fsync=true
20:07:25.145 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:25.144780  9889 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
20:07:25.145 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:25.144874  9889 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.230.129
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.146819  9889 tablet_server_runner.cc:78] Tablet server non-default flags:
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.129:42709
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data/info.pb
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.129
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=44243
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.9889
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.8.230.129
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/logs
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 66827b15b158c37363c37baa160f29a0ca1f1e63
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type RELEASE
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 18 May 2026 19:43:16 UTC on e7f111948823
20:07:25.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 11961
20:07:25.148 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.147128  9889 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
20:07:25.148 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.147331  9889 file_cache.cc:492] Constructed file cache file cache with capacity 419430
20:07:25.150 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:25.149842  9898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:25.150 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:25.149835  9896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:25.150 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:25.149894  9895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:25.150 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.150472  9889 server_base.cc:1061] running on GCE node
20:07:25.151 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.150624  9889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
20:07:25.151 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.150842  9889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
20:07:25.152 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.152104  9889 hybrid_clock.cc:648] HybridClock initialized: now 1779134845152069 us; error 47 us; skew 500 ppm
20:07:25.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.153600  9889 webserver.cc:533] Webserver started at http://127.8.230.129:44243/ using document root <none> and password file <none>
20:07:25.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.153878  9889 fs_manager.cc:362] Metadata directory not provided
20:07:25.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.153985  9889 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
20:07:25.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.155687  9889 fs_manager.cc:714] Time spent opening directory manager: real 0.001s	user 0.001s	sys 0.000s
20:07:25.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.156708  9904 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
20:07:25.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.156981  9889 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
20:07:25.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.157063  9889 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data,/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal
20:07:25.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "3ab092e11f74445fbbe7a4c29511b76b"
20:07:25.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:21 on dist-test-slave-mzrs"
20:07:25.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.157508  9889 fs_report.cc:389] FS layout report
20:07:25.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
20:07:25.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal
20:07:25.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal
20:07:25.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data/data
20:07:25.158 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
20:07:25.158 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
20:07:25.158 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
20:07:25.158 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
20:07:25.158 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
20:07:25.158 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
20:07:25.158 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
20:07:25.158 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
20:07:25.158 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
20:07:25.158 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
20:07:25.158 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
20:07:25.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.191788  9889 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
20:07:25.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.192203  9889 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
20:07:25.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.192348  9889 kserver.cc:163] Server-wide thread pool size limit: 3276
20:07:25.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.192622  9889 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
20:07:25.193 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.193218  9911 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
20:07:25.194 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.194347  9889 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
20:07:25.194 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.194432  9889 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
20:07:25.194 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.194473  9889 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
20:07:25.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.195188  9889 ts_tablet_manager.cc:616] Registered 1 tablets
20:07:25.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.195257  9889 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
20:07:25.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.195295  9911 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Bootstrap starting.
20:07:25.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.203380  9889 rpc_server.cc:307] RPC server started. Bound to: 127.8.230.129:42709
20:07:25.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.203477 10018 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.230.129:42709 every 8 connection(s)
20:07:25.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.203933  9889 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data/info.pb
20:07:25.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.204218  9911 log.cc:826] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Log is configured to *not* fsync() on all Append() calls
20:07:25.208 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.207763  9911 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
20:07:25.208 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.208122  9911 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Bootstrap complete.
20:07:25.210 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.209337  9911 ts_tablet_manager.cc:1403] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Time spent bootstrapping tablet: real 0.014s	user 0.009s	sys 0.001s
20:07:25.211 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
20:07:25.214 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.210677  9114 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskx5o66C/build/release/bin/kudu as pid 9889
20:07:25.214 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.210966  9911 raft_consensus.cc:359] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:25.214 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.211311  9911 raft_consensus.cc:740] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ab092e11f74445fbbe7a4c29511b76b, State: Initialized, Role: FOLLOWER
20:07:25.214 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.211468  9911 consensus_queue.cc:260] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:25.214 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.211784  9911 ts_tablet_manager.cc:1434] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.004s
20:07:25.214 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.213330 10021 heartbeater.cc:344] Connected to a master server at 127.8.230.189:44351
20:07:25.214 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.213466 10021 heartbeater.cc:461] Registering TS with master...
20:07:25.216 [DEBUG - kudu-nio-1] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 6 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181, tablet=a4035636acf142eeb16e984e2f510768, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
20:07:25.217 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.8.230.131:37545
20:07:25.217 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 2
  }
}

20:07:25.222 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.218055  9114 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskx5o66C/build/release/bin/kudu with pid 9668
20:07:25.222 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.217509 10021 heartbeater.cc:507] Master 127.8.230.189:44351 requested a full tablet report, sending...
20:07:25.222 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.221277  9227 ts_manager.cc:194] Re-registered known tserver with Master: 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709)
20:07:25.222 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.222527 10019 heartbeater.cc:344] Connected to a master server at 127.8.230.188:39477
20:07:25.222 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.222616 10019 heartbeater.cc:461] Registering TS with master...
20:07:25.222 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.222754 10019 heartbeater.cc:507] Master 127.8.230.188:39477 requested a full tablet report, sending...
20:07:25.223 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.223129  9301 ts_manager.cc:194] Re-registered known tserver with Master: 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709)
20:07:25.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.223683  9301 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.8.230.129:42705
20:07:25.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.224504 10019 heartbeater.cc:499] Master 127.8.230.188:39477 was elected leader, sending a full tablet report...
20:07:25.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.227187 10020 heartbeater.cc:344] Connected to a master server at 127.8.230.190:34269
20:07:25.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.227278 10020 heartbeater.cc:461] Registering TS with master...
20:07:25.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.227339 10020 heartbeater.cc:507] Master 127.8.230.190:34269 requested a full tablet report, sending...
20:07:25.228 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:25.228550  9156 ts_manager.cc:194] Re-registered known tserver with Master: 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709)
20:07:25.232 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
20:07:25.232 [DEBUG - kudu-nio-1] (Connection.java:239) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] handling channelInactive
20:07:25.232 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] cleaning up while in state READY due to: connection closed
20:07:25.233 [DEBUG - Time-limited test] (Connection.java:748) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] connecting to peer
20:07:25.235 [INFO - kudu-nio-2] (Connection.java:429) Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:25.235 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] cleaning up while in state CONNECTING due to: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:25.236 [INFO - kudu-nio-2] (AsyncKuduClient.java:2355) Invalidating location 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:25.258 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:25.259 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)[L]]}]
20:07:25.259 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=2, sequence_id=7, TimeoutTracker(timeout=30000, elapsed=26), Traces: [0ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [3ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [3ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [22ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [25ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@641673414(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@7d6f314c -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@63be6daa -> wakeup thread Time-limited test)) after lookup
20:07:25.261 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] connecting to peer
20:07:25.262 [INFO - kudu-nio-3] (Connection.java:429) Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:25.262 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] cleaning up while in state CONNECTING due to: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:25.263 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:25.279 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:25.279 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)[L]]}]
20:07:25.280 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=4, sequence_id=7, TimeoutTracker(timeout=30000, elapsed=46), Traces: [0ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [3ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [3ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [22ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [25ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [27ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [30ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [30ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [45ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@641673414(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@7d6f314c -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@908593223 after retry RPC@644608021), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@63be6daa -> wakeup thread Time-limited test -> (continuation of Deferred@908593223 after retry RPC@644608021))) after lookup
20:07:25.280 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] connecting to peer
20:07:25.282 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:25.282 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] cleaning up while in state CONNECTING due to: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:25.282 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:25.298 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:25.299 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)[L]]}]
20:07:25.300 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=6, sequence_id=7, TimeoutTracker(timeout=30000, elapsed=67), Traces: [0ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [3ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [3ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [22ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [25ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [27ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [30ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [30ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [45ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [47ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [49ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [49ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [62ms] refreshing cache from master, [62ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [65ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@641673414(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@7d6f314c -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@908593223 after retry RPC@644608021) -> (continuation of Deferred@1012797508 after retry RPC@1171043458), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@63be6daa -> wakeup thread Time-limited test -> (continuation of Deferred@908593223 after retry RPC@644608021) -> (continuation of Deferred@1012797508 after retry RPC@1171043458))) after lookup
20:07:25.301 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] connecting to peer
20:07:25.303 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:25.303 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] cleaning up while in state CONNECTING due to: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:25.303 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:25.437 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:25.438 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)[L]]}]
20:07:25.438 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=8, sequence_id=7, TimeoutTracker(timeout=30000, elapsed=205), Traces: [0ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [3ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [3ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [22ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [25ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [27ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [30ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [30ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [45ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [47ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [49ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [49ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [62ms] refreshing cache from master, [62ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [65ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [68ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [70ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [70ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [201ms] refreshing cache from master, [201ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [204ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@641673414(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@7d6f314c -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@908593223 after retry RPC@644608021) -> (continuation of Deferred@1012797508 after retry RPC@1171043458) -> (continuation of Deferred@1193070671 after retry RPC@1741952138), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@63be6daa -> wakeup thread Time-limited test -> (continuation of Deferred@908593223 after retry RPC@644608021) -> (continuation of Deferred@1012797508 after retry RPC@1171043458) -> (continuation of Deferred@1193070671 after retry RPC@1741952138))) after lookup
20:07:25.439 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] connecting to peer
20:07:25.440 [INFO - kudu-nio-6] (Connection.java:429) Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:25.440 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] cleaning up while in state CONNECTING due to: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:25.441 [INFO - kudu-nio-6] (AsyncKuduClient.java:2355) Invalidating location 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:25.778 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:25.779 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)[L]]}]
20:07:25.779 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=10, sequence_id=7, TimeoutTracker(timeout=30000, elapsed=546), Traces: [0ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [3ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [3ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [22ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [25ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [27ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [30ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [30ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [45ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [47ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [49ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [49ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [62ms] refreshing cache from master, [62ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [65ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [68ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [70ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [70ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [201ms] refreshing cache from master, [201ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [204ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [206ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [208ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [208ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [542ms] refreshing cache from master, [542ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [545ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@641673414(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@7d6f314c -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@908593223 after retry RPC@644608021) -> (continuation of Deferred@1012797508 after retry RPC@1171043458) -> (continuation of Deferred@1193070671 after retry RPC@1741952138) -> (continuation of Deferred@276607623 after retry RPC@1464566854), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@63be6daa -> wakeup thread Time-limited test -> (continuation of Deferred@908593223 after retry RPC@644608021) -> (continuation of Deferred@1012797508 after retry RPC@1171043458) -> (continuation of Deferred@1193070671 after retry RPC@1741952138) -> (continuation of Deferred@276607623 after retry RPC@1464566854))) after lookup
20:07:25.780 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] connecting to peer
20:07:25.782 [INFO - kudu-nio-7] (Connection.java:429) Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:25.782 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] cleaning up while in state CONNECTING due to: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:25.782 [INFO - kudu-nio-7] (AsyncKuduClient.java:2355) Invalidating location 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:26.552 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.551875 10032 raft_consensus.cc:493] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95c1f61bee2d40849c44869e11134c4a)
20:07:26.552 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.551976 10032 raft_consensus.cc:515] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:26.552 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.552153 10032 leader_election.cc:290] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709), 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:26.553 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:26.553053  9551 leader_election.cc:336] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545): Network error: Client connection negotiation failed: client connection to 127.8.230.131:37545: connect: Connection refused (error 111)
20:07:26.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.555663  9973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "57af19dfdfce4211824a07be0443b43a" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: false dest_uuid: "3ab092e11f74445fbbe7a4c29511b76b" is_pre_election: true
20:07:26.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.555830  9973 raft_consensus.cc:2468] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57af19dfdfce4211824a07be0443b43a in term 1.
20:07:26.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.556095  9548 leader_election.cc:304] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ab092e11f74445fbbe7a4c29511b76b, 57af19dfdfce4211824a07be0443b43a; no voters: 95c1f61bee2d40849c44869e11134c4a
20:07:26.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.556216 10032 raft_consensus.cc:2804] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 2 FOLLOWER]: Leader pre-election won for term 3
20:07:26.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.556262 10032 raft_consensus.cc:493] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 95c1f61bee2d40849c44869e11134c4a)
20:07:26.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.556278 10032 raft_consensus.cc:3060] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 2 FOLLOWER]: Advancing to term 3
20:07:26.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.556922 10032 raft_consensus.cc:515] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:26.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.557091 10032 leader_election.cc:290] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [CANDIDATE]: Term 3 election: Requested vote from peers 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709), 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:26.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.557308  9973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "57af19dfdfce4211824a07be0443b43a" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: false dest_uuid: "3ab092e11f74445fbbe7a4c29511b76b"
20:07:26.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.557411  9973 raft_consensus.cc:3060] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 1 FOLLOWER]: Advancing to term 3
20:07:26.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:26.557752  9551 leader_election.cc:336] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545): Network error: Client connection negotiation failed: client connection to 127.8.230.131:37545: connect: Connection refused (error 111)
20:07:26.558 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.558643  9973 raft_consensus.cc:2468] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57af19dfdfce4211824a07be0443b43a in term 3.
20:07:26.559 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.558904  9548 leader_election.cc:304] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ab092e11f74445fbbe7a4c29511b76b, 57af19dfdfce4211824a07be0443b43a; no voters: 95c1f61bee2d40849c44869e11134c4a
20:07:26.559 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.559054 10032 raft_consensus.cc:2804] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 3 FOLLOWER]: Leader election won for term 3
20:07:26.559 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.559247 10032 raft_consensus.cc:697] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 3 LEADER]: Becoming Leader. State: Replica: 57af19dfdfce4211824a07be0443b43a, State: Running, Role: LEADER
20:07:26.559 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.559492 10032 consensus_queue.cc:237] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 8, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:26.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.560302  9301 catalog_manager.cc:5696] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a reported cstate change: term changed from 2 to 3, leader changed from 95c1f61bee2d40849c44869e11134c4a (127.8.230.131) to 57af19dfdfce4211824a07be0443b43a (127.8.230.130). New cstate: current_term: 3 leader_uuid: "57af19dfdfce4211824a07be0443b43a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } health_report { overall_health: UNKNOWN } } }
20:07:26.658 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:26.658 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)[L],95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:26.658 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=12, sequence_id=7, TimeoutTracker(timeout=30000, elapsed=1425), Traces: [0ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [3ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [3ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [22ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [25ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [27ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [30ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [30ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [45ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [47ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [49ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [49ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [62ms] refreshing cache from master, [62ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [65ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [68ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [70ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [70ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [201ms] refreshing cache from master, [201ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [204ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [206ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [208ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [208ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [542ms] refreshing cache from master, [542ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [545ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [547ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [549ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [549ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [1422ms] refreshing cache from master, [1422ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [1424ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@641673414(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@7d6f314c -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@908593223 after retry RPC@644608021) -> (continuation of Deferred@1012797508 after retry RPC@1171043458) -> (continuation of Deferred@1193070671 after retry RPC@1741952138) -> (continuation of Deferred@276607623 after retry RPC@1464566854) -> (continuation of Deferred@1215355677 after retry RPC@1852653378), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@63be6daa -> wakeup thread Time-limited test -> (continuation of Deferred@908593223 after retry RPC@644608021) -> (continuation of Deferred@1012797508 after retry RPC@1171043458) -> (continuation of Deferred@1193070671 after retry RPC@1741952138) -> (continuation of Deferred@276607623 after retry RPC@1464566854) -> (continuation of Deferred@1215355677 after retry RPC@1852653378))) after lookup
20:07:26.660 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] connecting to peer
20:07:26.661 [DEBUG - kudu-nio-0] (Connection.java:757) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] Successfully connected to peer
20:07:26.678 [DEBUG - kudu-nio-0] (Negotiator.java:919) Authenticated connection [id: 0x232fb4c5, L:/127.0.0.1:34622 - R:/127.8.230.130:37897] using TOKEN/null
20:07:26.681 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.680788  9973 raft_consensus.cc:1275] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 3 FOLLOWER]: Refusing update from remote peer 57af19dfdfce4211824a07be0443b43a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 10. (index mismatch)
20:07:26.681 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:26.680984  9551 consensus_peers.cc:597] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a -> Peer 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545): Couldn't send request to peer 95c1f61bee2d40849c44869e11134c4a. Status: Network error: Client connection negotiation failed: client connection to 127.8.230.131:37545: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
20:07:26.681 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.681140 10037 consensus_queue.cc:1048] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 4, Time since last communication: 0.000s
20:07:26.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.682896 10053 mvcc.cc:204] Tried to move back new op lower bound from 7287336325323792384 to 7287336323016773632. Current Snapshot: MvccSnapshot[applied={T|T < 7287336325290299392}]
20:07:26.696 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.8.230.131:37545
20:07:26.697 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 2
  }
}

20:07:26.697 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.697657  9114 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:26.697 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:26.697 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/wal
20:07:26.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data
20:07:26.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
20:07:26.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
20:07:26.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
20:07:26.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:26.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:26.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:26.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/logs
20:07:26.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data/info.pb
20:07:26.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:26.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
20:07:26.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
20:07:26.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
20:07:26.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
20:07:26.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:26.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:26.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:26.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
20:07:26.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
20:07:26.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.131:37545
20:07:26.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.8.230.131
20:07:26.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:26.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=45551
20:07:26.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.131
20:07:26.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:26.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:26.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:26.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
20:07:26.787 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:26.786574 10060 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
20:07:26.787 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:26.786801 10060 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
20:07:26.787 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:26.786824 10060 flags.cc:432] Enabled unsafe flag: --never_fsync=true
20:07:26.788 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:26.788662 10060 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
20:07:26.788 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:26.788753 10060 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.230.131
20:07:26.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.790616 10060 tablet_server_runner.cc:78] Tablet server non-default flags:
20:07:26.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:26.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:26.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:26.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
20:07:26.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data
20:07:26.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/wal
20:07:26.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:26.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:26.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.131:37545
20:07:26.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
20:07:26.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:26.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:26.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data/info.pb
20:07:26.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.131
20:07:26.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=45551
20:07:26.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:26.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
20:07:26.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.10060
20:07:26.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:26.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
20:07:26.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
20:07:26.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:26.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.8.230.131
20:07:26.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/logs
20:07:26.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:26.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
20:07:26.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
20:07:26.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
20:07:26.792 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
20:07:26.792 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 66827b15b158c37363c37baa160f29a0ca1f1e63
20:07:26.792 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type RELEASE
20:07:26.792 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 18 May 2026 19:43:16 UTC on e7f111948823
20:07:26.792 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 11961
20:07:26.792 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.790973 10060 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
20:07:26.792 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.791220 10060 file_cache.cc:492] Constructed file cache file cache with capacity 419430
20:07:26.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:26.793996 10066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:26.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:26.794006 10068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:26.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:26.794023 10065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:26.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.794410 10060 server_base.cc:1061] running on GCE node
20:07:26.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.794584 10060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
20:07:26.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.794847 10060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
20:07:26.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.796031 10060 hybrid_clock.cc:648] HybridClock initialized: now 1779134846795996 us; error 52 us; skew 500 ppm
20:07:26.797 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.797504 10060 webserver.cc:533] Webserver started at http://127.8.230.131:45551/ using document root <none> and password file <none>
20:07:26.797 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.797785 10060 fs_manager.cc:362] Metadata directory not provided
20:07:26.798 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.797909 10060 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
20:07:26.799 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.799424 10060 fs_manager.cc:714] Time spent opening directory manager: real 0.001s	user 0.001s	sys 0.000s
20:07:26.800 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.800354 10074 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
20:07:26.800 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.800595 10060 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
20:07:26.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.800671 10060 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data,/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/wal
20:07:26.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "95c1f61bee2d40849c44869e11134c4a"
20:07:26.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:21 on dist-test-slave-mzrs"
20:07:26.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.801039 10060 fs_report.cc:389] FS layout report
20:07:26.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
20:07:26.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/wal
20:07:26.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/wal
20:07:26.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data/data
20:07:26.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
20:07:26.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
20:07:26.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
20:07:26.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
20:07:26.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
20:07:26.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
20:07:26.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
20:07:26.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
20:07:26.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
20:07:26.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
20:07:26.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
20:07:26.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.810972 10060 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
20:07:26.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.811379 10060 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
20:07:26.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.811520 10060 kserver.cc:163] Server-wide thread pool size limit: 3276
20:07:26.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.811792 10060 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
20:07:26.812 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.812359 10081 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
20:07:26.813 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.813241 10060 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
20:07:26.813 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.813316 10060 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
20:07:26.813 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.813355 10060 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
20:07:26.814 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.813982 10060 ts_tablet_manager.cc:616] Registered 1 tablets
20:07:26.814 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.814043 10060 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
20:07:26.814 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.814076 10081 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a: Bootstrap starting.
20:07:26.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.822306 10060 rpc_server.cc:307] RPC server started. Bound to: 127.8.230.131:37545
20:07:26.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.823205 10060 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data/info.pb
20:07:26.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.824499 10081 log.cc:826] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a: Log is configured to *not* fsync() on all Append() calls
20:07:26.825 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.825347 10188 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.230.131:37545 every 8 connection(s)
20:07:26.827 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.827744  9114 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskx5o66C/build/release/bin/kudu as pid 10060
20:07:26.828 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
20:07:26.831 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.830859 10081 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=6 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
20:07:26.831 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.831507 10081 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a: Bootstrap complete.
20:07:26.832 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.832825 10081 ts_tablet_manager.cc:1403] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a: Time spent bootstrapping tablet: real 0.019s	user 0.007s	sys 0.008s
20:07:26.834 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.834432 10081 raft_consensus.cc:359] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:26.835 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.835008 10081 raft_consensus.cc:740] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95c1f61bee2d40849c44869e11134c4a, State: Initialized, Role: FOLLOWER
20:07:26.835 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.835807 10081 consensus_queue.cc:260] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 2.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:26.836 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.836316 10081 ts_tablet_manager.cc:1434] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a: Time spent starting tablet: real 0.003s	user 0.002s	sys 0.000s
20:07:26.837 [DEBUG - kudu-nio-0] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 9 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181, tablet=a4035636acf142eeb16e984e2f510768, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
20:07:26.837 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.8.230.130:37897
20:07:26.838 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 1
  }
}

20:07:26.843 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.838423  9114 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskx5o66C/build/release/bin/kudu with pid 9533
20:07:26.843 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.839864 10189 heartbeater.cc:344] Connected to a master server at 127.8.230.188:39477
20:07:26.843 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.839998 10189 heartbeater.cc:461] Registering TS with master...
20:07:26.843 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.840253 10189 heartbeater.cc:507] Master 127.8.230.188:39477 requested a full tablet report, sending...
20:07:26.844 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.840601 10190 heartbeater.cc:344] Connected to a master server at 127.8.230.190:34269
20:07:26.844 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.840648 10190 heartbeater.cc:461] Registering TS with master...
20:07:26.844 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.840767 10190 heartbeater.cc:507] Master 127.8.230.190:34269 requested a full tablet report, sending...
20:07:26.844 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.840932  9300 ts_manager.cc:194] Re-registered known tserver with Master: 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:26.844 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.841132  9156 ts_manager.cc:194] Re-registered known tserver with Master: 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:26.844 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.841539  9300 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.8.230.131:49341
20:07:26.844 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.842365 10189 heartbeater.cc:499] Master 127.8.230.188:39477 was elected leader, sending a full tablet report...
20:07:26.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.845065 10191 heartbeater.cc:344] Connected to a master server at 127.8.230.189:44351
20:07:26.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.845165 10191 heartbeater.cc:461] Registering TS with master...
20:07:26.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.845220 10191 heartbeater.cc:507] Master 127.8.230.189:44351 requested a full tablet report, sending...
20:07:26.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:26.845818  9228 ts_manager.cc:194] Re-registered known tserver with Master: 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:26.847 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
20:07:26.847 [DEBUG - kudu-nio-0] (Connection.java:239) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] handling channelInactive
20:07:26.847 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] cleaning up while in state READY due to: connection closed
20:07:26.855 [DEBUG - pool-2-thread-1] (Connection.java:748) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] connecting to peer
20:07:26.857 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:26.857 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] cleaning up while in state CONNECTING due to: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:26.858 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:26.878 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:26.878 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)[L],95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:26.878 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=3, sequence_id=10, TimeoutTracker(timeout=30000, elapsed=31), Traces: [0ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [1ms] delaying RPC due to: Illegal state: connection is terminated, [8ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [11ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [11ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [30ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@1035145324(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@3aa3bfb7 -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1e8ced1e -> wakeup thread Time-limited test)) after lookup
20:07:26.879 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] connecting to peer
20:07:26.880 [INFO - kudu-nio-2] (Connection.java:429) Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:26.881 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] cleaning up while in state CONNECTING due to: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:26.881 [INFO - kudu-nio-2] (AsyncKuduClient.java:2355) Invalidating location 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:26.898 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:26.899 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)[L],95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:26.899 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=5, sequence_id=10, TimeoutTracker(timeout=30000, elapsed=52), Traces: [0ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [1ms] delaying RPC due to: Illegal state: connection is terminated, [8ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [11ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [11ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [30ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [32ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [34ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [34ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [47ms] refreshing cache from master, [48ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [51ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@1035145324(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@3aa3bfb7 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@497545548 after retry RPC@133728083), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1e8ced1e -> wakeup thread Time-limited test -> (continuation of Deferred@497545548 after retry RPC@133728083))) after lookup
20:07:26.900 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] connecting to peer
20:07:26.901 [INFO - kudu-nio-3] (Connection.java:429) Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:26.901 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] cleaning up while in state CONNECTING due to: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:26.901 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:26.978 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:26.978 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)[L],95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:26.978 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=7, sequence_id=10, TimeoutTracker(timeout=30000, elapsed=131), Traces: [0ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [1ms] delaying RPC due to: Illegal state: connection is terminated, [8ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [11ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [11ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [30ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [32ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [34ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [34ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [47ms] refreshing cache from master, [48ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [51ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [53ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [54ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [54ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [128ms] refreshing cache from master, [128ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [130ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@1035145324(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@3aa3bfb7 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@497545548 after retry RPC@133728083) -> (continuation of Deferred@383631731 after retry RPC@1440489188), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1e8ced1e -> wakeup thread Time-limited test -> (continuation of Deferred@497545548 after retry RPC@133728083) -> (continuation of Deferred@383631731 after retry RPC@1440489188))) after lookup
20:07:26.979 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] connecting to peer
20:07:26.980 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:26.981 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] cleaning up while in state CONNECTING due to: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:26.981 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:27.158 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:27.158 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)[L],95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:27.159 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=9, sequence_id=10, TimeoutTracker(timeout=30000, elapsed=311), Traces: [0ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [1ms] delaying RPC due to: Illegal state: connection is terminated, [8ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [11ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [11ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [30ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [32ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [34ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [34ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [47ms] refreshing cache from master, [48ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [51ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [53ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [54ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [54ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [128ms] refreshing cache from master, [128ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [130ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [132ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [134ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [134ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [308ms] refreshing cache from master, [308ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [310ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@1035145324(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@3aa3bfb7 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@497545548 after retry RPC@133728083) -> (continuation of Deferred@383631731 after retry RPC@1440489188) -> (continuation of Deferred@1569058302 after retry RPC@1755050900), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1e8ced1e -> wakeup thread Time-limited test -> (continuation of Deferred@497545548 after retry RPC@133728083) -> (continuation of Deferred@383631731 after retry RPC@1440489188) -> (continuation of Deferred@1569058302 after retry RPC@1755050900))) after lookup
20:07:27.159 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] connecting to peer
20:07:27.160 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:27.160 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] cleaning up while in state CONNECTING due to: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:27.160 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:27.578 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:27.578 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)[L],95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:27.579 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=11, sequence_id=10, TimeoutTracker(timeout=30000, elapsed=731), Traces: [0ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [1ms] delaying RPC due to: Illegal state: connection is terminated, [8ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [11ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [11ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [30ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [32ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [34ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [34ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [47ms] refreshing cache from master, [48ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [51ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [53ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [54ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [54ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [128ms] refreshing cache from master, [128ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [130ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [132ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [134ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [134ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [308ms] refreshing cache from master, [308ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [310ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [312ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [313ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [314ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [728ms] refreshing cache from master, [728ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [731ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@1035145324(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@3aa3bfb7 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@497545548 after retry RPC@133728083) -> (continuation of Deferred@383631731 after retry RPC@1440489188) -> (continuation of Deferred@1569058302 after retry RPC@1755050900) -> (continuation of Deferred@391760372 after retry RPC@630411916), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1e8ced1e -> wakeup thread Time-limited test -> (continuation of Deferred@497545548 after retry RPC@133728083) -> (continuation of Deferred@383631731 after retry RPC@1440489188) -> (continuation of Deferred@1569058302 after retry RPC@1755050900) -> (continuation of Deferred@391760372 after retry RPC@630411916))) after lookup
20:07:27.580 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] connecting to peer
20:07:27.581 [INFO - kudu-nio-6] (Connection.java:429) Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:27.581 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] cleaning up while in state CONNECTING due to: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:27.581 [INFO - kudu-nio-6] (AsyncKuduClient.java:2355) Invalidating location 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:28.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:28.196485 10203 raft_consensus.cc:493] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57af19dfdfce4211824a07be0443b43a)
20:07:28.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:28.196595 10203 raft_consensus.cc:515] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:28.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:28.196951 10203 leader_election.cc:290] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897), 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:28.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:28.197521  9906 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.230.130:37897: connect: Connection refused (error 111)
20:07:28.198 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:28.198060  9906 leader_election.cc:336] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897): Network error: Client connection negotiation failed: client connection to 127.8.230.130:37897: connect: Connection refused (error 111)
20:07:28.201 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:28.201315 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "3ab092e11f74445fbbe7a4c29511b76b" candidate_term: 4 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: false dest_uuid: "95c1f61bee2d40849c44869e11134c4a" is_pre_election: true
20:07:28.201 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:28.201498 10128 raft_consensus.cc:2468] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ab092e11f74445fbbe7a4c29511b76b in term 2.
20:07:28.201 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:28.201849  9908 leader_election.cc:304] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ab092e11f74445fbbe7a4c29511b76b, 95c1f61bee2d40849c44869e11134c4a; no voters: 57af19dfdfce4211824a07be0443b43a
20:07:28.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:28.202028 10203 raft_consensus.cc:2804] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 3 FOLLOWER]: Leader pre-election won for term 4
20:07:28.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:28.202135 10203 raft_consensus.cc:493] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 57af19dfdfce4211824a07be0443b43a)
20:07:28.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:28.202163 10203 raft_consensus.cc:3060] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 3 FOLLOWER]: Advancing to term 4
20:07:28.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:28.202900 10203 raft_consensus.cc:515] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:28.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:28.203096 10203 leader_election.cc:290] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [CANDIDATE]: Term 4 election: Requested vote from peers 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897), 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:28.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:28.203454 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "3ab092e11f74445fbbe7a4c29511b76b" candidate_term: 4 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: false dest_uuid: "95c1f61bee2d40849c44869e11134c4a"
20:07:28.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:28.203549 10128 raft_consensus.cc:3060] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 2 FOLLOWER]: Advancing to term 4
20:07:28.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:28.204376  9906 leader_election.cc:336] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897): Network error: Client connection negotiation failed: client connection to 127.8.230.130:37897: connect: Connection refused (error 111)
20:07:28.205 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:28.204941 10128 raft_consensus.cc:2468] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab092e11f74445fbbe7a4c29511b76b in term 4.
20:07:28.205 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:28.205215  9908 leader_election.cc:304] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ab092e11f74445fbbe7a4c29511b76b, 95c1f61bee2d40849c44869e11134c4a; no voters: 57af19dfdfce4211824a07be0443b43a
20:07:28.205 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:28.205348 10203 raft_consensus.cc:2804] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 4 FOLLOWER]: Leader election won for term 4
20:07:28.205 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:28.205513 10203 raft_consensus.cc:697] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 4 LEADER]: Becoming Leader. State: Replica: 3ab092e11f74445fbbe7a4c29511b76b, State: Running, Role: LEADER
20:07:28.205 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:28.205614 10203 consensus_queue.cc:237] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 3.12, Last appended by leader: 12, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:28.206 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:28.206393  9300 catalog_manager.cc:5696] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b reported cstate change: term changed from 3 to 4, leader changed from 57af19dfdfce4211824a07be0443b43a (127.8.230.130) to 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129). New cstate: current_term: 4 leader_uuid: "3ab092e11f74445fbbe7a4c29511b76b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } health_report { overall_health: UNKNOWN } } }
20:07:28.629 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:28.629529 10128 raft_consensus.cc:1275] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 4 FOLLOWER]: Refusing update from remote peer 3ab092e11f74445fbbe7a4c29511b76b: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 4 index: 13. (index mismatch)
20:07:28.630 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:28.629952 10203 consensus_queue.cc:1048] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [LEADER]: Connected to new peer: Peer: permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 8, Time since last communication: 0.000s
20:07:28.633 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:28.633162  9906 consensus_peers.cc:597] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b -> Peer 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897): Couldn't send request to peer 57af19dfdfce4211824a07be0443b43a. Status: Network error: Client connection negotiation failed: client connection to 127.8.230.130:37897: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
20:07:29.717 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:29.717 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)[L],57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:29.718 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=13, sequence_id=10, TimeoutTracker(timeout=30000, elapsed=2870), Traces: [0ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [1ms] delaying RPC due to: Illegal state: connection is terminated, [8ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [11ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [11ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [30ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [32ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [34ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [34ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [47ms] refreshing cache from master, [48ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [51ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [53ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [54ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [54ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [128ms] refreshing cache from master, [128ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [130ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [132ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [134ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [134ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [308ms] refreshing cache from master, [308ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [310ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [312ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [313ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [314ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [728ms] refreshing cache from master, [728ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [731ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [732ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [734ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [734ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [2867ms] refreshing cache from master, [2868ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [2870ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@1035145324(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@3aa3bfb7 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@497545548 after retry RPC@133728083) -> (continuation of Deferred@383631731 after retry RPC@1440489188) -> (continuation of Deferred@1569058302 after retry RPC@1755050900) -> (continuation of Deferred@391760372 after retry RPC@630411916) -> (continuation of Deferred@1235282263 after retry RPC@1981339101), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1e8ced1e -> wakeup thread Time-limited test -> (continuation of Deferred@497545548 after retry RPC@133728083) -> (continuation of Deferred@383631731 after retry RPC@1440489188) -> (continuation of Deferred@1569058302 after retry RPC@1755050900) -> (continuation of Deferred@391760372 after retry RPC@630411916) -> (continuation of Deferred@1235282263 after retry RPC@1981339101))) after lookup
20:07:29.719 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:29.720 [DEBUG - kudu-nio-7] (Connection.java:757) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] Successfully connected to peer
20:07:29.747 [DEBUG - kudu-nio-7] (Negotiator.java:919) Authenticated connection [id: 0xc083ad67, L:/127.0.0.1:47872 - R:/127.8.230.129:42709] using TOKEN/null
20:07:29.765 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.8.230.130:37897
20:07:29.765 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 1
  }
}

20:07:29.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.766119  9114 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:29.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:29.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/wal
20:07:29.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data
20:07:29.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
20:07:29.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
20:07:29.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
20:07:29.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:29.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:29.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:29.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/logs
20:07:29.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data/info.pb
20:07:29.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:29.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
20:07:29.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
20:07:29.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
20:07:29.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
20:07:29.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:29.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:29.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:29.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
20:07:29.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
20:07:29.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.130:37897
20:07:29.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.8.230.130
20:07:29.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:29.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=43383
20:07:29.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.130
20:07:29.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:29.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:29.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:29.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
20:07:29.852 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:29.852085 10238 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
20:07:29.852 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:29.852305 10238 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
20:07:29.852 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:29.852329 10238 flags.cc:432] Enabled unsafe flag: --never_fsync=true
20:07:29.854 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:29.853936 10238 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
20:07:29.854 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:29.854012 10238 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.230.130
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.855835 10238 tablet_server_runner.cc:78] Tablet server non-default flags:
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/wal
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.130:37897
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data/info.pb
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.130
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=43383
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.10238
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.8.230.130
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/logs
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
20:07:29.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 66827b15b158c37363c37baa160f29a0ca1f1e63
20:07:29.857 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type RELEASE
20:07:29.857 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 18 May 2026 19:43:16 UTC on e7f111948823
20:07:29.857 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 11961
20:07:29.857 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.856102 10238 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
20:07:29.857 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.856298 10238 file_cache.cc:492] Constructed file cache file cache with capacity 419430
20:07:29.859 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:29.859057 10246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:29.859 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:29.859053 10244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:29.859 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:29.859061 10243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:29.859 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.859323 10238 server_base.cc:1061] running on GCE node
20:07:29.859 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.859467 10238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
20:07:29.859 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.859650 10238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
20:07:29.860 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.860790 10238 hybrid_clock.cc:648] HybridClock initialized: now 1779134849860735 us; error 73 us; skew 500 ppm
20:07:29.862 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.862126 10238 webserver.cc:533] Webserver started at http://127.8.230.130:43383/ using document root <none> and password file <none>
20:07:29.862 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.862398 10238 fs_manager.cc:362] Metadata directory not provided
20:07:29.862 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.862465 10238 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
20:07:29.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.863698 10238 fs_manager.cc:714] Time spent opening directory manager: real 0.001s	user 0.001s	sys 0.000s
20:07:29.864 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.864339 10252 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
20:07:29.864 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.864516 10238 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
20:07:29.865 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.864588 10238 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data,/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/wal
20:07:29.865 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "57af19dfdfce4211824a07be0443b43a"
20:07:29.865 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:21 on dist-test-slave-mzrs"
20:07:29.865 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.864939 10238 fs_report.cc:389] FS layout report
20:07:29.865 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
20:07:29.865 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/wal
20:07:29.865 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/wal
20:07:29.865 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data/data
20:07:29.865 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
20:07:29.865 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
20:07:29.865 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
20:07:29.865 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
20:07:29.865 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
20:07:29.865 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
20:07:29.866 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
20:07:29.866 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
20:07:29.866 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
20:07:29.866 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
20:07:29.866 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
20:07:29.887 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.887665 10238 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
20:07:29.888 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.888105 10238 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
20:07:29.888 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.888270 10238 kserver.cc:163] Server-wide thread pool size limit: 3276
20:07:29.888 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.888540 10238 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
20:07:29.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.889076 10259 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
20:07:29.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.889969 10238 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
20:07:29.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.890041 10238 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
20:07:29.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.890080 10238 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
20:07:29.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.890751 10238 ts_tablet_manager.cc:616] Registered 1 tablets
20:07:29.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.890813 10238 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
20:07:29.891 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.890843 10259 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a: Bootstrap starting.
20:07:29.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.898382 10238 rpc_server.cc:307] RPC server started. Bound to: 127.8.230.130:37897
20:07:29.899 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.898780 10366 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.230.130:37897 every 8 connection(s)
20:07:29.899 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.898901 10238 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data/info.pb
20:07:29.900 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.900286 10259 log.cc:826] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a: Log is configured to *not* fsync() on all Append() calls
20:07:29.901 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.901698  9114 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskx5o66C/build/release/bin/kudu as pid 10238
20:07:29.902 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
20:07:29.904 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.903513 10259 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=9 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
20:07:29.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.905941 10259 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a: Bootstrap complete.
20:07:29.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.906759 10367 heartbeater.cc:344] Connected to a master server at 127.8.230.188:39477
20:07:29.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.906869 10367 heartbeater.cc:461] Registering TS with master...
20:07:29.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.907084 10367 heartbeater.cc:507] Master 127.8.230.188:39477 requested a full tablet report, sending...
20:07:29.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.907258 10259 ts_tablet_manager.cc:1403] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a: Time spent bootstrapping tablet: real 0.016s	user 0.006s	sys 0.007s
20:07:29.907 [DEBUG - kudu-nio-7] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 12 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181, tablet=a4035636acf142eeb16e984e2f510768, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
20:07:29.908 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.907829  9300 ts_manager.cc:194] Re-registered known tserver with Master: 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897)
20:07:29.908 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.908371  9300 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.8.230.130:41565
20:07:29.908 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.908537 10259 raft_consensus.cc:359] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:29.909 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.908830 10259 raft_consensus.cc:740] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57af19dfdfce4211824a07be0443b43a, State: Initialized, Role: FOLLOWER
20:07:29.909 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.909072 10259 consensus_queue.cc:260] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 3.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:29.909 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.909403 10259 ts_tablet_manager.cc:1434] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a: Time spent starting tablet: real 0.002s	user 0.001s	sys 0.001s
20:07:29.909 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.909528 10367 heartbeater.cc:499] Master 127.8.230.188:39477 was elected leader, sending a full tablet report...
20:07:29.909 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.909782 10368 heartbeater.cc:344] Connected to a master server at 127.8.230.190:34269
20:07:29.909 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.909842 10368 heartbeater.cc:461] Registering TS with master...
20:07:29.910 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.909893 10368 heartbeater.cc:507] Master 127.8.230.190:34269 requested a full tablet report, sending...
20:07:29.911 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.910918 10369 heartbeater.cc:344] Connected to a master server at 127.8.230.189:44351
20:07:29.911 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.910991 10369 heartbeater.cc:461] Registering TS with master...
20:07:29.911 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.911041 10369 heartbeater.cc:507] Master 127.8.230.189:44351 requested a full tablet report, sending...
20:07:29.911 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.911371  9227 ts_manager.cc:194] Re-registered known tserver with Master: 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897)
20:07:29.915 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.914584  9156 ts_manager.cc:194] Re-registered known tserver with Master: 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897)
20:07:29.916 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.8.230.129:42709
20:07:29.917 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

20:07:29.919 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:29.917511  9114 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskx5o66C/build/release/bin/kudu with pid 9889
20:07:29.924 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
20:07:29.924 [DEBUG - kudu-nio-7] (Connection.java:239) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] handling channelInactive
20:07:29.924 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state READY due to: connection closed
20:07:29.935 [DEBUG - pool-2-thread-1] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:29.937 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:29.937 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:29.937 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:29.957 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:29.958 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)[L],57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:29.958 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=3, sequence_id=13, TimeoutTracker(timeout=30000, elapsed=33), Traces: [0ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [1ms] delaying RPC due to: Illegal state: connection is terminated, [11ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [13ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [13ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [31ms] refreshing cache from master, [31ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [33ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@1142168194(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@749289e0 -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@3c005c98 -> wakeup thread Time-limited test)) after lookup
20:07:29.959 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:29.960 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:29.960 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:29.960 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:29.977 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:29.978 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)[L],57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:29.978 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=5, sequence_id=13, TimeoutTracker(timeout=30000, elapsed=53), Traces: [0ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [1ms] delaying RPC due to: Illegal state: connection is terminated, [11ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [13ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [13ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [31ms] refreshing cache from master, [31ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [33ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [34ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [36ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [36ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [51ms] refreshing cache from master, [51ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [53ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@1142168194(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@749289e0 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@924897183 after retry RPC@159031679), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@3c005c98 -> wakeup thread Time-limited test -> (continuation of Deferred@924897183 after retry RPC@159031679))) after lookup
20:07:29.979 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:29.980 [INFO - kudu-nio-2] (Connection.java:429) Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:29.980 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:29.980 [INFO - kudu-nio-2] (AsyncKuduClient.java:2355) Invalidating location 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:30.017 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:30.017 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)[L],57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:30.017 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=7, sequence_id=13, TimeoutTracker(timeout=30000, elapsed=93), Traces: [0ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [1ms] delaying RPC due to: Illegal state: connection is terminated, [11ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [13ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [13ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [31ms] refreshing cache from master, [31ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [33ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [34ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [36ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [36ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [51ms] refreshing cache from master, [51ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [53ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [54ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [56ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [56ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [90ms] refreshing cache from master, [91ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [93ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@1142168194(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@749289e0 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@924897183 after retry RPC@159031679) -> (continuation of Deferred@2112410369 after retry RPC@1816837884), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@3c005c98 -> wakeup thread Time-limited test -> (continuation of Deferred@924897183 after retry RPC@159031679) -> (continuation of Deferred@2112410369 after retry RPC@1816837884))) after lookup
20:07:30.018 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:30.019 [INFO - kudu-nio-3] (Connection.java:429) Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:30.019 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:30.019 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:30.177 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:30.177 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)[L],57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:30.178 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=9, sequence_id=13, TimeoutTracker(timeout=30000, elapsed=253), Traces: [0ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [1ms] delaying RPC due to: Illegal state: connection is terminated, [11ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [13ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [13ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [31ms] refreshing cache from master, [31ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [33ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [34ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [36ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [36ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [51ms] refreshing cache from master, [51ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [53ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [54ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [56ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [56ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [90ms] refreshing cache from master, [91ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [93ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [94ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [95ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [95ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [250ms] refreshing cache from master, [250ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [252ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@1142168194(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@749289e0 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@924897183 after retry RPC@159031679) -> (continuation of Deferred@2112410369 after retry RPC@1816837884) -> (continuation of Deferred@1646429832 after retry RPC@1537632622), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@3c005c98 -> wakeup thread Time-limited test -> (continuation of Deferred@924897183 after retry RPC@159031679) -> (continuation of Deferred@2112410369 after retry RPC@1816837884) -> (continuation of Deferred@1646429832 after retry RPC@1537632622))) after lookup
20:07:30.178 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:30.179 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:30.180 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:30.180 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:30.457 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:30.458 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)[L],57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:30.458 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=11, sequence_id=13, TimeoutTracker(timeout=30000, elapsed=533), Traces: [0ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [1ms] delaying RPC due to: Illegal state: connection is terminated, [11ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [13ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [13ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [31ms] refreshing cache from master, [31ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [33ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [34ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [36ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [36ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [51ms] refreshing cache from master, [51ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [53ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [54ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [56ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [56ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [90ms] refreshing cache from master, [91ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [93ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [94ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [95ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [95ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [250ms] refreshing cache from master, [250ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [252ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [254ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [256ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [256ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [531ms] refreshing cache from master, [531ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [533ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@1142168194(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@749289e0 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@924897183 after retry RPC@159031679) -> (continuation of Deferred@2112410369 after retry RPC@1816837884) -> (continuation of Deferred@1646429832 after retry RPC@1537632622) -> (continuation of Deferred@1370249444 after retry RPC@959035046), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@3c005c98 -> wakeup thread Time-limited test -> (continuation of Deferred@924897183 after retry RPC@159031679) -> (continuation of Deferred@2112410369 after retry RPC@1816837884) -> (continuation of Deferred@1646429832 after retry RPC@1537632622) -> (continuation of Deferred@1370249444 after retry RPC@959035046))) after lookup
20:07:30.459 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:30.460 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:30.460 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:30.460 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:31.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.197232 10381 raft_consensus.cc:493] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
20:07:31.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.197420 10381 raft_consensus.cc:515] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:31.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.197839 10381 leader_election.cc:290] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709), 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:31.198 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:31.198328 10253 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.230.129:42709: connect: Connection refused (error 111)
20:07:31.199 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:31.198993 10253 leader_election.cc:336] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709): Network error: Client connection negotiation failed: client connection to 127.8.230.129:42709: connect: Connection refused (error 111)
20:07:31.201 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.201220 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "57af19dfdfce4211824a07be0443b43a" candidate_term: 4 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: false dest_uuid: "95c1f61bee2d40849c44869e11134c4a" is_pre_election: true
20:07:31.201 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.201643 10256 leader_election.cc:304] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57af19dfdfce4211824a07be0443b43a; no voters: 3ab092e11f74445fbbe7a4c29511b76b, 95c1f61bee2d40849c44869e11134c4a
20:07:31.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.201843 10381 raft_consensus.cc:2749] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
20:07:31.264 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.264230 10386 raft_consensus.cc:493] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ab092e11f74445fbbe7a4c29511b76b)
20:07:31.264 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.264339 10386 raft_consensus.cc:515] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:31.264 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.264672 10386 leader_election.cc:290] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709), 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897)
20:07:31.265 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:31.265105 10075 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.230.129:42709: connect: Connection refused (error 111)
20:07:31.265 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:31.265705 10075 leader_election.cc:336] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709): Network error: Client connection negotiation failed: client connection to 127.8.230.129:42709: connect: Connection refused (error 111)
20:07:31.268 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.268056 10321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "95c1f61bee2d40849c44869e11134c4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 16 } } ignore_live_leader: false dest_uuid: "57af19dfdfce4211824a07be0443b43a" is_pre_election: true
20:07:31.268 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.268255 10321 raft_consensus.cc:2468] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95c1f61bee2d40849c44869e11134c4a in term 3.
20:07:31.268 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.268530 10076 leader_election.cc:304] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57af19dfdfce4211824a07be0443b43a, 95c1f61bee2d40849c44869e11134c4a; no voters: 3ab092e11f74445fbbe7a4c29511b76b
20:07:31.268 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.268675 10386 raft_consensus.cc:2804] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 4 FOLLOWER]: Leader pre-election won for term 5
20:07:31.268 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.268740 10386 raft_consensus.cc:493] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ab092e11f74445fbbe7a4c29511b76b)
20:07:31.268 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.268764 10386 raft_consensus.cc:3060] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 4 FOLLOWER]: Advancing to term 5
20:07:31.269 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.269465 10386 raft_consensus.cc:515] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:31.269 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.269671 10386 leader_election.cc:290] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [CANDIDATE]: Term 5 election: Requested vote from peers 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709), 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897)
20:07:31.270 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.269870 10321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "95c1f61bee2d40849c44869e11134c4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 16 } } ignore_live_leader: false dest_uuid: "57af19dfdfce4211824a07be0443b43a"
20:07:31.270 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.269976 10321 raft_consensus.cc:3060] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 3 FOLLOWER]: Advancing to term 5
20:07:31.270 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:31.270246 10075 leader_election.cc:336] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709): Network error: Client connection negotiation failed: client connection to 127.8.230.129:42709: connect: Connection refused (error 111)
20:07:31.271 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.271111 10321 raft_consensus.cc:2468] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95c1f61bee2d40849c44869e11134c4a in term 5.
20:07:31.271 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.271333 10076 leader_election.cc:304] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57af19dfdfce4211824a07be0443b43a, 95c1f61bee2d40849c44869e11134c4a; no voters: 3ab092e11f74445fbbe7a4c29511b76b
20:07:31.271 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.271441 10386 raft_consensus.cc:2804] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 5 FOLLOWER]: Leader election won for term 5
20:07:31.271 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.271587 10386 raft_consensus.cc:697] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 5 LEADER]: Becoming Leader. State: Replica: 95c1f61bee2d40849c44869e11134c4a, State: Running, Role: LEADER
20:07:31.271 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.271682 10386 consensus_queue.cc:237] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 4.16, Last appended by leader: 16, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:31.272 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.272403  9300 catalog_manager.cc:5696] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a reported cstate change: term changed from 4 to 5, leader changed from 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129) to 95c1f61bee2d40849c44869e11134c4a (127.8.230.131). New cstate: current_term: 5 leader_uuid: "95c1f61bee2d40849c44869e11134c4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } health_report { overall_health: HEALTHY } } }
20:07:31.666 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.665529 10321 raft_consensus.cc:1275] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 5 FOLLOWER]: Refusing update from remote peer 95c1f61bee2d40849c44869e11134c4a: Log matching property violated. Preceding OpId in replica: term: 3 index: 12. Preceding OpId from leader: term: 5 index: 17. (index mismatch)
20:07:31.666 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.666126 10386 consensus_queue.cc:1048] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 12, Time since last communication: 0.000s
20:07:31.670 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:31.670022 10075 consensus_peers.cc:597] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a -> Peer 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709): Couldn't send request to peer 3ab092e11f74445fbbe7a4c29511b76b. Status: Network error: Client connection negotiation failed: client connection to 127.8.230.129:42709: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
20:07:31.737 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:31.738 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)[L]]}]
20:07:31.738 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=13, sequence_id=13, TimeoutTracker(timeout=30000, elapsed=1813), Traces: [0ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [1ms] delaying RPC due to: Illegal state: connection is terminated, [11ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [13ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [13ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [31ms] refreshing cache from master, [31ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [33ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [34ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [36ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [36ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [51ms] refreshing cache from master, [51ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [53ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [54ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [56ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [56ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [90ms] refreshing cache from master, [91ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [93ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [94ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [95ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [95ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [250ms] refreshing cache from master, [250ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [252ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [254ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [256ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [256ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [531ms] refreshing cache from master, [531ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [533ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [535ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [536ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [536ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [1810ms] refreshing cache from master, [1811ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [1813ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@1142168194(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@749289e0 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@924897183 after retry RPC@159031679) -> (continuation of Deferred@2112410369 after retry RPC@1816837884) -> (continuation of Deferred@1646429832 after retry RPC@1537632622) -> (continuation of Deferred@1370249444 after retry RPC@959035046) -> (continuation of Deferred@1248718009 after retry RPC@1040576468), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@3c005c98 -> wakeup thread Time-limited test -> (continuation of Deferred@924897183 after retry RPC@159031679) -> (continuation of Deferred@2112410369 after retry RPC@1816837884) -> (continuation of Deferred@1646429832 after retry RPC@1537632622) -> (continuation of Deferred@1370249444 after retry RPC@959035046) -> (continuation of Deferred@1248718009 after retry RPC@1040576468))) after lookup
20:07:31.739 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] connecting to peer
20:07:31.741 [DEBUG - kudu-nio-6] (Connection.java:757) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] Successfully connected to peer
20:07:31.773 [DEBUG - kudu-nio-6] (Negotiator.java:919) Authenticated connection [id: 0x8053519c, L:/127.0.0.1:41170 - R:/127.8.230.131:37545] using TOKEN/null
20:07:31.793 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.8.230.129:42709
20:07:31.794 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

20:07:31.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.794402  9114 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:31.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:31.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal
20:07:31.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data
20:07:31.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
20:07:31.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
20:07:31.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
20:07:31.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:31.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:31.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:31.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/logs
20:07:31.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data/info.pb
20:07:31.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:31.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
20:07:31.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
20:07:31.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
20:07:31.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
20:07:31.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:31.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:31.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:31.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
20:07:31.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
20:07:31.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.129:42709
20:07:31.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.8.230.129
20:07:31.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:31.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=44243
20:07:31.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.129
20:07:31.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:31.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:31.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:31.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
20:07:31.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:31.878818 10413 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
20:07:31.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:31.879027 10413 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
20:07:31.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:31.879050 10413 flags.cc:432] Enabled unsafe flag: --never_fsync=true
20:07:31.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:31.881024 10413 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
20:07:31.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:31.881114 10413 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.230.129
20:07:31.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.883213 10413 tablet_server_runner.cc:78] Tablet server non-default flags:
20:07:31.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:31.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:31.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:31.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
20:07:31.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data
20:07:31.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal
20:07:31.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:31.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:31.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.129:42709
20:07:31.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
20:07:31.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:31.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:31.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data/info.pb
20:07:31.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.129
20:07:31.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=44243
20:07:31.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:31.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
20:07:31.884 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.10413
20:07:31.884 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:31.884 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
20:07:31.884 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
20:07:31.884 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:31.884 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.8.230.129
20:07:31.884 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/logs
20:07:31.884 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:31.884 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
20:07:31.884 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
20:07:31.884 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
20:07:31.884 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
20:07:31.884 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 66827b15b158c37363c37baa160f29a0ca1f1e63
20:07:31.884 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type RELEASE
20:07:31.884 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 18 May 2026 19:43:16 UTC on e7f111948823
20:07:31.884 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 11961
20:07:31.884 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.883482 10413 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
20:07:31.884 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.883740 10413 file_cache.cc:492] Constructed file cache file cache with capacity 419430
20:07:31.886 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:31.886523 10418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:31.887 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:31.886557 10419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:31.887 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.886790 10413 server_base.cc:1061] running on GCE node
20:07:31.887 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:31.887423 10421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:31.887 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.887674 10413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
20:07:31.888 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.887898 10413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
20:07:31.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.889084 10413 hybrid_clock.cc:648] HybridClock initialized: now 1779134851889046 us; error 53 us; skew 500 ppm
20:07:31.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.890633 10413 webserver.cc:533] Webserver started at http://127.8.230.129:44243/ using document root <none> and password file <none>
20:07:31.891 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.890893 10413 fs_manager.cc:362] Metadata directory not provided
20:07:31.891 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.890976 10413 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
20:07:31.892 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.892333 10413 fs_manager.cc:714] Time spent opening directory manager: real 0.001s	user 0.001s	sys 0.000s
20:07:31.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.893090 10427 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
20:07:31.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.893322 10413 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
20:07:31.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.893427 10413 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data,/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal
20:07:31.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "3ab092e11f74445fbbe7a4c29511b76b"
20:07:31.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:21 on dist-test-slave-mzrs"
20:07:31.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.893774 10413 fs_report.cc:389] FS layout report
20:07:31.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
20:07:31.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal
20:07:31.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal
20:07:31.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data/data
20:07:31.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
20:07:31.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
20:07:31.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
20:07:31.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
20:07:31.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
20:07:31.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
20:07:31.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
20:07:31.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
20:07:31.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
20:07:31.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
20:07:31.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
20:07:31.908 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.908258 10413 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
20:07:31.908 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.908648 10413 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
20:07:31.908 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.908785 10413 kserver.cc:163] Server-wide thread pool size limit: 3276
20:07:31.909 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.909047 10413 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
20:07:31.909 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.909608 10434 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
20:07:31.910 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.910786 10413 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
20:07:31.911 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.910858 10413 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
20:07:31.911 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.910897 10413 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
20:07:31.911 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.911548 10413 ts_tablet_manager.cc:616] Registered 1 tablets
20:07:31.911 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.911609 10413 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
20:07:31.911 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.911636 10434 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Bootstrap starting.
20:07:31.919 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.919202 10413 rpc_server.cc:307] RPC server started. Bound to: 127.8.230.129:42709
20:07:31.919 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.919252 10541 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.230.129:42709 every 8 connection(s)
20:07:31.919 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.919615 10413 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data/info.pb
20:07:31.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.921139 10434 log.cc:826] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Log is configured to *not* fsync() on all Append() calls
20:07:31.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.921360  9114 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskx5o66C/build/release/bin/kudu as pid 10413
20:07:31.921 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
20:07:31.925 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.924651 10434 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=0 applied=16 ignored=0} inserts{seen=12 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
20:07:31.925 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.925060 10434 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Bootstrap complete.
20:07:31.926 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.926187 10434 ts_tablet_manager.cc:1403] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Time spent bootstrapping tablet: real 0.015s	user 0.003s	sys 0.007s
20:07:31.926 [DEBUG - kudu-nio-6] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 15 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181, tablet=a4035636acf142eeb16e984e2f510768, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
20:07:31.927 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.8.230.131:37545
20:07:31.927 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 2
  }
}

20:07:31.928 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.927348 10434 raft_consensus.cc:359] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:31.929 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.927665 10434 raft_consensus.cc:740] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ab092e11f74445fbbe7a4c29511b76b, State: Initialized, Role: FOLLOWER
20:07:31.929 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.927829 10434 consensus_queue.cc:260] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 4.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:31.929 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.928105  9114 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskx5o66C/build/release/bin/kudu with pid 10060
20:07:31.931 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.931361 10434 ts_tablet_manager.cc:1434] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.000s
20:07:31.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.933872 10544 heartbeater.cc:344] Connected to a master server at 127.8.230.189:44351
20:07:31.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.934031 10544 heartbeater.cc:461] Registering TS with master...
20:07:31.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.934329 10544 heartbeater.cc:507] Master 127.8.230.189:44351 requested a full tablet report, sending...
20:07:31.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.933917 10542 heartbeater.cc:344] Connected to a master server at 127.8.230.188:39477
20:07:31.935 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.935006  9227 ts_manager.cc:194] Re-registered known tserver with Master: 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709)
20:07:31.935 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.934755 10543 heartbeater.cc:344] Connected to a master server at 127.8.230.190:34269
20:07:31.935 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.935174 10543 heartbeater.cc:461] Registering TS with master...
20:07:31.935 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.935298 10543 heartbeater.cc:507] Master 127.8.230.190:34269 requested a full tablet report, sending...
20:07:31.935 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.935645  9156 ts_manager.cc:194] Re-registered known tserver with Master: 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709)
20:07:31.936 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.934769 10542 heartbeater.cc:461] Registering TS with master...
20:07:31.936 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.936030 10542 heartbeater.cc:507] Master 127.8.230.188:39477 requested a full tablet report, sending...
20:07:31.936 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.936321  9300 ts_manager.cc:194] Re-registered known tserver with Master: 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709)
20:07:31.936 [DEBUG - kudu-nio-6] (Connection.java:239) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] handling channelInactive
20:07:31.936 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] cleaning up while in state READY due to: connection closed
20:07:31.936 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.936762  9300 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.8.230.129:52745
20:07:31.937 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
20:07:31.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:31.937940 10542 heartbeater.cc:499] Master 127.8.230.188:39477 was elected leader, sending a full tablet report...
20:07:31.938 [DEBUG - Time-limited test] (Connection.java:748) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] connecting to peer
20:07:31.939 [INFO - kudu-nio-7] (Connection.java:429) Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:31.939 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] cleaning up while in state CONNECTING due to: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:31.940 [INFO - kudu-nio-7] (AsyncKuduClient.java:2355) Invalidating location 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:31.957 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:31.958 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)[L]]}]
20:07:31.959 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=2, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=21), Traces: [0ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [2ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [2ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [17ms] refreshing cache from master, [17ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [19ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@822544020(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@206d342f -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@29400991 -> wakeup thread Time-limited test)) after lookup
20:07:31.959 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] connecting to peer
20:07:31.960 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:31.961 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] cleaning up while in state CONNECTING due to: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:31.961 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:31.977 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:31.978 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)[L]]}]
20:07:31.978 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=4, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=40), Traces: [0ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [2ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [2ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [17ms] refreshing cache from master, [17ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [19ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [21ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [23ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [23ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [36ms] refreshing cache from master, [37ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [39ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@822544020(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@206d342f -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@743747211 after retry RPC@91305489), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@29400991 -> wakeup thread Time-limited test -> (continuation of Deferred@743747211 after retry RPC@91305489))) after lookup
20:07:31.979 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] connecting to peer
20:07:31.980 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:31.980 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] cleaning up while in state CONNECTING due to: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:31.980 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:32.017 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:32.018 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)[L]]}]
20:07:32.018 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=6, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=80), Traces: [0ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [2ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [2ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [17ms] refreshing cache from master, [17ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [19ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [21ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [23ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [23ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [36ms] refreshing cache from master, [37ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [39ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [40ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [42ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [42ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [76ms] refreshing cache from master, [76ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [79ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@822544020(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@206d342f -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@743747211 after retry RPC@91305489) -> (continuation of Deferred@1375367704 after retry RPC@1563887496), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@29400991 -> wakeup thread Time-limited test -> (continuation of Deferred@743747211 after retry RPC@91305489) -> (continuation of Deferred@1375367704 after retry RPC@1563887496))) after lookup
20:07:32.019 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] connecting to peer
20:07:32.020 [INFO - kudu-nio-2] (Connection.java:429) Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:32.020 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] cleaning up while in state CONNECTING due to: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:32.021 [INFO - kudu-nio-2] (AsyncKuduClient.java:2355) Invalidating location 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:32.117 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:32.117 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)[L]]}]
20:07:32.118 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=8, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=180), Traces: [0ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [2ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [2ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [17ms] refreshing cache from master, [17ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [19ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [21ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [23ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [23ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [36ms] refreshing cache from master, [37ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [39ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [40ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [42ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [42ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [76ms] refreshing cache from master, [76ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [79ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [81ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [82ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [83ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [177ms] refreshing cache from master, [177ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [179ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@822544020(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@206d342f -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@743747211 after retry RPC@91305489) -> (continuation of Deferred@1375367704 after retry RPC@1563887496) -> (continuation of Deferred@1238123925 after retry RPC@58980135), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@29400991 -> wakeup thread Time-limited test -> (continuation of Deferred@743747211 after retry RPC@91305489) -> (continuation of Deferred@1375367704 after retry RPC@1563887496) -> (continuation of Deferred@1238123925 after retry RPC@58980135))) after lookup
20:07:32.118 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] connecting to peer
20:07:32.119 [INFO - kudu-nio-3] (Connection.java:429) Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:32.120 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] cleaning up while in state CONNECTING due to: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:32.120 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:32.358 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:32.358 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)[L]]}]
20:07:32.358 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=10, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=420), Traces: [0ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [2ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [2ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [17ms] refreshing cache from master, [17ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [19ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [21ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [23ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [23ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [36ms] refreshing cache from master, [37ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [39ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [40ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [42ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [42ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [76ms] refreshing cache from master, [76ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [79ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [81ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [82ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [83ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [177ms] refreshing cache from master, [177ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [179ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [180ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [182ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [182ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [417ms] refreshing cache from master, [417ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [420ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@822544020(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@206d342f -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@743747211 after retry RPC@91305489) -> (continuation of Deferred@1375367704 after retry RPC@1563887496) -> (continuation of Deferred@1238123925 after retry RPC@58980135) -> (continuation of Deferred@1067141289 after retry RPC@591248050), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@29400991 -> wakeup thread Time-limited test -> (continuation of Deferred@743747211 after retry RPC@91305489) -> (continuation of Deferred@1375367704 after retry RPC@1563887496) -> (continuation of Deferred@1238123925 after retry RPC@58980135) -> (continuation of Deferred@1067141289 after retry RPC@591248050))) after lookup
20:07:32.359 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] connecting to peer
20:07:32.360 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:32.361 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] cleaning up while in state CONNECTING due to: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:32.361 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:32.837 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:32.837 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)[L]]}]
20:07:32.838 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=12, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=900), Traces: [0ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [2ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [2ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [17ms] refreshing cache from master, [17ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [19ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [21ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [23ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [23ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [36ms] refreshing cache from master, [37ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [39ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [40ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [42ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [42ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [76ms] refreshing cache from master, [76ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [79ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [81ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [82ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [83ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [177ms] refreshing cache from master, [177ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [179ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [180ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [182ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [182ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [417ms] refreshing cache from master, [417ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [420ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [421ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [423ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [423ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [896ms] refreshing cache from master, [896ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [899ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@822544020(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@206d342f -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@743747211 after retry RPC@91305489) -> (continuation of Deferred@1375367704 after retry RPC@1563887496) -> (continuation of Deferred@1238123925 after retry RPC@58980135) -> (continuation of Deferred@1067141289 after retry RPC@591248050) -> (continuation of Deferred@19925728 after retry RPC@1335281758), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@29400991 -> wakeup thread Time-limited test -> (continuation of Deferred@743747211 after retry RPC@91305489) -> (continuation of Deferred@1375367704 after retry RPC@1563887496) -> (continuation of Deferred@1238123925 after retry RPC@58980135) -> (continuation of Deferred@1067141289 after retry RPC@591248050) -> (continuation of Deferred@19925728 after retry RPC@1335281758))) after lookup
20:07:32.839 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] connecting to peer
20:07:32.840 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:32.840 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] cleaning up while in state CONNECTING due to: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:32.840 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:33.167 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.167064 10555 raft_consensus.cc:493] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
20:07:33.167 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.167266 10555 raft_consensus.cc:515] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:33.167 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.167717 10555 leader_election.cc:290] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897), 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:33.168 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:33.168254 10431 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.230.131:37545: connect: Connection refused (error 111)
20:07:33.168 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:33.168809 10431 leader_election.cc:336] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545): Network error: Client connection negotiation failed: client connection to 127.8.230.131:37545: connect: Connection refused (error 111)
20:07:33.171 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.171177 10321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "3ab092e11f74445fbbe7a4c29511b76b" candidate_term: 5 candidate_status { last_received { term: 4 index: 16 } } ignore_live_leader: false dest_uuid: "57af19dfdfce4211824a07be0443b43a" is_pre_election: true
20:07:33.171 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.171646 10429 leader_election.cc:304] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ab092e11f74445fbbe7a4c29511b76b; no voters: 57af19dfdfce4211824a07be0443b43a, 95c1f61bee2d40849c44869e11134c4a
20:07:33.171 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.171837 10555 raft_consensus.cc:2749] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
20:07:33.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.292933 10560 raft_consensus.cc:493] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 95c1f61bee2d40849c44869e11134c4a)
20:07:33.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.293040 10560 raft_consensus.cc:515] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:33.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.293241 10560 leader_election.cc:290] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709), 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:33.294 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:33.294281 10256 leader_election.cc:336] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545): Network error: Client connection negotiation failed: client connection to 127.8.230.131:37545: connect: Connection refused (error 111)
20:07:33.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.297008 10496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "57af19dfdfce4211824a07be0443b43a" candidate_term: 6 candidate_status { last_received { term: 5 index: 20 } } ignore_live_leader: false dest_uuid: "3ab092e11f74445fbbe7a4c29511b76b" is_pre_election: true
20:07:33.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.297145 10496 raft_consensus.cc:2468] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57af19dfdfce4211824a07be0443b43a in term 4.
20:07:33.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.297411 10253 leader_election.cc:304] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ab092e11f74445fbbe7a4c29511b76b, 57af19dfdfce4211824a07be0443b43a; no voters: 95c1f61bee2d40849c44869e11134c4a
20:07:33.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.297539 10560 raft_consensus.cc:2804] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 5 FOLLOWER]: Leader pre-election won for term 6
20:07:33.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.297600 10560 raft_consensus.cc:493] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 95c1f61bee2d40849c44869e11134c4a)
20:07:33.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.297617 10560 raft_consensus.cc:3060] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 5 FOLLOWER]: Advancing to term 6
20:07:33.298 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.298441 10560 raft_consensus.cc:515] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:33.298 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.298626 10560 leader_election.cc:290] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [CANDIDATE]: Term 6 election: Requested vote from peers 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709), 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:33.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.298800 10496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "57af19dfdfce4211824a07be0443b43a" candidate_term: 6 candidate_status { last_received { term: 5 index: 20 } } ignore_live_leader: false dest_uuid: "3ab092e11f74445fbbe7a4c29511b76b"
20:07:33.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.298920 10496 raft_consensus.cc:3060] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 4 FOLLOWER]: Advancing to term 6
20:07:33.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:33.299332 10256 leader_election.cc:336] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545): Network error: Client connection negotiation failed: client connection to 127.8.230.131:37545: connect: Connection refused (error 111)
20:07:33.300 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.300096 10496 raft_consensus.cc:2468] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57af19dfdfce4211824a07be0443b43a in term 6.
20:07:33.300 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.300333 10253 leader_election.cc:304] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ab092e11f74445fbbe7a4c29511b76b, 57af19dfdfce4211824a07be0443b43a; no voters: 95c1f61bee2d40849c44869e11134c4a
20:07:33.300 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.300444 10560 raft_consensus.cc:2804] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 6 FOLLOWER]: Leader election won for term 6
20:07:33.300 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.300614 10560 raft_consensus.cc:697] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 6 LEADER]: Becoming Leader. State: Replica: 57af19dfdfce4211824a07be0443b43a, State: Running, Role: LEADER
20:07:33.300 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.300742 10560 consensus_queue.cc:237] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 5.20, Last appended by leader: 20, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:33.301 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.301542  9300 catalog_manager.cc:5696] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a reported cstate change: term changed from 5 to 6, leader changed from 95c1f61bee2d40849c44869e11134c4a (127.8.230.131) to 57af19dfdfce4211824a07be0443b43a (127.8.230.130). New cstate: current_term: 6 leader_uuid: "57af19dfdfce4211824a07be0443b43a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } health_report { overall_health: UNKNOWN } } }
20:07:33.884 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:33.883981 10256 consensus_peers.cc:597] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a -> Peer 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545): Couldn't send request to peer 95c1f61bee2d40849c44869e11134c4a. Status: Network error: Client connection negotiation failed: client connection to 127.8.230.131:37545: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
20:07:33.899 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.898743 10496 raft_consensus.cc:1275] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 6 FOLLOWER]: Refusing update from remote peer 57af19dfdfce4211824a07be0443b43a: Log matching property violated. Preceding OpId in replica: term: 4 index: 16. Preceding OpId from leader: term: 6 index: 21. (index mismatch)
20:07:33.899 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:33.899135 10578 consensus_queue.cc:1048] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 16, Time since last communication: 0.000s
20:07:34.577 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:34.578 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)[L],95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:34.578 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=14, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=2640), Traces: [0ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [2ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [2ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [17ms] refreshing cache from master, [17ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [19ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [21ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [23ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [23ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [36ms] refreshing cache from master, [37ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [39ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [40ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [42ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [42ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [76ms] refreshing cache from master, [76ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [79ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [81ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [82ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [83ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [177ms] refreshing cache from master, [177ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [179ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [180ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [182ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [182ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [417ms] refreshing cache from master, [417ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [420ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [421ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [423ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [423ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [896ms] refreshing cache from master, [896ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [899ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [900ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [902ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [902ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [2637ms] refreshing cache from master, [2637ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [2639ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@822544020(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@206d342f -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@743747211 after retry RPC@91305489) -> (continuation of Deferred@1375367704 after retry RPC@1563887496) -> (continuation of Deferred@1238123925 after retry RPC@58980135) -> (continuation of Deferred@1067141289 after retry RPC@591248050) -> (continuation of Deferred@19925728 after retry RPC@1335281758) -> (continuation of Deferred@1059172521 after retry RPC@1175210795), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@29400991 -> wakeup thread Time-limited test -> (continuation of Deferred@743747211 after retry RPC@91305489) -> (continuation of Deferred@1375367704 after retry RPC@1563887496) -> (continuation of Deferred@1238123925 after retry RPC@58980135) -> (continuation of Deferred@1067141289 after retry RPC@591248050) -> (continuation of Deferred@19925728 after retry RPC@1335281758) -> (continuation of Deferred@1059172521 after retry RPC@1175210795))) after lookup
20:07:34.579 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] connecting to peer
20:07:34.580 [DEBUG - kudu-nio-6] (Connection.java:757) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] Successfully connected to peer
20:07:34.595 [DEBUG - kudu-nio-6] (Negotiator.java:919) Authenticated connection [id: 0x9c2f1dd8, L:/127.0.0.1:57324 - R:/127.8.230.130:37897] using TOKEN/null
20:07:34.609 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.8.230.131:37545
20:07:34.611 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 2
  }
}

20:07:34.611 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.611564  9114 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:34.611 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:34.611 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/wal
20:07:34.611 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data
20:07:34.611 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
20:07:34.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
20:07:34.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
20:07:34.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:34.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:34.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:34.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/logs
20:07:34.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data/info.pb
20:07:34.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:34.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
20:07:34.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
20:07:34.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
20:07:34.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
20:07:34.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:34.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:34.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:34.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
20:07:34.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
20:07:34.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.131:37545
20:07:34.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.8.230.131
20:07:34.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:34.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=45551
20:07:34.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.131
20:07:34.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:34.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:34.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:34.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
20:07:34.705 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:34.704913 10599 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
20:07:34.705 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:34.705152 10599 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
20:07:34.705 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:34.705174 10599 flags.cc:432] Enabled unsafe flag: --never_fsync=true
20:07:34.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:34.707103 10599 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
20:07:34.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:34.707188 10599 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.230.131
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.708971 10599 tablet_server_runner.cc:78] Tablet server non-default flags:
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/wal
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.131:37545
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data/info.pb
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.131
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=45551
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.10599
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.8.230.131
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/logs
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
20:07:34.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 66827b15b158c37363c37baa160f29a0ca1f1e63
20:07:34.710 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type RELEASE
20:07:34.710 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 18 May 2026 19:43:16 UTC on e7f111948823
20:07:34.710 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 11961
20:07:34.710 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.709281 10599 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
20:07:34.710 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.709513 10599 file_cache.cc:492] Constructed file cache file cache with capacity 419430
20:07:34.712 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:34.712007 10604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:34.712 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:34.712013 10607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:34.712 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.712210 10599 server_base.cc:1061] running on GCE node
20:07:34.712 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:34.712010 10605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:34.712 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.712733 10599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
20:07:34.713 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.713045 10599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
20:07:34.714 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.714231 10599 hybrid_clock.cc:648] HybridClock initialized: now 1779134854714205 us; error 42 us; skew 500 ppm
20:07:34.715 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.715639 10599 webserver.cc:533] Webserver started at http://127.8.230.131:45551/ using document root <none> and password file <none>
20:07:34.716 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.715900 10599 fs_manager.cc:362] Metadata directory not provided
20:07:34.716 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.715993 10599 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
20:07:34.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.717289 10599 fs_manager.cc:714] Time spent opening directory manager: real 0.001s	user 0.000s	sys 0.001s
20:07:34.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.718016 10613 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
20:07:34.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.718209 10599 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
20:07:34.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.718353 10599 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data,/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/wal
20:07:34.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "95c1f61bee2d40849c44869e11134c4a"
20:07:34.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:21 on dist-test-slave-mzrs"
20:07:34.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.718690 10599 fs_report.cc:389] FS layout report
20:07:34.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
20:07:34.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/wal
20:07:34.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/wal
20:07:34.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data/data
20:07:34.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
20:07:34.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
20:07:34.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
20:07:34.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
20:07:34.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
20:07:34.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
20:07:34.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
20:07:34.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
20:07:34.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
20:07:34.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
20:07:34.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
20:07:34.748 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.747893 10599 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
20:07:34.748 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.748309 10599 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
20:07:34.748 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.748452 10599 kserver.cc:163] Server-wide thread pool size limit: 3276
20:07:34.748 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.748728 10599 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
20:07:34.749 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.749359 10620 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
20:07:34.751 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.750895 10599 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
20:07:34.751 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.751019 10599 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.002s	user 0.000s	sys 0.000s
20:07:34.751 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.751066 10599 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
20:07:34.751 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.751777 10599 ts_tablet_manager.cc:616] Registered 1 tablets
20:07:34.752 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.751845 10599 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
20:07:34.752 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.751891 10620 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a: Bootstrap starting.
20:07:34.759 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.759104 10599 rpc_server.cc:307] RPC server started. Bound to: 127.8.230.131:37545
20:07:34.760 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.759171 10727 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.230.131:37545 every 8 connection(s)
20:07:34.760 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.759516 10599 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data/info.pb
20:07:34.760 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.760401  9114 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskx5o66C/build/release/bin/kudu as pid 10599
20:07:34.761 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
20:07:34.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.764146 10620 log.cc:826] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a: Log is configured to *not* fsync() on all Append() calls
20:07:34.765 [DEBUG - kudu-nio-6] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 18 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181, tablet=a4035636acf142eeb16e984e2f510768, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
20:07:34.766 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.8.230.130:37897
20:07:34.767 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 1
  }
}

20:07:34.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.767381  9114 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskx5o66C/build/release/bin/kudu with pid 10238
20:07:34.769 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.768921 10730 heartbeater.cc:344] Connected to a master server at 127.8.230.189:44351
20:07:34.769 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.769078 10730 heartbeater.cc:461] Registering TS with master...
20:07:34.769 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.769366 10730 heartbeater.cc:507] Master 127.8.230.189:44351 requested a full tablet report, sending...
20:07:34.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.769937 10728 heartbeater.cc:344] Connected to a master server at 127.8.230.188:39477
20:07:34.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.770002 10728 heartbeater.cc:461] Registering TS with master...
20:07:34.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.770109 10728 heartbeater.cc:507] Master 127.8.230.188:39477 requested a full tablet report, sending...
20:07:34.771 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.770398 10729 heartbeater.cc:344] Connected to a master server at 127.8.230.190:34269
20:07:34.771 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.770454 10729 heartbeater.cc:461] Registering TS with master...
20:07:34.771 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.770545 10729 heartbeater.cc:507] Master 127.8.230.190:34269 requested a full tablet report, sending...
20:07:34.771 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.770545  9300 ts_manager.cc:194] Re-registered known tserver with Master: 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:34.771 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.770823  9227 ts_manager.cc:194] Re-registered known tserver with Master: 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:34.771 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.771047  9300 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.8.230.131:60987
20:07:34.771 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.771128  9156 ts_manager.cc:194] Re-registered known tserver with Master: 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:34.774 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.773962 10620 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a: Bootstrap replayed 1/1 log segments. Stats: ops{read=20 overwritten=0 applied=20 ignored=0} inserts{seen=15 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
20:07:34.774 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.774381 10620 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a: Bootstrap complete.
20:07:34.775 [DEBUG - kudu-nio-6] (Connection.java:239) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] handling channelInactive
20:07:34.775 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.775483 10620 ts_tablet_manager.cc:1403] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a: Time spent bootstrapping tablet: real 0.024s	user 0.002s	sys 0.014s
20:07:34.775 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] cleaning up while in state READY due to: connection closed
20:07:34.776 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
20:07:34.777 [DEBUG - Time-limited test] (Connection.java:748) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] connecting to peer
20:07:34.777 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.776705 10620 raft_consensus.cc:359] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:34.777 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.776957 10620 raft_consensus.cc:740] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95c1f61bee2d40849c44869e11134c4a, State: Initialized, Role: FOLLOWER
20:07:34.777 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.777096 10620 consensus_queue.cc:260] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 20, Last appended: 5.20, Last appended by leader: 20, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:34.777 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.777417 10620 ts_tablet_manager.cc:1434] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a: Time spent starting tablet: real 0.002s	user 0.000s	sys 0.003s
20:07:34.777 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:34.777541 10728 heartbeater.cc:499] Master 127.8.230.188:39477 was elected leader, sending a full tablet report...
20:07:34.778 [INFO - kudu-nio-7] (Connection.java:429) Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:34.779 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] cleaning up while in state CONNECTING due to: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:34.779 [INFO - kudu-nio-7] (AsyncKuduClient.java:2355) Invalidating location 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:34.797 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:34.798 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)[L],95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:34.798 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=2, sequence_id=19, TimeoutTracker(timeout=30000, elapsed=22), Traces: [0ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [2ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [2ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [17ms] refreshing cache from master, [18ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [20ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@1883743130(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@5436fbaa -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@153bf12 -> wakeup thread Time-limited test)) after lookup
20:07:34.799 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] connecting to peer
20:07:34.801 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:34.801 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] cleaning up while in state CONNECTING due to: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:34.801 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:34.817 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:34.818 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)[L],95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:34.818 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=4, sequence_id=19, TimeoutTracker(timeout=30000, elapsed=41), Traces: [0ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [2ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [2ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [17ms] refreshing cache from master, [18ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [20ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [22ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [24ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [25ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [38ms] refreshing cache from master, [38ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [40ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@1883743130(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@5436fbaa -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1651362844 after retry RPC@2128237676), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@153bf12 -> wakeup thread Time-limited test -> (continuation of Deferred@1651362844 after retry RPC@2128237676))) after lookup
20:07:34.819 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] connecting to peer
20:07:34.820 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:34.820 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] cleaning up while in state CONNECTING due to: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:34.820 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:34.837 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:34.837 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)[L],95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:34.837 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=6, sequence_id=19, TimeoutTracker(timeout=30000, elapsed=60), Traces: [0ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [2ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [2ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [17ms] refreshing cache from master, [18ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [20ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [22ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [24ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [25ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [38ms] refreshing cache from master, [38ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [40ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [41ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [43ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [43ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [58ms] refreshing cache from master, [58ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [60ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@1883743130(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@5436fbaa -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1651362844 after retry RPC@2128237676) -> (continuation of Deferred@247135052 after retry RPC@586135491), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@153bf12 -> wakeup thread Time-limited test -> (continuation of Deferred@1651362844 after retry RPC@2128237676) -> (continuation of Deferred@247135052 after retry RPC@586135491))) after lookup
20:07:34.838 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] connecting to peer
20:07:34.839 [INFO - kudu-nio-2] (Connection.java:429) Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:34.839 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] cleaning up while in state CONNECTING due to: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:34.840 [INFO - kudu-nio-2] (AsyncKuduClient.java:2355) Invalidating location 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:34.937 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:34.938 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)[L],95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:34.938 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=8, sequence_id=19, TimeoutTracker(timeout=30000, elapsed=161), Traces: [0ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [2ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [2ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [17ms] refreshing cache from master, [18ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [20ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [22ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [24ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [25ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [38ms] refreshing cache from master, [38ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [40ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [41ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [43ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [43ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [58ms] refreshing cache from master, [58ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [60ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [61ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [62ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [63ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [158ms] refreshing cache from master, [158ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [160ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@1883743130(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@5436fbaa -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1651362844 after retry RPC@2128237676) -> (continuation of Deferred@247135052 after retry RPC@586135491) -> (continuation of Deferred@32474232 after retry RPC@708668224), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@153bf12 -> wakeup thread Time-limited test -> (continuation of Deferred@1651362844 after retry RPC@2128237676) -> (continuation of Deferred@247135052 after retry RPC@586135491) -> (continuation of Deferred@32474232 after retry RPC@708668224))) after lookup
20:07:34.939 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] connecting to peer
20:07:34.940 [INFO - kudu-nio-3] (Connection.java:429) Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:34.940 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] cleaning up while in state CONNECTING due to: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:34.940 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:34.997 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:34.998 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)[L],95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:34.999 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=10, sequence_id=19, TimeoutTracker(timeout=30000, elapsed=222), Traces: [0ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [2ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [2ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [17ms] refreshing cache from master, [18ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [20ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [22ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [24ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [25ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [38ms] refreshing cache from master, [38ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [40ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [41ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [43ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [43ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [58ms] refreshing cache from master, [58ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [60ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [61ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [62ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [63ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [158ms] refreshing cache from master, [158ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [160ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [161ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [163ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [163ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [218ms] refreshing cache from master, [218ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [220ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@1883743130(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@5436fbaa -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1651362844 after retry RPC@2128237676) -> (continuation of Deferred@247135052 after retry RPC@586135491) -> (continuation of Deferred@32474232 after retry RPC@708668224) -> (continuation of Deferred@1123229231 after retry RPC@1436630500), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@153bf12 -> wakeup thread Time-limited test -> (continuation of Deferred@1651362844 after retry RPC@2128237676) -> (continuation of Deferred@247135052 after retry RPC@586135491) -> (continuation of Deferred@32474232 after retry RPC@708668224) -> (continuation of Deferred@1123229231 after retry RPC@1436630500))) after lookup
20:07:34.999 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] connecting to peer
20:07:35.000 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:35.000 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] cleaning up while in state CONNECTING due to: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:35.000 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:35.517 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:35.517 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)[L],95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:35.517 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=12, sequence_id=19, TimeoutTracker(timeout=30000, elapsed=740), Traces: [0ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [2ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [2ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [17ms] refreshing cache from master, [18ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [20ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [22ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [24ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [25ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [38ms] refreshing cache from master, [38ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [40ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [41ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [43ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [43ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [58ms] refreshing cache from master, [58ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [60ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [61ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [62ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [63ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [158ms] refreshing cache from master, [158ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [160ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [161ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [163ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [163ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [218ms] refreshing cache from master, [218ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [220ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [222ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [223ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [223ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [737ms] refreshing cache from master, [738ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [739ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@1883743130(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@5436fbaa -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1651362844 after retry RPC@2128237676) -> (continuation of Deferred@247135052 after retry RPC@586135491) -> (continuation of Deferred@32474232 after retry RPC@708668224) -> (continuation of Deferred@1123229231 after retry RPC@1436630500) -> (continuation of Deferred@2125525554 after retry RPC@581090845), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@153bf12 -> wakeup thread Time-limited test -> (continuation of Deferred@1651362844 after retry RPC@2128237676) -> (continuation of Deferred@247135052 after retry RPC@586135491) -> (continuation of Deferred@32474232 after retry RPC@708668224) -> (continuation of Deferred@1123229231 after retry RPC@1436630500) -> (continuation of Deferred@2125525554 after retry RPC@581090845))) after lookup
20:07:35.519 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] connecting to peer
20:07:35.520 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:35.520 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] cleaning up while in state CONNECTING due to: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:35.521 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:36.109 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:36.109026 10741 raft_consensus.cc:493] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 57af19dfdfce4211824a07be0443b43a)
20:07:36.109 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:36.109180 10741 raft_consensus.cc:515] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:36.109 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:36.109375 10741 leader_election.cc:290] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897), 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:36.110 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:36.110255 10429 leader_election.cc:336] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897): Network error: Client connection negotiation failed: client connection to 127.8.230.130:37897: connect: Connection refused (error 111)
20:07:36.113 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:36.112975 10682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "3ab092e11f74445fbbe7a4c29511b76b" candidate_term: 7 candidate_status { last_received { term: 6 index: 24 } } ignore_live_leader: false dest_uuid: "95c1f61bee2d40849c44869e11134c4a" is_pre_election: true
20:07:36.113 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:36.113127 10682 raft_consensus.cc:2468] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ab092e11f74445fbbe7a4c29511b76b in term 5.
20:07:36.113 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:36.113409 10431 leader_election.cc:304] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ab092e11f74445fbbe7a4c29511b76b, 95c1f61bee2d40849c44869e11134c4a; no voters: 57af19dfdfce4211824a07be0443b43a
20:07:36.113 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:36.113565 10741 raft_consensus.cc:2804] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 6 FOLLOWER]: Leader pre-election won for term 7
20:07:36.113 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:36.113646 10741 raft_consensus.cc:493] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 57af19dfdfce4211824a07be0443b43a)
20:07:36.113 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:36.113672 10741 raft_consensus.cc:3060] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 6 FOLLOWER]: Advancing to term 7
20:07:36.114 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:36.114492 10741 raft_consensus.cc:515] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:36.114 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:36.114729 10741 leader_election.cc:290] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [CANDIDATE]: Term 7 election: Requested vote from peers 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897), 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:36.115 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:36.115003 10682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "3ab092e11f74445fbbe7a4c29511b76b" candidate_term: 7 candidate_status { last_received { term: 6 index: 24 } } ignore_live_leader: false dest_uuid: "95c1f61bee2d40849c44869e11134c4a"
20:07:36.115 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:36.115118 10682 raft_consensus.cc:3060] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 5 FOLLOWER]: Advancing to term 7
20:07:36.115 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:36.115501 10429 leader_election.cc:336] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897): Network error: Client connection negotiation failed: client connection to 127.8.230.130:37897: connect: Connection refused (error 111)
20:07:36.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:36.116324 10682 raft_consensus.cc:2468] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab092e11f74445fbbe7a4c29511b76b in term 7.
20:07:36.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:36.116590 10431 leader_election.cc:304] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ab092e11f74445fbbe7a4c29511b76b, 95c1f61bee2d40849c44869e11134c4a; no voters: 57af19dfdfce4211824a07be0443b43a
20:07:36.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:36.116807 10741 raft_consensus.cc:2804] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 7 FOLLOWER]: Leader election won for term 7
20:07:36.117 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:36.117202 10741 raft_consensus.cc:697] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 7 LEADER]: Becoming Leader. State: Replica: 3ab092e11f74445fbbe7a4c29511b76b, State: Running, Role: LEADER
20:07:36.117 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:36.117417 10741 consensus_queue.cc:237] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 6.24, Last appended by leader: 24, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:36.122 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:36.122059  9300 catalog_manager.cc:5696] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b reported cstate change: term changed from 6 to 7, leader changed from 57af19dfdfce4211824a07be0443b43a (127.8.230.130) to 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129). New cstate: current_term: 7 leader_uuid: "3ab092e11f74445fbbe7a4c29511b76b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } health_report { overall_health: UNKNOWN } } }
20:07:36.550 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:36.550591 10429 consensus_peers.cc:597] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b -> Peer 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897): Couldn't send request to peer 57af19dfdfce4211824a07be0443b43a. Status: Network error: Client connection negotiation failed: client connection to 127.8.230.130:37897: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
20:07:36.652 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:36.652598 10682 raft_consensus.cc:1275] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 7 FOLLOWER]: Refusing update from remote peer 3ab092e11f74445fbbe7a4c29511b76b: Log matching property violated. Preceding OpId in replica: term: 5 index: 20. Preceding OpId from leader: term: 7 index: 25. (index mismatch)
20:07:36.653 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:36.653018 10741 consensus_queue.cc:1048] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [LEADER]: Connected to new peer: Peer: permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 20, Time since last communication: 0.000s
20:07:38.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:38.224207 10429 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.230.130:37897: connect: Connection refused (error 111) [suppressed 5 similar messages]
20:07:39.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:39.177125 10429 consensus_peers.cc:597] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b -> Peer 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897): Couldn't send request to peer 57af19dfdfce4211824a07be0443b43a. Status: Network error: Client connection negotiation failed: client connection to 127.8.230.130:37897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
20:07:39.477 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:39.477 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)[L],57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:39.478 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=14, sequence_id=19, TimeoutTracker(timeout=30000, elapsed=4701), Traces: [0ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [2ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [2ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [17ms] refreshing cache from master, [18ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [20ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [22ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [24ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [25ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [38ms] refreshing cache from master, [38ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [40ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [41ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [43ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [43ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [58ms] refreshing cache from master, [58ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [60ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [61ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [62ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [63ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [158ms] refreshing cache from master, [158ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [160ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [161ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [163ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [163ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [218ms] refreshing cache from master, [218ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [220ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [222ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [223ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [223ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [737ms] refreshing cache from master, [738ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [739ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [741ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [743ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [744ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [4697ms] refreshing cache from master, [4698ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [4700ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@1883743130(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@5436fbaa -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1651362844 after retry RPC@2128237676) -> (continuation of Deferred@247135052 after retry RPC@586135491) -> (continuation of Deferred@32474232 after retry RPC@708668224) -> (continuation of Deferred@1123229231 after retry RPC@1436630500) -> (continuation of Deferred@2125525554 after retry RPC@581090845) -> (continuation of Deferred@252766063 after retry RPC@955642566), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@153bf12 -> wakeup thread Time-limited test -> (continuation of Deferred@1651362844 after retry RPC@2128237676) -> (continuation of Deferred@247135052 after retry RPC@586135491) -> (continuation of Deferred@32474232 after retry RPC@708668224) -> (continuation of Deferred@1123229231 after retry RPC@1436630500) -> (continuation of Deferred@2125525554 after retry RPC@581090845) -> (continuation of Deferred@252766063 after retry RPC@955642566))) after lookup
20:07:39.479 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:39.480 [DEBUG - kudu-nio-6] (Connection.java:757) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] Successfully connected to peer
20:07:39.499 [DEBUG - kudu-nio-6] (Negotiator.java:919) Authenticated connection [id: 0x74366f7c, L:/127.0.0.1:38560 - R:/127.8.230.129:42709] using TOKEN/null
20:07:39.515 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.8.230.130:37897
20:07:39.515 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 1
  }
}

20:07:39.516 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.515856  9114 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:39.516 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:39.516 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/wal
20:07:39.516 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data
20:07:39.516 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
20:07:39.516 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
20:07:39.516 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
20:07:39.516 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:39.516 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:39.516 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:39.516 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/logs
20:07:39.517 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data/info.pb
20:07:39.517 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:39.517 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
20:07:39.517 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
20:07:39.517 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
20:07:39.517 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
20:07:39.517 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:39.517 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:39.517 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:39.517 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
20:07:39.517 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
20:07:39.517 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.130:37897
20:07:39.517 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.8.230.130
20:07:39.517 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:39.517 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=43383
20:07:39.517 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.130
20:07:39.517 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:39.517 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:39.517 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:39.517 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
20:07:39.600 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:39.599861 10786 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
20:07:39.600 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:39.600075 10786 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
20:07:39.600 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:39.600098 10786 flags.cc:432] Enabled unsafe flag: --never_fsync=true
20:07:39.602 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:39.601918 10786 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
20:07:39.602 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:39.602048 10786 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.230.130
20:07:39.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.604033 10786 tablet_server_runner.cc:78] Tablet server non-default flags:
20:07:39.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:39.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:39.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:39.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
20:07:39.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data
20:07:39.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/wal
20:07:39.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:39.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:39.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.130:37897
20:07:39.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
20:07:39.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:39.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:39.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data/info.pb
20:07:39.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.130
20:07:39.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=43383
20:07:39.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:39.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
20:07:39.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.10786
20:07:39.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:39.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
20:07:39.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
20:07:39.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:39.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.8.230.130
20:07:39.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/logs
20:07:39.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:39.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
20:07:39.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
20:07:39.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
20:07:39.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
20:07:39.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 66827b15b158c37363c37baa160f29a0ca1f1e63
20:07:39.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type RELEASE
20:07:39.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 18 May 2026 19:43:16 UTC on e7f111948823
20:07:39.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 11961
20:07:39.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.604305 10786 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
20:07:39.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.604588 10786 file_cache.cc:492] Constructed file cache file cache with capacity 419430
20:07:39.607 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:39.607224 10794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:39.607 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:39.607307 10792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:39.607 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:39.607240 10791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:39.607 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.607630 10786 server_base.cc:1061] running on GCE node
20:07:39.608 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.607797 10786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
20:07:39.608 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.607987 10786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
20:07:39.609 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.609149 10786 hybrid_clock.cc:648] HybridClock initialized: now 1779134859609103 us; error 60 us; skew 500 ppm
20:07:39.610 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.610518 10786 webserver.cc:533] Webserver started at http://127.8.230.130:43383/ using document root <none> and password file <none>
20:07:39.611 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.610838 10786 fs_manager.cc:362] Metadata directory not provided
20:07:39.611 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.610957 10786 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
20:07:39.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.612480 10786 fs_manager.cc:714] Time spent opening directory manager: real 0.001s	user 0.002s	sys 0.000s
20:07:39.613 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.613353 10800 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
20:07:39.613 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.613620 10786 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
20:07:39.614 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.613739 10786 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data,/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/wal
20:07:39.614 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "57af19dfdfce4211824a07be0443b43a"
20:07:39.614 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:21 on dist-test-slave-mzrs"
20:07:39.614 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.614111 10786 fs_report.cc:389] FS layout report
20:07:39.614 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
20:07:39.614 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/wal
20:07:39.614 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/wal
20:07:39.614 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data/data
20:07:39.614 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
20:07:39.614 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
20:07:39.614 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
20:07:39.614 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
20:07:39.614 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
20:07:39.614 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
20:07:39.614 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
20:07:39.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
20:07:39.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
20:07:39.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
20:07:39.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
20:07:39.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.637187 10786 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
20:07:39.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.637591 10786 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
20:07:39.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.637753 10786 kserver.cc:163] Server-wide thread pool size limit: 3276
20:07:39.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.638032 10786 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
20:07:39.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.638628 10807 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
20:07:39.639 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.639518 10786 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
20:07:39.639 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.639600 10786 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
20:07:39.639 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.639643 10786 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
20:07:39.640 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.640301 10786 ts_tablet_manager.cc:616] Registered 1 tablets
20:07:39.640 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.640362 10786 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
20:07:39.640 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.640441 10807 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a: Bootstrap starting.
20:07:39.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.648794 10786 rpc_server.cc:307] RPC server started. Bound to: 127.8.230.130:37897
20:07:39.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.649271 10786 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data/info.pb
20:07:39.650 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.650439 10914 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.230.130:37897 every 8 connection(s)
20:07:39.651 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.651224 10807 log.cc:826] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a: Log is configured to *not* fsync() on all Append() calls
20:07:39.651 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.651392  9114 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskx5o66C/build/release/bin/kudu as pid 10786
20:07:39.652 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
20:07:39.657 [DEBUG - kudu-nio-6] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 21 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181, tablet=a4035636acf142eeb16e984e2f510768, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
20:07:39.657 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.8.230.129:42709
20:07:39.658 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

20:07:39.662 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.659008  9114 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskx5o66C/build/release/bin/kudu with pid 10413
20:07:39.662 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.660579 10807 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a: Bootstrap replayed 1/1 log segments. Stats: ops{read=24 overwritten=0 applied=24 ignored=0} inserts{seen=18 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
20:07:39.662 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.660966 10807 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a: Bootstrap complete.
20:07:39.662 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.662143 10807 ts_tablet_manager.cc:1403] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.004s
20:07:39.664 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.663672 10807 raft_consensus.cc:359] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 6 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:39.664 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.663981 10807 raft_consensus.cc:740] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57af19dfdfce4211824a07be0443b43a, State: Initialized, Role: FOLLOWER
20:07:39.664 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.664026 10915 heartbeater.cc:344] Connected to a master server at 127.8.230.188:39477
20:07:39.664 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.664132 10915 heartbeater.cc:461] Registering TS with master...
20:07:39.664 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.664112 10807 consensus_queue.cc:260] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 24, Last appended: 6.24, Last appended by leader: 24, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:39.664 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.664368 10915 heartbeater.cc:507] Master 127.8.230.188:39477 requested a full tablet report, sending...
20:07:39.665 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.664381 10807 ts_tablet_manager.cc:1434] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a: Time spent starting tablet: real 0.002s	user 0.004s	sys 0.000s
20:07:39.665 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.665001 10916 heartbeater.cc:344] Connected to a master server at 127.8.230.190:34269
20:07:39.665 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.665002  9300 ts_manager.cc:194] Re-registered known tserver with Master: 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897)
20:07:39.665 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.665055 10916 heartbeater.cc:461] Registering TS with master...
20:07:39.665 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.665153 10916 heartbeater.cc:507] Master 127.8.230.190:34269 requested a full tablet report, sending...
20:07:39.665 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.665493  9300 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.8.230.130:47631
20:07:39.665 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.665540  9154 ts_manager.cc:194] Re-registered known tserver with Master: 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897)
20:07:39.666 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.666513 10917 heartbeater.cc:344] Connected to a master server at 127.8.230.189:44351
20:07:39.666 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.666580 10917 heartbeater.cc:461] Registering TS with master...
20:07:39.666 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.666842 10917 heartbeater.cc:507] Master 127.8.230.189:44351 requested a full tablet report, sending...
20:07:39.667 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.666896 10915 heartbeater.cc:499] Master 127.8.230.188:39477 was elected leader, sending a full tablet report...
20:07:39.667 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:39.667225  9227 ts_manager.cc:194] Re-registered known tserver with Master: 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897)
20:07:39.671 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:39.671458  9287 connection.cc:582] server connection from 127.8.230.129:52745 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
20:07:39.671 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:39.671490  9143 connection.cc:582] server connection from 127.8.230.129:45571 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
20:07:39.672 [DEBUG - kudu-nio-6] (Connection.java:239) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] handling channelInactive
20:07:39.672 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state READY due to: connection closed
20:07:39.672 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
20:07:39.673 [DEBUG - Time-limited test] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:39.675 [INFO - kudu-nio-7] (Connection.java:429) Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:39.675 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:39.675 [INFO - kudu-nio-7] (AsyncKuduClient.java:2355) Invalidating location 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:39.697 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:39.697 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)[L],57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:39.698 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=2, sequence_id=22, TimeoutTracker(timeout=30000, elapsed=24), Traces: [0ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [2ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [21ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [24ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@783224828(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@395bbb2d -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2d2b4803 -> wakeup thread Time-limited test)) after lookup
20:07:39.698 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:39.700 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:39.700 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:39.700 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:39.717 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:39.718 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)[L],57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:39.718 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=4, sequence_id=22, TimeoutTracker(timeout=30000, elapsed=45), Traces: [0ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [2ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [21ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [24ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [25ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [27ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [27ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [44ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@783224828(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@395bbb2d -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1551535121 after retry RPC@728149197), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2d2b4803 -> wakeup thread Time-limited test -> (continuation of Deferred@1551535121 after retry RPC@728149197))) after lookup
20:07:39.719 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:39.720 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:39.720 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:39.720 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:39.757 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:39.757 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)[L],57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:39.758 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=6, sequence_id=22, TimeoutTracker(timeout=30000, elapsed=84), Traces: [0ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [2ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [21ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [24ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [25ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [27ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [27ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [44ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [45ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [47ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [47ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [82ms] refreshing cache from master, [82ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [84ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@783224828(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@395bbb2d -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1551535121 after retry RPC@728149197) -> (continuation of Deferred@1149439492 after retry RPC@1511899805), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2d2b4803 -> wakeup thread Time-limited test -> (continuation of Deferred@1551535121 after retry RPC@728149197) -> (continuation of Deferred@1149439492 after retry RPC@1511899805))) after lookup
20:07:39.759 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:39.760 [INFO - kudu-nio-2] (Connection.java:429) Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:39.760 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:39.760 [INFO - kudu-nio-2] (AsyncKuduClient.java:2355) Invalidating location 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:39.897 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:39.897 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)[L],57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:39.897 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=8, sequence_id=22, TimeoutTracker(timeout=30000, elapsed=224), Traces: [0ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [2ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [21ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [24ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [25ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [27ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [27ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [44ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [45ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [47ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [47ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [82ms] refreshing cache from master, [82ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [84ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [85ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [87ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [87ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [221ms] refreshing cache from master, [221ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [224ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@783224828(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@395bbb2d -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1551535121 after retry RPC@728149197) -> (continuation of Deferred@1149439492 after retry RPC@1511899805) -> (continuation of Deferred@1247727768 after retry RPC@1240449293), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2d2b4803 -> wakeup thread Time-limited test -> (continuation of Deferred@1551535121 after retry RPC@728149197) -> (continuation of Deferred@1149439492 after retry RPC@1511899805) -> (continuation of Deferred@1247727768 after retry RPC@1240449293))) after lookup
20:07:39.898 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:39.899 [INFO - kudu-nio-3] (Connection.java:429) Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:39.899 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:39.899 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:40.278 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:40.278 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)[L],57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:40.278 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=10, sequence_id=22, TimeoutTracker(timeout=30000, elapsed=605), Traces: [0ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [2ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [21ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [24ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [25ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [27ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [27ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [44ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [45ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [47ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [47ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [82ms] refreshing cache from master, [82ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [84ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [85ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [87ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [87ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [221ms] refreshing cache from master, [221ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [224ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [225ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [226ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [226ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [602ms] refreshing cache from master, [602ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [604ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@783224828(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@395bbb2d -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1551535121 after retry RPC@728149197) -> (continuation of Deferred@1149439492 after retry RPC@1511899805) -> (continuation of Deferred@1247727768 after retry RPC@1240449293) -> (continuation of Deferred@679969717 after retry RPC@534239233), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2d2b4803 -> wakeup thread Time-limited test -> (continuation of Deferred@1551535121 after retry RPC@728149197) -> (continuation of Deferred@1149439492 after retry RPC@1511899805) -> (continuation of Deferred@1247727768 after retry RPC@1240449293) -> (continuation of Deferred@679969717 after retry RPC@534239233))) after lookup
20:07:40.279 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:40.280 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:40.280 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:40.280 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:41.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.014837 10929 raft_consensus.cc:493] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ab092e11f74445fbbe7a4c29511b76b)
20:07:41.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.014947 10929 raft_consensus.cc:515] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:41.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.015273 10929 leader_election.cc:290] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709), 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897)
20:07:41.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:41.015770 10614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.230.129:42709: connect: Connection refused (error 111)
20:07:41.016 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:41.016266 10614 leader_election.cc:336] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709): Network error: Client connection negotiation failed: client connection to 127.8.230.129:42709: connect: Connection refused (error 111)
20:07:41.019 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.019061 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "95c1f61bee2d40849c44869e11134c4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 28 } } ignore_live_leader: false dest_uuid: "57af19dfdfce4211824a07be0443b43a" is_pre_election: true
20:07:41.019 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.019229 10869 raft_consensus.cc:2468] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95c1f61bee2d40849c44869e11134c4a in term 6.
20:07:41.019 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.019524 10615 leader_election.cc:304] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57af19dfdfce4211824a07be0443b43a, 95c1f61bee2d40849c44869e11134c4a; no voters: 3ab092e11f74445fbbe7a4c29511b76b
20:07:41.019 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.019778 10929 raft_consensus.cc:2804] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 7 FOLLOWER]: Leader pre-election won for term 8
20:07:41.020 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.019848 10929 raft_consensus.cc:493] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3ab092e11f74445fbbe7a4c29511b76b)
20:07:41.020 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.019865 10929 raft_consensus.cc:3060] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 7 FOLLOWER]: Advancing to term 8
20:07:41.020 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.020494 10929 raft_consensus.cc:515] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:41.020 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.020685 10929 leader_election.cc:290] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [CANDIDATE]: Term 8 election: Requested vote from peers 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709), 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897)
20:07:41.021 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.020937 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "95c1f61bee2d40849c44869e11134c4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 28 } } ignore_live_leader: false dest_uuid: "57af19dfdfce4211824a07be0443b43a"
20:07:41.021 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.021020 10869 raft_consensus.cc:3060] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 6 FOLLOWER]: Advancing to term 8
20:07:41.021 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:41.021387 10614 leader_election.cc:336] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709): Network error: Client connection negotiation failed: client connection to 127.8.230.129:42709: connect: Connection refused (error 111)
20:07:41.022 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.022343 10869 raft_consensus.cc:2468] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95c1f61bee2d40849c44869e11134c4a in term 8.
20:07:41.022 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.022637 10615 leader_election.cc:304] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57af19dfdfce4211824a07be0443b43a, 95c1f61bee2d40849c44869e11134c4a; no voters: 3ab092e11f74445fbbe7a4c29511b76b
20:07:41.022 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.022761 10929 raft_consensus.cc:2804] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 8 FOLLOWER]: Leader election won for term 8
20:07:41.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.022975 10929 raft_consensus.cc:697] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 8 LEADER]: Becoming Leader. State: Replica: 95c1f61bee2d40849c44869e11134c4a, State: Running, Role: LEADER
20:07:41.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.023114 10929 consensus_queue.cc:237] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 7.28, Last appended by leader: 28, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:41.024 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.023917  9300 catalog_manager.cc:5696] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a reported cstate change: term changed from 7 to 8, leader changed from 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129) to 95c1f61bee2d40849c44869e11134c4a (127.8.230.131). New cstate: current_term: 8 leader_uuid: "95c1f61bee2d40849c44869e11134c4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } health_report { overall_health: HEALTHY } } }
20:07:41.238 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:41.238 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)[L]]}]
20:07:41.238 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=12, sequence_id=22, TimeoutTracker(timeout=30000, elapsed=1565), Traces: [0ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [2ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [21ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [24ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [25ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [27ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [27ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [44ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [45ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [47ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [47ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [82ms] refreshing cache from master, [82ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [84ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [85ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [87ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [87ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [221ms] refreshing cache from master, [221ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [224ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [225ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [226ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [226ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [602ms] refreshing cache from master, [602ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [604ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [606ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [607ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [607ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [1562ms] refreshing cache from master, [1562ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [1564ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@783224828(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@395bbb2d -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1551535121 after retry RPC@728149197) -> (continuation of Deferred@1149439492 after retry RPC@1511899805) -> (continuation of Deferred@1247727768 after retry RPC@1240449293) -> (continuation of Deferred@679969717 after retry RPC@534239233) -> (continuation of Deferred@1403444434 after retry RPC@1913655671), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2d2b4803 -> wakeup thread Time-limited test -> (continuation of Deferred@1551535121 after retry RPC@728149197) -> (continuation of Deferred@1149439492 after retry RPC@1511899805) -> (continuation of Deferred@1247727768 after retry RPC@1240449293) -> (continuation of Deferred@679969717 after retry RPC@534239233) -> (continuation of Deferred@1403444434 after retry RPC@1913655671))) after lookup
20:07:41.239 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] connecting to peer
20:07:41.240 [DEBUG - kudu-nio-5] (Connection.java:757) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] Successfully connected to peer
20:07:41.274 [DEBUG - kudu-nio-5] (Negotiator.java:919) Authenticated connection [id: 0x85d18f89, L:/127.0.0.1:40296 - R:/127.8.230.131:37545] using TOKEN/null
20:07:41.276 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.276561 10869 raft_consensus.cc:1275] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 8 FOLLOWER]: Refusing update from remote peer 95c1f61bee2d40849c44869e11134c4a: Log matching property violated. Preceding OpId in replica: term: 6 index: 24. Preceding OpId from leader: term: 8 index: 30. (index mismatch)
20:07:41.277 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:41.276846 10614 consensus_peers.cc:597] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a -> Peer 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709): Couldn't send request to peer 3ab092e11f74445fbbe7a4c29511b76b. Status: Network error: Client connection negotiation failed: client connection to 127.8.230.129:42709: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
20:07:41.277 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.276940 10929 consensus_queue.cc:1048] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 24, Time since last communication: 0.000s
20:07:41.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.278443 10950 mvcc.cc:204] Tried to move back new op lower bound from 7287336384513556480 to 7287336370657730560. Current Snapshot: MvccSnapshot[applied={T|T < 7287336384513556480}]
20:07:41.279 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.279610 10948 mvcc.cc:204] Tried to move back new op lower bound from 7287336391785975808 to 7287336390751342592. Current Snapshot: MvccSnapshot[applied={T|T < 7287336391785975808}]
20:07:41.295 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.8.230.129:42709
20:07:41.295 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

20:07:41.296 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.295977  9114 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:41.296 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:41.296 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal
20:07:41.296 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data
20:07:41.296 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
20:07:41.296 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
20:07:41.296 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
20:07:41.296 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:41.296 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:41.296 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:41.296 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/logs
20:07:41.296 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data/info.pb
20:07:41.296 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:41.296 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
20:07:41.296 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
20:07:41.296 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
20:07:41.296 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
20:07:41.296 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:41.296 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:41.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:41.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
20:07:41.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
20:07:41.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.129:42709
20:07:41.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.8.230.129
20:07:41.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:41.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=44243
20:07:41.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.129
20:07:41.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:41.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:41.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:41.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
20:07:41.377 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:41.377301 10957 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
20:07:41.377 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:41.377564 10957 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
20:07:41.377 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:41.377609 10957 flags.cc:432] Enabled unsafe flag: --never_fsync=true
20:07:41.379 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:41.379626 10957 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
20:07:41.380 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:41.379738 10957 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.230.129
20:07:41.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.382067 10957 tablet_server_runner.cc:78] Tablet server non-default flags:
20:07:41.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:41.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:41.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:41.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
20:07:41.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data
20:07:41.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal
20:07:41.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:41.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:41.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.129:42709
20:07:41.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
20:07:41.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:41.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:41.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data/info.pb
20:07:41.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.129
20:07:41.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=44243
20:07:41.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:41.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
20:07:41.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.10957
20:07:41.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:41.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
20:07:41.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
20:07:41.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:41.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.8.230.129
20:07:41.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/logs
20:07:41.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:41.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
20:07:41.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
20:07:41.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
20:07:41.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
20:07:41.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 66827b15b158c37363c37baa160f29a0ca1f1e63
20:07:41.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type RELEASE
20:07:41.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 18 May 2026 19:43:16 UTC on e7f111948823
20:07:41.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 11961
20:07:41.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.382436 10957 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
20:07:41.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.382758 10957 file_cache.cc:492] Constructed file cache file cache with capacity 419430
20:07:41.386 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:41.385800 10965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:41.386 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:41.385831 10963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:41.386 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:41.385845 10962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:41.386 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.386319 10957 server_base.cc:1061] running on GCE node
20:07:41.386 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.386551 10957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
20:07:41.386 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.386853 10957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
20:07:41.388 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.388041 10957 hybrid_clock.cc:648] HybridClock initialized: now 1779134861388023 us; error 30 us; skew 500 ppm
20:07:41.389 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.389396 10957 webserver.cc:533] Webserver started at http://127.8.230.129:44243/ using document root <none> and password file <none>
20:07:41.389 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.389597 10957 fs_manager.cc:362] Metadata directory not provided
20:07:41.389 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.389638 10957 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
20:07:41.391 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.390991 10957 fs_manager.cc:714] Time spent opening directory manager: real 0.001s	user 0.001s	sys 0.000s
20:07:41.391 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.391640 10971 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
20:07:41.391 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.391804 10957 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
20:07:41.392 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.391865 10957 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data,/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal
20:07:41.392 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "3ab092e11f74445fbbe7a4c29511b76b"
20:07:41.392 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:21 on dist-test-slave-mzrs"
20:07:41.392 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.392143 10957 fs_report.cc:389] FS layout report
20:07:41.392 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
20:07:41.392 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal
20:07:41.392 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal
20:07:41.392 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data/data
20:07:41.392 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
20:07:41.392 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
20:07:41.392 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
20:07:41.392 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
20:07:41.392 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
20:07:41.392 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
20:07:41.392 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
20:07:41.392 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
20:07:41.392 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
20:07:41.392 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
20:07:41.392 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
20:07:41.416 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.415853 10957 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
20:07:41.416 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.416168 10957 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
20:07:41.416 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.416262 10957 kserver.cc:163] Server-wide thread pool size limit: 3276
20:07:41.416 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.416451 10957 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
20:07:41.417 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.417053 10978 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
20:07:41.418 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.417958 10957 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
20:07:41.418 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.418051 10957 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
20:07:41.418 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.418094 10957 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
20:07:41.418 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.418833 10957 ts_tablet_manager.cc:616] Registered 1 tablets
20:07:41.419 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.418896 10957 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
20:07:41.419 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.418921 10978 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Bootstrap starting.
20:07:41.427 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.426946 10957 rpc_server.cc:307] RPC server started. Bound to: 127.8.230.129:42709
20:07:41.427 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.427039 11085 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.230.129:42709 every 8 connection(s)
20:07:41.427 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.427415 10957 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data/info.pb
20:07:41.428 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.428525 10978 log.cc:826] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Log is configured to *not* fsync() on all Append() calls
20:07:41.431 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.431391  9114 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskx5o66C/build/release/bin/kudu as pid 10957
20:07:41.431 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
20:07:41.438 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.433836 10978 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Bootstrap replayed 1/1 log segments. Stats: ops{read=28 overwritten=0 applied=28 ignored=0} inserts{seen=21 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
20:07:41.438 [DEBUG - kudu-nio-5] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 24 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181, tablet=a4035636acf142eeb16e984e2f510768, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
20:07:41.438 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.434254 10978 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Bootstrap complete.
20:07:41.439 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.435853 10978 ts_tablet_manager.cc:1403] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Time spent bootstrapping tablet: real 0.017s	user 0.005s	sys 0.010s
20:07:41.439 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.437178 10978 raft_consensus.cc:359] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:41.439 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.437505 10978 raft_consensus.cc:740] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ab092e11f74445fbbe7a4c29511b76b, State: Initialized, Role: FOLLOWER
20:07:41.439 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.8.230.131:37545
20:07:41.439 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.437631 10978 consensus_queue.cc:260] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 28, Last appended: 7.28, Last appended by leader: 28, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:41.439 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.437933 10978 ts_tablet_manager.cc:1434] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Time spent starting tablet: real 0.002s	user 0.003s	sys 0.000s
20:07:41.439 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 2
  }
}

20:07:41.439 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.439599  9114 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskx5o66C/build/release/bin/kudu with pid 10599
20:07:41.442 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.442169 11088 heartbeater.cc:344] Connected to a master server at 127.8.230.189:44351
20:07:41.442 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.442369 11088 heartbeater.cc:461] Registering TS with master...
20:07:41.442 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.442608 11088 heartbeater.cc:507] Master 127.8.230.189:44351 requested a full tablet report, sending...
20:07:41.443 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.443264  9227 ts_manager.cc:194] Re-registered known tserver with Master: 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709)
20:07:41.443 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.442154 11087 heartbeater.cc:344] Connected to a master server at 127.8.230.190:34269
20:07:41.443 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.443202 11086 heartbeater.cc:344] Connected to a master server at 127.8.230.188:39477
20:07:41.443 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.443403 11086 heartbeater.cc:461] Registering TS with master...
20:07:41.443 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.443522 11086 heartbeater.cc:507] Master 127.8.230.188:39477 requested a full tablet report, sending...
20:07:41.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.443728 11087 heartbeater.cc:461] Registering TS with master...
20:07:41.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.443830 11087 heartbeater.cc:507] Master 127.8.230.190:34269 requested a full tablet report, sending...
20:07:41.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.443872  9300 ts_manager.cc:194] Re-registered known tserver with Master: 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709)
20:07:41.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.444187  9154 ts_manager.cc:194] Re-registered known tserver with Master: 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709)
20:07:41.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.444397  9300 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.8.230.129:32867
20:07:41.445 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:41.445122 11086 heartbeater.cc:499] Master 127.8.230.188:39477 was elected leader, sending a full tablet report...
20:07:41.449 [DEBUG - kudu-nio-5] (Connection.java:239) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] handling channelInactive
20:07:41.450 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] cleaning up while in state READY due to: connection closed
20:07:41.451 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
20:07:41.452 [DEBUG - Time-limited test] (Connection.java:748) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] connecting to peer
20:07:41.453 [INFO - kudu-nio-6] (Connection.java:429) Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:41.453 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] cleaning up while in state CONNECTING due to: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:41.453 [INFO - kudu-nio-6] (AsyncKuduClient.java:2355) Invalidating location 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:41.477 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:41.477 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)[L]]}]
20:07:41.477 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=2, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=26), Traces: [0ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [2ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [3ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [23ms] refreshing cache from master, [23ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [25ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@515402500(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@6c514a0d -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@75ac8c36 -> wakeup thread Time-limited test)) after lookup
20:07:41.478 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] connecting to peer
20:07:41.479 [INFO - kudu-nio-7] (Connection.java:429) Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:41.479 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] cleaning up while in state CONNECTING due to: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:41.479 [INFO - kudu-nio-7] (AsyncKuduClient.java:2355) Invalidating location 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:41.497 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:41.497 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)[L]]}]
20:07:41.498 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=4, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=46), Traces: [0ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [2ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [3ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [23ms] refreshing cache from master, [23ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [25ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [27ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [28ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [28ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [44ms] refreshing cache from master, [44ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [46ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@515402500(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@6c514a0d -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1581017997 after retry RPC@1340117312), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@75ac8c36 -> wakeup thread Time-limited test -> (continuation of Deferred@1581017997 after retry RPC@1340117312))) after lookup
20:07:41.498 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] connecting to peer
20:07:41.499 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:41.499 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] cleaning up while in state CONNECTING due to: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:41.499 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:41.517 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:41.517 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)[L]]}]
20:07:41.518 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=6, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=66), Traces: [0ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [2ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [3ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [23ms] refreshing cache from master, [23ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [25ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [27ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [28ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [28ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [44ms] refreshing cache from master, [44ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [46ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [47ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [48ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [49ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [64ms] refreshing cache from master, [64ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [66ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@515402500(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@6c514a0d -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1581017997 after retry RPC@1340117312) -> (continuation of Deferred@1108302502 after retry RPC@154052918), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@75ac8c36 -> wakeup thread Time-limited test -> (continuation of Deferred@1581017997 after retry RPC@1340117312) -> (continuation of Deferred@1108302502 after retry RPC@154052918))) after lookup
20:07:41.519 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] connecting to peer
20:07:41.520 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:41.520 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] cleaning up while in state CONNECTING due to: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:41.520 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:41.577 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:41.577 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)[L]]}]
20:07:41.578 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=8, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=126), Traces: [0ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [2ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [3ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [23ms] refreshing cache from master, [23ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [25ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [27ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [28ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [28ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [44ms] refreshing cache from master, [44ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [46ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [47ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [48ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [49ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [64ms] refreshing cache from master, [64ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [66ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [68ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [69ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [69ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [124ms] refreshing cache from master, [124ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [126ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@515402500(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@6c514a0d -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1581017997 after retry RPC@1340117312) -> (continuation of Deferred@1108302502 after retry RPC@154052918) -> (continuation of Deferred@1926988608 after retry RPC@247522100), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@75ac8c36 -> wakeup thread Time-limited test -> (continuation of Deferred@1581017997 after retry RPC@1340117312) -> (continuation of Deferred@1108302502 after retry RPC@154052918) -> (continuation of Deferred@1926988608 after retry RPC@247522100))) after lookup
20:07:41.578 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] connecting to peer
20:07:41.579 [INFO - kudu-nio-2] (Connection.java:429) Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:41.579 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] cleaning up while in state CONNECTING due to: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:41.580 [INFO - kudu-nio-2] (AsyncKuduClient.java:2355) Invalidating location 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:41.857 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:41.857 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)[L]]}]
20:07:41.857 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=10, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=406), Traces: [0ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [2ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [3ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [23ms] refreshing cache from master, [23ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [25ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [27ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [28ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [28ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [44ms] refreshing cache from master, [44ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [46ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [47ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [48ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [49ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [64ms] refreshing cache from master, [64ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [66ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [68ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [69ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [69ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [124ms] refreshing cache from master, [124ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [126ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [127ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [129ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [129ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [403ms] refreshing cache from master, [403ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [405ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@515402500(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@6c514a0d -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1581017997 after retry RPC@1340117312) -> (continuation of Deferred@1108302502 after retry RPC@154052918) -> (continuation of Deferred@1926988608 after retry RPC@247522100) -> (continuation of Deferred@1210929734 after retry RPC@1991086665), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@75ac8c36 -> wakeup thread Time-limited test -> (continuation of Deferred@1581017997 after retry RPC@1340117312) -> (continuation of Deferred@1108302502 after retry RPC@154052918) -> (continuation of Deferred@1926988608 after retry RPC@247522100) -> (continuation of Deferred@1210929734 after retry RPC@1991086665))) after lookup
20:07:41.858 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] connecting to peer
20:07:41.859 [INFO - kudu-nio-3] (Connection.java:429) Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:41.859 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] cleaning up while in state CONNECTING due to: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:41.859 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:42.379 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:42.379 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)[L]]}]
20:07:42.380 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=12, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=929), Traces: [0ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [2ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [3ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [23ms] refreshing cache from master, [23ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [25ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [27ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [28ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [28ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [44ms] refreshing cache from master, [44ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [46ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [47ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [48ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [49ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [64ms] refreshing cache from master, [64ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [66ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [68ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [69ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [69ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [124ms] refreshing cache from master, [124ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [126ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [127ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [129ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [129ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [403ms] refreshing cache from master, [403ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [405ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [407ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [408ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [408ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [923ms] refreshing cache from master, [924ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [927ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@515402500(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@6c514a0d -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1581017997 after retry RPC@1340117312) -> (continuation of Deferred@1108302502 after retry RPC@154052918) -> (continuation of Deferred@1926988608 after retry RPC@247522100) -> (continuation of Deferred@1210929734 after retry RPC@1991086665) -> (continuation of Deferred@1503845170 after retry RPC@1132930453), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@75ac8c36 -> wakeup thread Time-limited test -> (continuation of Deferred@1581017997 after retry RPC@1340117312) -> (continuation of Deferred@1108302502 after retry RPC@154052918) -> (continuation of Deferred@1926988608 after retry RPC@247522100) -> (continuation of Deferred@1210929734 after retry RPC@1991086665) -> (continuation of Deferred@1503845170 after retry RPC@1132930453))) after lookup
20:07:42.381 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] connecting to peer
20:07:42.383 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:42.383 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] cleaning up while in state CONNECTING due to: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:42.383 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:42.418 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:42.418 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)[L]]}]
20:07:42.419 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=14, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=967), Traces: [0ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [2ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [3ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [23ms] refreshing cache from master, [23ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [25ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [27ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [28ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [28ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [44ms] refreshing cache from master, [44ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [46ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [47ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [48ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [49ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [64ms] refreshing cache from master, [64ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [66ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [68ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [69ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [69ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [124ms] refreshing cache from master, [124ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [126ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [127ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [129ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [129ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [403ms] refreshing cache from master, [403ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [405ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [407ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [408ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [408ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [923ms] refreshing cache from master, [924ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [927ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [930ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [932ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [932ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [963ms] refreshing cache from master, [964ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [966ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@515402500(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@6c514a0d -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1581017997 after retry RPC@1340117312) -> (continuation of Deferred@1108302502 after retry RPC@154052918) -> (continuation of Deferred@1926988608 after retry RPC@247522100) -> (continuation of Deferred@1210929734 after retry RPC@1991086665) -> (continuation of Deferred@1503845170 after retry RPC@1132930453) -> (continuation of Deferred@57294857 after retry RPC@2033940092), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@75ac8c36 -> wakeup thread Time-limited test -> (continuation of Deferred@1581017997 after retry RPC@1340117312) -> (continuation of Deferred@1108302502 after retry RPC@154052918) -> (continuation of Deferred@1926988608 after retry RPC@247522100) -> (continuation of Deferred@1210929734 after retry RPC@1991086665) -> (continuation of Deferred@1503845170 after retry RPC@1132930453) -> (continuation of Deferred@57294857 after retry RPC@2033940092))) after lookup
20:07:42.420 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] connecting to peer
20:07:42.421 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:42.421 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] cleaning up while in state CONNECTING due to: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:42.421 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545
20:07:42.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:42.794687 11100 raft_consensus.cc:493] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 95c1f61bee2d40849c44869e11134c4a)
20:07:42.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:42.794795 11100 raft_consensus.cc:515] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:42.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:42.795104 11100 leader_election.cc:290] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709), 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:42.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:42.795598 10804 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.230.131:37545: connect: Connection refused (error 111)
20:07:42.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:42.796267 10804 leader_election.cc:336] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545): Network error: Client connection negotiation failed: client connection to 127.8.230.131:37545: connect: Connection refused (error 111)
20:07:42.799 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:42.798976 11040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "57af19dfdfce4211824a07be0443b43a" candidate_term: 9 candidate_status { last_received { term: 8 index: 32 } } ignore_live_leader: false dest_uuid: "3ab092e11f74445fbbe7a4c29511b76b" is_pre_election: true
20:07:42.799 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:42.799208 11040 raft_consensus.cc:2468] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57af19dfdfce4211824a07be0443b43a in term 7.
20:07:42.799 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:42.799502 10801 leader_election.cc:304] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ab092e11f74445fbbe7a4c29511b76b, 57af19dfdfce4211824a07be0443b43a; no voters: 95c1f61bee2d40849c44869e11134c4a
20:07:42.799 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:42.799674 11100 raft_consensus.cc:2804] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 8 FOLLOWER]: Leader pre-election won for term 9
20:07:42.799 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:42.799767 11100 raft_consensus.cc:493] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 95c1f61bee2d40849c44869e11134c4a)
20:07:42.799 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:42.799793 11100 raft_consensus.cc:3060] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 8 FOLLOWER]: Advancing to term 9
20:07:42.800 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:42.800479 11100 raft_consensus.cc:515] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:42.800 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:42.800701 11100 leader_election.cc:290] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [CANDIDATE]: Term 9 election: Requested vote from peers 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709), 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:42.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:42.800988 11040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "57af19dfdfce4211824a07be0443b43a" candidate_term: 9 candidate_status { last_received { term: 8 index: 32 } } ignore_live_leader: false dest_uuid: "3ab092e11f74445fbbe7a4c29511b76b"
20:07:42.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:42.801116 11040 raft_consensus.cc:3060] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 7 FOLLOWER]: Advancing to term 9
20:07:42.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:42.801405 10804 leader_election.cc:336] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545): Network error: Client connection negotiation failed: client connection to 127.8.230.131:37545: connect: Connection refused (error 111)
20:07:42.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:42.802400 11040 raft_consensus.cc:2468] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57af19dfdfce4211824a07be0443b43a in term 9.
20:07:42.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:42.802695 10801 leader_election.cc:304] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ab092e11f74445fbbe7a4c29511b76b, 57af19dfdfce4211824a07be0443b43a; no voters: 95c1f61bee2d40849c44869e11134c4a
20:07:42.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:42.802824 11100 raft_consensus.cc:2804] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 9 FOLLOWER]: Leader election won for term 9
20:07:42.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:42.803054 11100 raft_consensus.cc:697] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 9 LEADER]: Becoming Leader. State: Replica: 57af19dfdfce4211824a07be0443b43a, State: Running, Role: LEADER
20:07:42.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:42.803160 11100 consensus_queue.cc:237] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 8.32, Last appended by leader: 32, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:42.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:42.804039  9300 catalog_manager.cc:5696] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a reported cstate change: term changed from 8 to 9, leader changed from 95c1f61bee2d40849c44869e11134c4a (127.8.230.131) to 57af19dfdfce4211824a07be0443b43a (127.8.230.130). New cstate: current_term: 9 leader_uuid: "57af19dfdfce4211824a07be0443b43a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } health_report { overall_health: UNKNOWN } } }
20:07:43.348 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:43.348277 10804 consensus_peers.cc:597] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a -> Peer 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545): Couldn't send request to peer 95c1f61bee2d40849c44869e11134c4a. Status: Network error: Client connection negotiation failed: client connection to 127.8.230.131:37545: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
20:07:43.386 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.386627 11040 raft_consensus.cc:1275] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 9 FOLLOWER]: Refusing update from remote peer 57af19dfdfce4211824a07be0443b43a: Log matching property violated. Preceding OpId in replica: term: 7 index: 28. Preceding OpId from leader: term: 9 index: 33. (index mismatch)
20:07:43.387 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.387023 11118 consensus_queue.cc:1048] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 28, Time since last communication: 0.000s
20:07:43.659 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:43.659 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)[L],95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:43.660 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=16, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=2208), Traces: [0ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [2ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [3ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [23ms] refreshing cache from master, [23ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [25ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [27ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [28ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [28ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [44ms] refreshing cache from master, [44ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [46ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [47ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [48ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [49ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [64ms] refreshing cache from master, [64ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [66ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [68ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [69ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [69ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [124ms] refreshing cache from master, [124ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [126ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [127ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [129ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [129ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [403ms] refreshing cache from master, [403ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [405ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [407ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [408ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [408ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [923ms] refreshing cache from master, [924ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [927ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [930ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [932ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [932ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [963ms] refreshing cache from master, [964ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [966ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [968ms] sending RPC to server 95c1f61bee2d40849c44869e11134c4a, [970ms] received response from server 95c1f61bee2d40849c44869e11134c4a: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [970ms] delaying RPC due to: Network error: Failed to connect to peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545): Connection refused: /127.8.230.131:37545, [2204ms] refreshing cache from master, [2204ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [2208ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@515402500(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@6c514a0d -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1581017997 after retry RPC@1340117312) -> (continuation of Deferred@1108302502 after retry RPC@154052918) -> (continuation of Deferred@1926988608 after retry RPC@247522100) -> (continuation of Deferred@1210929734 after retry RPC@1991086665) -> (continuation of Deferred@1503845170 after retry RPC@1132930453) -> (continuation of Deferred@57294857 after retry RPC@2033940092) -> (continuation of Deferred@347036542 after retry RPC@914708225), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@75ac8c36 -> wakeup thread Time-limited test -> (continuation of Deferred@1581017997 after retry RPC@1340117312) -> (continuation of Deferred@1108302502 after retry RPC@154052918) -> (continuation of Deferred@1926988608 after retry RPC@247522100) -> (continuation of Deferred@1210929734 after retry RPC@1991086665) -> (continuation of Deferred@1503845170 after retry RPC@1132930453) -> (continuation of Deferred@57294857 after retry RPC@2033940092) -> (continuation of Deferred@347036542 after retry RPC@914708225))) after lookup
20:07:43.661 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] connecting to peer
20:07:43.663 [DEBUG - kudu-nio-6] (Connection.java:757) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] Successfully connected to peer
20:07:43.684 [DEBUG - kudu-nio-6] (Negotiator.java:919) Authenticated connection [id: 0xdd18873a, L:/127.0.0.1:40628 - R:/127.8.230.130:37897] using TOKEN/null
20:07:43.698 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.8.230.131:37545
20:07:43.699 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 2
  }
}

20:07:43.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.699662  9114 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:43.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:43.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/wal
20:07:43.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/logs
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data/info.pb
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.131:37545
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.8.230.131
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=45551
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.131
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
20:07:43.788 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:43.788532 11132 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
20:07:43.789 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:43.788774 11132 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
20:07:43.789 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:43.788808 11132 flags.cc:432] Enabled unsafe flag: --never_fsync=true
20:07:43.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:43.790925 11132 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
20:07:43.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:43.791038 11132 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.230.131
20:07:43.793 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.793359 11132 tablet_server_runner.cc:78] Tablet server non-default flags:
20:07:43.793 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:43.793 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:43.793 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:43.793 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
20:07:43.793 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data
20:07:43.793 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/wal
20:07:43.793 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:43.793 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:43.793 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.131:37545
20:07:43.793 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
20:07:43.793 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:43.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:43.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data/info.pb
20:07:43.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.131
20:07:43.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=45551
20:07:43.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:43.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
20:07:43.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.11132
20:07:43.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:43.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
20:07:43.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
20:07:43.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:43.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.8.230.131
20:07:43.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/logs
20:07:43.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:43.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
20:07:43.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
20:07:43.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
20:07:43.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
20:07:43.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 66827b15b158c37363c37baa160f29a0ca1f1e63
20:07:43.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type RELEASE
20:07:43.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 18 May 2026 19:43:16 UTC on e7f111948823
20:07:43.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 11961
20:07:43.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.793725 11132 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
20:07:43.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.794020 11132 file_cache.cc:492] Constructed file cache file cache with capacity 419430
20:07:43.797 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:43.797060 11137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:43.797 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:43.797178 11138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:43.797 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.797355 11132 server_base.cc:1061] running on GCE node
20:07:43.797 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:43.797077 11140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:43.798 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.797899 11132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
20:07:43.798 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.798130 11132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
20:07:43.799 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.799283 11132 hybrid_clock.cc:648] HybridClock initialized: now 1779134863799260 us; error 40 us; skew 500 ppm
20:07:43.800 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.800519 11132 webserver.cc:533] Webserver started at http://127.8.230.131:45551/ using document root <none> and password file <none>
20:07:43.800 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.800695 11132 fs_manager.cc:362] Metadata directory not provided
20:07:43.800 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.800740 11132 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
20:07:43.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.801964 11132 fs_manager.cc:714] Time spent opening directory manager: real 0.001s	user 0.001s	sys 0.000s
20:07:43.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.802883 11146 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
20:07:43.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.803160 11132 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.000s
20:07:43.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.803228 11132 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data,/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/wal
20:07:43.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "95c1f61bee2d40849c44869e11134c4a"
20:07:43.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:21 on dist-test-slave-mzrs"
20:07:43.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.803534 11132 fs_report.cc:389] FS layout report
20:07:43.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
20:07:43.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/wal
20:07:43.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/wal
20:07:43.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data/data
20:07:43.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
20:07:43.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
20:07:43.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
20:07:43.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
20:07:43.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
20:07:43.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
20:07:43.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
20:07:43.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
20:07:43.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
20:07:43.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
20:07:43.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
20:07:43.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.822422 11132 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
20:07:43.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.822734 11132 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
20:07:43.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.822827 11132 kserver.cc:163] Server-wide thread pool size limit: 3276
20:07:43.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.823009 11132 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
20:07:43.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.823462 11153 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
20:07:43.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.824462 11132 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
20:07:43.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.824522 11132 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
20:07:43.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.824548 11132 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
20:07:43.825 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.825119 11132 ts_tablet_manager.cc:616] Registered 1 tablets
20:07:43.825 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.825153 11132 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
20:07:43.825 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.825278 11153 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a: Bootstrap starting.
20:07:43.832 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.832634 11132 rpc_server.cc:307] RPC server started. Bound to: 127.8.230.131:37545
20:07:43.833 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.832666 11260 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.230.131:37545 every 8 connection(s)
20:07:43.833 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.833045 11132 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-2/data/info.pb
20:07:43.836 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.835835  9114 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskx5o66C/build/release/bin/kudu as pid 11132
20:07:43.836 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
20:07:43.837 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.837025 11153 log.cc:826] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a: Log is configured to *not* fsync() on all Append() calls
20:07:43.840 [DEBUG - kudu-nio-6] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 27 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181, tablet=a4035636acf142eeb16e984e2f510768, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
20:07:43.841 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.8.230.130:37897
20:07:43.841 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 1
  }
}

20:07:43.842 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.841625 11153 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a: Bootstrap replayed 1/1 log segments. Stats: ops{read=32 overwritten=0 applied=32 ignored=0} inserts{seen=24 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
20:07:43.842 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.841974  9114 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskx5o66C/build/release/bin/kudu with pid 10786
20:07:43.842 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.842001 11153 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a: Bootstrap complete.
20:07:43.843 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.843748 11153 ts_tablet_manager.cc:1403] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.009s
20:07:43.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.844529 11263 heartbeater.cc:344] Connected to a master server at 127.8.230.189:44351
20:07:43.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.844647 11263 heartbeater.cc:461] Registering TS with master...
20:07:43.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.844882 11263 heartbeater.cc:507] Master 127.8.230.189:44351 requested a full tablet report, sending...
20:07:43.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.845323 11153 raft_consensus.cc:359] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 8 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:43.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.845691 11153 raft_consensus.cc:740] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 8 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95c1f61bee2d40849c44869e11134c4a, State: Initialized, Role: FOLLOWER
20:07:43.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.845862  9228 ts_manager.cc:194] Re-registered known tserver with Master: 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:43.848 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.845841 11153 consensus_queue.cc:260] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 32, Last appended: 8.32, Last appended by leader: 32, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:43.848 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.845713 11262 heartbeater.cc:344] Connected to a master server at 127.8.230.190:34269
20:07:43.848 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.845999 11262 heartbeater.cc:461] Registering TS with master...
20:07:43.848 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.846122 11262 heartbeater.cc:507] Master 127.8.230.190:34269 requested a full tablet report, sending...
20:07:43.848 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.846118 11153 ts_tablet_manager.cc:1434] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.002s
20:07:43.848 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.846736  9154 ts_manager.cc:194] Re-registered known tserver with Master: 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:43.849 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.849316 11261 heartbeater.cc:344] Connected to a master server at 127.8.230.188:39477
20:07:43.849 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.849402 11261 heartbeater.cc:461] Registering TS with master...
20:07:43.849 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.849507 11261 heartbeater.cc:507] Master 127.8.230.188:39477 requested a full tablet report, sending...
20:07:43.849 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.849817  9301 ts_manager.cc:194] Re-registered known tserver with Master: 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:43.850 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.850195  9301 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.8.230.131:53103
20:07:43.851 [DEBUG - kudu-nio-6] (Connection.java:239) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] handling channelInactive
20:07:43.851 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:43.851073 11261 heartbeater.cc:499] Master 127.8.230.188:39477 was elected leader, sending a full tablet report...
20:07:43.851 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] cleaning up while in state READY due to: connection closed
20:07:43.852 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
20:07:43.852 [DEBUG - Time-limited test] (Connection.java:748) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] connecting to peer
20:07:43.854 [INFO - kudu-nio-7] (Connection.java:429) Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:43.854 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] cleaning up while in state CONNECTING due to: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:43.854 [INFO - kudu-nio-7] (AsyncKuduClient.java:2355) Invalidating location 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:43.856 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:43.857 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)[L],95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:43.857 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=2, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=4), Traces: [0ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [2ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [2ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [2ms] refreshing cache from master, [2ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [4ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@369330129(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@a22e74f -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2edd42fb -> wakeup thread Time-limited test)) after lookup
20:07:43.857 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] connecting to peer
20:07:43.858 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:43.858 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] cleaning up while in state CONNECTING due to: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:43.858 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:43.877 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:43.877 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)[L],95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:43.878 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=4, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=25), Traces: [0ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [2ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [2ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [2ms] refreshing cache from master, [2ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [4ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [5ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [6ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [6ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [23ms] refreshing cache from master, [23ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [25ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@369330129(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@a22e74f -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@192720256 after retry RPC@1332935822), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2edd42fb -> wakeup thread Time-limited test -> (continuation of Deferred@192720256 after retry RPC@1332935822))) after lookup
20:07:43.878 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] connecting to peer
20:07:43.879 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:43.879 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] cleaning up while in state CONNECTING due to: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:43.879 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:43.897 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:43.897 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)[L],95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:43.897 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=6, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=45), Traces: [0ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [2ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [2ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [2ms] refreshing cache from master, [2ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [4ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [5ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [6ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [6ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [23ms] refreshing cache from master, [23ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [25ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [26ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [27ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [28ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [42ms] refreshing cache from master, [43ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [44ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@369330129(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@a22e74f -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@192720256 after retry RPC@1332935822) -> (continuation of Deferred@310540143 after retry RPC@1246527293), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2edd42fb -> wakeup thread Time-limited test -> (continuation of Deferred@192720256 after retry RPC@1332935822) -> (continuation of Deferred@310540143 after retry RPC@1246527293))) after lookup
20:07:43.898 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] connecting to peer
20:07:43.899 [INFO - kudu-nio-2] (Connection.java:429) Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:43.899 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] cleaning up while in state CONNECTING due to: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:43.899 [INFO - kudu-nio-2] (AsyncKuduClient.java:2355) Invalidating location 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:43.997 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:43.997 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)[L],95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:43.998 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=8, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=145), Traces: [0ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [2ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [2ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [2ms] refreshing cache from master, [2ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [4ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [5ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [6ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [6ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [23ms] refreshing cache from master, [23ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [25ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [26ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [27ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [28ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [42ms] refreshing cache from master, [43ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [44ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [45ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [47ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [47ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [143ms] refreshing cache from master, [143ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [145ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@369330129(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@a22e74f -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@192720256 after retry RPC@1332935822) -> (continuation of Deferred@310540143 after retry RPC@1246527293) -> (continuation of Deferred@261118963 after retry RPC@2004709619), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2edd42fb -> wakeup thread Time-limited test -> (continuation of Deferred@192720256 after retry RPC@1332935822) -> (continuation of Deferred@310540143 after retry RPC@1246527293) -> (continuation of Deferred@261118963 after retry RPC@2004709619))) after lookup
20:07:43.999 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] connecting to peer
20:07:44.000 [INFO - kudu-nio-3] (Connection.java:429) Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:44.000 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] cleaning up while in state CONNECTING due to: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:44.000 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:44.077 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:44.077 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)[L],95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:44.077 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=10, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=225), Traces: [0ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [2ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [2ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [2ms] refreshing cache from master, [2ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [4ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [5ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [6ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [6ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [23ms] refreshing cache from master, [23ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [25ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [26ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [27ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [28ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [42ms] refreshing cache from master, [43ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [44ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [45ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [47ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [47ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [143ms] refreshing cache from master, [143ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [145ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [146ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [148ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [148ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [222ms] refreshing cache from master, [222ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [224ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@369330129(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@a22e74f -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@192720256 after retry RPC@1332935822) -> (continuation of Deferred@310540143 after retry RPC@1246527293) -> (continuation of Deferred@261118963 after retry RPC@2004709619) -> (continuation of Deferred@855446682 after retry RPC@773525726), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2edd42fb -> wakeup thread Time-limited test -> (continuation of Deferred@192720256 after retry RPC@1332935822) -> (continuation of Deferred@310540143 after retry RPC@1246527293) -> (continuation of Deferred@261118963 after retry RPC@2004709619) -> (continuation of Deferred@855446682 after retry RPC@773525726))) after lookup
20:07:44.078 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] connecting to peer
20:07:44.079 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:44.079 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] cleaning up while in state CONNECTING due to: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:44.079 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:45.177 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:45.177 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)[L],95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:45.178 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=12, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=1325), Traces: [0ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [2ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [2ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [2ms] refreshing cache from master, [2ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [4ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [5ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [6ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [6ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [23ms] refreshing cache from master, [23ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [25ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [26ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [27ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [28ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [42ms] refreshing cache from master, [43ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [44ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [45ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [47ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [47ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [143ms] refreshing cache from master, [143ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [145ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [146ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [148ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [148ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [222ms] refreshing cache from master, [222ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [224ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [225ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [227ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [227ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [1323ms] refreshing cache from master, [1323ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [1325ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@369330129(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@a22e74f -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@192720256 after retry RPC@1332935822) -> (continuation of Deferred@310540143 after retry RPC@1246527293) -> (continuation of Deferred@261118963 after retry RPC@2004709619) -> (continuation of Deferred@855446682 after retry RPC@773525726) -> (continuation of Deferred@912005799 after retry RPC@1085763922), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2edd42fb -> wakeup thread Time-limited test -> (continuation of Deferred@192720256 after retry RPC@1332935822) -> (continuation of Deferred@310540143 after retry RPC@1246527293) -> (continuation of Deferred@261118963 after retry RPC@2004709619) -> (continuation of Deferred@855446682 after retry RPC@773525726) -> (continuation of Deferred@912005799 after retry RPC@1085763922))) after lookup
20:07:45.178 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] connecting to peer
20:07:45.179 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:45.180 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] cleaning up while in state CONNECTING due to: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:45.180 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897
20:07:45.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:45.197427 11274 raft_consensus.cc:493] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 57af19dfdfce4211824a07be0443b43a)
20:07:45.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:45.197544 11274 raft_consensus.cc:515] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:45.198 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:45.197926 11274 leader_election.cc:290] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897), 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:45.198 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:45.198422 10973 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.230.130:37897: connect: Connection refused (error 111)
20:07:45.199 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:45.198969 10973 leader_election.cc:336] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897): Network error: Client connection negotiation failed: client connection to 127.8.230.130:37897: connect: Connection refused (error 111)
20:07:45.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:45.201551 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "3ab092e11f74445fbbe7a4c29511b76b" candidate_term: 10 candidate_status { last_received { term: 9 index: 36 } } ignore_live_leader: false dest_uuid: "95c1f61bee2d40849c44869e11134c4a" is_pre_election: true
20:07:45.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:45.201730 11215 raft_consensus.cc:2468] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ab092e11f74445fbbe7a4c29511b76b in term 8.
20:07:45.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:45.202127 10975 leader_election.cc:304] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ab092e11f74445fbbe7a4c29511b76b, 95c1f61bee2d40849c44869e11134c4a; no voters: 57af19dfdfce4211824a07be0443b43a
20:07:45.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:45.202307 11274 raft_consensus.cc:2804] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 9 FOLLOWER]: Leader pre-election won for term 10
20:07:45.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:45.202360 11274 raft_consensus.cc:493] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 57af19dfdfce4211824a07be0443b43a)
20:07:45.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:45.202390 11274 raft_consensus.cc:3060] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 9 FOLLOWER]: Advancing to term 10
20:07:45.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:45.203027 11274 raft_consensus.cc:515] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:45.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:45.203208 11274 leader_election.cc:290] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [CANDIDATE]: Term 10 election: Requested vote from peers 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897), 95c1f61bee2d40849c44869e11134c4a (127.8.230.131:37545)
20:07:45.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:45.203473 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "3ab092e11f74445fbbe7a4c29511b76b" candidate_term: 10 candidate_status { last_received { term: 9 index: 36 } } ignore_live_leader: false dest_uuid: "95c1f61bee2d40849c44869e11134c4a"
20:07:45.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:45.203598 11215 raft_consensus.cc:3060] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 8 FOLLOWER]: Advancing to term 10
20:07:45.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:45.204378 10973 leader_election.cc:336] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897): Network error: Client connection negotiation failed: client connection to 127.8.230.130:37897: connect: Connection refused (error 111)
20:07:45.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:45.204741 11215 raft_consensus.cc:2468] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab092e11f74445fbbe7a4c29511b76b in term 10.
20:07:45.205 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:45.204964 10975 leader_election.cc:304] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ab092e11f74445fbbe7a4c29511b76b, 95c1f61bee2d40849c44869e11134c4a; no voters: 57af19dfdfce4211824a07be0443b43a
20:07:45.205 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:45.205080 11274 raft_consensus.cc:2804] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 10 FOLLOWER]: Leader election won for term 10
20:07:45.205 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:45.205224 11274 raft_consensus.cc:697] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 10 LEADER]: Becoming Leader. State: Replica: 3ab092e11f74445fbbe7a4c29511b76b, State: Running, Role: LEADER
20:07:45.205 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:45.205314 11274 consensus_queue.cc:237] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 9.36, Last appended by leader: 36, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:45.206 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:45.206082  9301 catalog_manager.cc:5696] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b reported cstate change: term changed from 9 to 10, leader changed from 57af19dfdfce4211824a07be0443b43a (127.8.230.130) to 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129). New cstate: current_term: 10 leader_uuid: "3ab092e11f74445fbbe7a4c29511b76b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } health_report { overall_health: UNKNOWN } } }
20:07:45.675 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:45.675225 11215 raft_consensus.cc:1275] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 10 FOLLOWER]: Refusing update from remote peer 3ab092e11f74445fbbe7a4c29511b76b: Log matching property violated. Preceding OpId in replica: term: 8 index: 32. Preceding OpId from leader: term: 10 index: 37. (index mismatch)
20:07:45.675 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:45.675654 11274 consensus_queue.cc:1048] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [LEADER]: Connected to new peer: Peer: permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 32, Time since last communication: 0.000s
20:07:45.679 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:45.678782 10973 consensus_peers.cc:597] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b -> Peer 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897): Couldn't send request to peer 57af19dfdfce4211824a07be0443b43a. Status: Network error: Client connection negotiation failed: client connection to 127.8.230.130:37897: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
20:07:47.418 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:47.418 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)[L],57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:47.419 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=14, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=3566), Traces: [0ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [2ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [2ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [2ms] refreshing cache from master, [2ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [4ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [5ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [6ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [6ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [23ms] refreshing cache from master, [23ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [25ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [26ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [27ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [28ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [42ms] refreshing cache from master, [43ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [44ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [45ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [47ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [47ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [143ms] refreshing cache from master, [143ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [145ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [146ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [148ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [148ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [222ms] refreshing cache from master, [222ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [224ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [225ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [227ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [227ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [1323ms] refreshing cache from master, [1323ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [1325ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [1326ms] sending RPC to server 57af19dfdfce4211824a07be0443b43a, [1328ms] received response from server 57af19dfdfce4211824a07be0443b43a: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [1328ms] delaying RPC due to: Network error: Failed to connect to peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897): Connection refused: /127.8.230.130:37897, [3563ms] refreshing cache from master, [3563ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [3566ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@369330129(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@a22e74f -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@192720256 after retry RPC@1332935822) -> (continuation of Deferred@310540143 after retry RPC@1246527293) -> (continuation of Deferred@261118963 after retry RPC@2004709619) -> (continuation of Deferred@855446682 after retry RPC@773525726) -> (continuation of Deferred@912005799 after retry RPC@1085763922) -> (continuation of Deferred@1613121452 after retry RPC@459310875), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2edd42fb -> wakeup thread Time-limited test -> (continuation of Deferred@192720256 after retry RPC@1332935822) -> (continuation of Deferred@310540143 after retry RPC@1246527293) -> (continuation of Deferred@261118963 after retry RPC@2004709619) -> (continuation of Deferred@855446682 after retry RPC@773525726) -> (continuation of Deferred@912005799 after retry RPC@1085763922) -> (continuation of Deferred@1613121452 after retry RPC@459310875))) after lookup
20:07:47.420 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:47.421 [DEBUG - kudu-nio-6] (Connection.java:757) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] Successfully connected to peer
20:07:47.437 [DEBUG - kudu-nio-6] (Negotiator.java:919) Authenticated connection [id: 0x20e85ff4, L:/127.0.0.1:39288 - R:/127.8.230.129:42709] using TOKEN/null
20:07:47.448 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.8.230.130:37897
20:07:47.448 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 1
  }
}

20:07:47.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.449295  9114 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:47.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:47.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/wal
20:07:47.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data
20:07:47.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
20:07:47.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
20:07:47.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
20:07:47.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:47.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:47.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:47.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/logs
20:07:47.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data/info.pb
20:07:47.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:47.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
20:07:47.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
20:07:47.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
20:07:47.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
20:07:47.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:47.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:47.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:47.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
20:07:47.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
20:07:47.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.130:37897
20:07:47.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.8.230.130
20:07:47.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:47.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=43383
20:07:47.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.130
20:07:47.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:47.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:47.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:47.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
20:07:47.533 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:47.532495 11313 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
20:07:47.533 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:47.532773 11313 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
20:07:47.533 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:47.532810 11313 flags.cc:432] Enabled unsafe flag: --never_fsync=true
20:07:47.535 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:47.535310 11313 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
20:07:47.535 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:47.535437 11313 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.230.130
20:07:47.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.537474 11313 tablet_server_runner.cc:78] Tablet server non-default flags:
20:07:47.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:47.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:47.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:47.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
20:07:47.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data
20:07:47.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/wal
20:07:47.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:47.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:47.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.130:37897
20:07:47.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
20:07:47.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:47.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:47.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data/info.pb
20:07:47.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.130
20:07:47.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=43383
20:07:47.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:47.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
20:07:47.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.11313
20:07:47.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:47.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
20:07:47.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
20:07:47.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:47.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.8.230.130
20:07:47.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/logs
20:07:47.539 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:47.539 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
20:07:47.539 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
20:07:47.539 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
20:07:47.539 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
20:07:47.539 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 66827b15b158c37363c37baa160f29a0ca1f1e63
20:07:47.539 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type RELEASE
20:07:47.539 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 18 May 2026 19:43:16 UTC on e7f111948823
20:07:47.539 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 11961
20:07:47.539 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.537844 11313 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
20:07:47.539 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.538161 11313 file_cache.cc:492] Constructed file cache file cache with capacity 419430
20:07:47.541 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:47.541359 11318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:47.541 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:47.541366 11321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:47.541 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:47.541500 11319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:47.541 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.541791 11313 server_base.cc:1061] running on GCE node
20:07:47.542 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.541990 11313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
20:07:47.542 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.542313 11313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
20:07:47.543 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.543488 11313 hybrid_clock.cc:648] HybridClock initialized: now 1779134867543468 us; error 56 us; skew 500 ppm
20:07:47.545 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.544847 11313 webserver.cc:533] Webserver started at http://127.8.230.130:43383/ using document root <none> and password file <none>
20:07:47.545 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.545099 11313 fs_manager.cc:362] Metadata directory not provided
20:07:47.545 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.545188 11313 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
20:07:47.546 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.546617 11313 fs_manager.cc:714] Time spent opening directory manager: real 0.001s	user 0.000s	sys 0.001s
20:07:47.547 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.547472 11327 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
20:07:47.548 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.547906 11313 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
20:07:47.548 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.548007 11313 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data,/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/wal
20:07:47.548 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "57af19dfdfce4211824a07be0443b43a"
20:07:47.548 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:21 on dist-test-slave-mzrs"
20:07:47.548 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.548382 11313 fs_report.cc:389] FS layout report
20:07:47.548 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
20:07:47.548 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/wal
20:07:47.548 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/wal
20:07:47.548 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data/data
20:07:47.548 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
20:07:47.548 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
20:07:47.548 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
20:07:47.548 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
20:07:47.548 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
20:07:47.548 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
20:07:47.548 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
20:07:47.549 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
20:07:47.549 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
20:07:47.549 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
20:07:47.549 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
20:07:47.562 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.561640 11313 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
20:07:47.562 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.562126 11313 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
20:07:47.562 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.562368 11313 kserver.cc:163] Server-wide thread pool size limit: 3276
20:07:47.562 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.562633 11313 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
20:07:47.563 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.563236 11334 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
20:07:47.564 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.564257 11313 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
20:07:47.564 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.564342 11313 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
20:07:47.564 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.564383 11313 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
20:07:47.565 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.565181 11313 ts_tablet_manager.cc:616] Registered 1 tablets
20:07:47.565 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.565263 11313 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
20:07:47.565 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.565310 11334 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a: Bootstrap starting.
20:07:47.573 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.573308 11313 rpc_server.cc:307] RPC server started. Bound to: 127.8.230.130:37897
20:07:47.573 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.573403 11441 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.230.130:37897 every 8 connection(s)
20:07:47.574 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.573946 11313 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-1/data/info.pb
20:07:47.574 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.574509  9114 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskx5o66C/build/release/bin/kudu as pid 11313
20:07:47.574 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
20:07:47.579 [DEBUG - kudu-nio-6] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 30 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181, tablet=a4035636acf142eeb16e984e2f510768, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
20:07:47.580 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.8.230.129:42709
20:07:47.580 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

20:07:47.580 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.580775  9114 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskx5o66C/build/release/bin/kudu with pid 10957
20:07:47.583 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.582669 11442 heartbeater.cc:344] Connected to a master server at 127.8.230.188:39477
20:07:47.583 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.582798 11442 heartbeater.cc:461] Registering TS with master...
20:07:47.583 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.583025 11442 heartbeater.cc:507] Master 127.8.230.188:39477 requested a full tablet report, sending...
20:07:47.584 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.583941  9301 ts_manager.cc:194] Re-registered known tserver with Master: 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897)
20:07:47.584 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.584498  9301 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.8.230.130:53845
20:07:47.585 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.585031 11334 log.cc:826] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a: Log is configured to *not* fsync() on all Append() calls
20:07:47.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.586290 11443 heartbeater.cc:344] Connected to a master server at 127.8.230.190:34269
20:07:47.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.586385 11444 heartbeater.cc:344] Connected to a master server at 127.8.230.189:44351
20:07:47.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.586403 11443 heartbeater.cc:461] Registering TS with master...
20:07:47.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.586440 11444 heartbeater.cc:461] Registering TS with master...
20:07:47.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.586535 11443 heartbeater.cc:507] Master 127.8.230.190:34269 requested a full tablet report, sending...
20:07:47.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.586534 11444 heartbeater.cc:507] Master 127.8.230.189:44351 requested a full tablet report, sending...
20:07:47.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.586970  9154 ts_manager.cc:194] Re-registered known tserver with Master: 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897)
20:07:47.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.586978  9227 ts_manager.cc:194] Re-registered known tserver with Master: 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897)
20:07:47.590 [DEBUG - kudu-nio-6] (Connection.java:239) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] handling channelInactive
20:07:47.591 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state READY due to: connection closed
20:07:47.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.591760 11334 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a: Bootstrap replayed 1/1 log segments. Stats: ops{read=36 overwritten=0 applied=36 ignored=0} inserts{seen=27 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
20:07:47.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.592191 11334 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a: Bootstrap complete.
20:07:47.592 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
20:07:47.593 [DEBUG - Time-limited test] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:47.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.593341 11334 ts_tablet_manager.cc:1403] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.004s
20:07:47.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.594635 11334 raft_consensus.cc:359] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 9 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:47.595 [INFO - kudu-nio-7] (Connection.java:429) Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:47.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.594909 11334 raft_consensus.cc:740] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 9 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57af19dfdfce4211824a07be0443b43a, State: Initialized, Role: FOLLOWER
20:07:47.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.595057 11334 consensus_queue.cc:260] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 36, Last appended: 9.36, Last appended by leader: 36, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:47.595 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:47.596 [INFO - kudu-nio-7] (AsyncKuduClient.java:2355) Invalidating location 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:47.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.595351 11334 ts_tablet_manager.cc:1434] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.003s
20:07:47.597 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:47.595443 11442 heartbeater.cc:499] Master 127.8.230.188:39477 was elected leader, sending a full tablet report...
20:07:47.617 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:47.617 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)[L],57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:47.617 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=2, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=24), Traces: [0ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [2ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [3ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [21ms] refreshing cache from master, [21ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [23ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@1028955216(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@22938cbe -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6243bb02 -> wakeup thread Time-limited test)) after lookup
20:07:47.618 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:47.619 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:47.619 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:47.620 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:47.637 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:47.637 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)[L],57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:47.638 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=4, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=45), Traces: [0ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [2ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [3ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [21ms] refreshing cache from master, [21ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [23ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [25ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [26ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [27ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [44ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@1028955216(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@22938cbe -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1204981044 after retry RPC@943109479), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6243bb02 -> wakeup thread Time-limited test -> (continuation of Deferred@1204981044 after retry RPC@943109479))) after lookup
20:07:47.638 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:47.640 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:47.640 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:47.640 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:47.658 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:47.659 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)[L],57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:47.659 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=6, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=66), Traces: [0ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [2ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [3ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [21ms] refreshing cache from master, [21ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [23ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [25ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [26ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [27ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [44ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [45ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [47ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [47ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [62ms] refreshing cache from master, [62ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [65ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@1028955216(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@22938cbe -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1204981044 after retry RPC@943109479) -> (continuation of Deferred@254270202 after retry RPC@37215220), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6243bb02 -> wakeup thread Time-limited test -> (continuation of Deferred@1204981044 after retry RPC@943109479) -> (continuation of Deferred@254270202 after retry RPC@37215220))) after lookup
20:07:47.660 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:47.662 [INFO - kudu-nio-2] (Connection.java:429) Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:47.663 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:47.663 [INFO - kudu-nio-2] (AsyncKuduClient.java:2355) Invalidating location 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:47.758 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:47.759 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)[L],57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:47.759 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=8, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=166), Traces: [0ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [2ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [3ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [21ms] refreshing cache from master, [21ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [23ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [25ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [26ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [27ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [44ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [45ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [47ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [47ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [62ms] refreshing cache from master, [62ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [65ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [67ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [70ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [70ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [161ms] refreshing cache from master, [162ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [164ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@1028955216(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@22938cbe -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1204981044 after retry RPC@943109479) -> (continuation of Deferred@254270202 after retry RPC@37215220) -> (continuation of Deferred@713787927 after retry RPC@949862862), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6243bb02 -> wakeup thread Time-limited test -> (continuation of Deferred@1204981044 after retry RPC@943109479) -> (continuation of Deferred@254270202 after retry RPC@37215220) -> (continuation of Deferred@713787927 after retry RPC@949862862))) after lookup
20:07:47.761 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:47.763 [INFO - kudu-nio-3] (Connection.java:429) Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:47.763 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:47.763 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:47.918 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:47.918 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)[L],57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:47.918 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=10, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=325), Traces: [0ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [2ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [3ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [21ms] refreshing cache from master, [21ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [23ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [25ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [26ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [27ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [44ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [45ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [47ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [47ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [62ms] refreshing cache from master, [62ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [65ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [67ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [70ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [70ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [161ms] refreshing cache from master, [162ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [164ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [168ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [170ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [170ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [322ms] refreshing cache from master, [322ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [324ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@1028955216(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@22938cbe -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1204981044 after retry RPC@943109479) -> (continuation of Deferred@254270202 after retry RPC@37215220) -> (continuation of Deferred@713787927 after retry RPC@949862862) -> (continuation of Deferred@1456586163 after retry RPC@9737494), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6243bb02 -> wakeup thread Time-limited test -> (continuation of Deferred@1204981044 after retry RPC@943109479) -> (continuation of Deferred@254270202 after retry RPC@37215220) -> (continuation of Deferred@713787927 after retry RPC@949862862) -> (continuation of Deferred@1456586163 after retry RPC@9737494))) after lookup
20:07:47.919 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:47.920 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:47.921 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:47.921 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:48.098 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:48.098 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)[L],57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)]}]
20:07:48.099 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=12, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=506), Traces: [0ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [2ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [3ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [21ms] refreshing cache from master, [21ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [23ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [25ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [26ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [27ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [44ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [45ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [47ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [47ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [62ms] refreshing cache from master, [62ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [65ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [67ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [70ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [70ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [161ms] refreshing cache from master, [162ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [164ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [168ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [170ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [170ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [322ms] refreshing cache from master, [322ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [324ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [326ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [328ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [328ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [502ms] refreshing cache from master, [502ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [504ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@1028955216(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@22938cbe -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1204981044 after retry RPC@943109479) -> (continuation of Deferred@254270202 after retry RPC@37215220) -> (continuation of Deferred@713787927 after retry RPC@949862862) -> (continuation of Deferred@1456586163 after retry RPC@9737494) -> (continuation of Deferred@348646365 after retry RPC@819982454), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6243bb02 -> wakeup thread Time-limited test -> (continuation of Deferred@1204981044 after retry RPC@943109479) -> (continuation of Deferred@254270202 after retry RPC@37215220) -> (continuation of Deferred@713787927 after retry RPC@949862862) -> (continuation of Deferred@1456586163 after retry RPC@9737494) -> (continuation of Deferred@348646365 after retry RPC@819982454))) after lookup
20:07:48.100 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] connecting to peer
20:07:48.101 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:48.101 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:48.101 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709) for tablet a4035636acf142eeb16e984e2f510768: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709
20:07:48.948 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:48.948129 11455 raft_consensus.cc:493] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3ab092e11f74445fbbe7a4c29511b76b)
20:07:48.948 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:48.948287 11455 raft_consensus.cc:515] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:48.949 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:48.948844 11455 leader_election.cc:290] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709), 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897)
20:07:48.949 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:48.949404 11147 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.230.129:42709: connect: Connection refused (error 111)
20:07:48.950 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:48.950183 11147 leader_election.cc:336] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709): Network error: Client connection negotiation failed: client connection to 127.8.230.129:42709: connect: Connection refused (error 111)
20:07:48.953 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:48.952988 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "95c1f61bee2d40849c44869e11134c4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 40 } } ignore_live_leader: false dest_uuid: "57af19dfdfce4211824a07be0443b43a" is_pre_election: true
20:07:48.953 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:48.953169 11396 raft_consensus.cc:2468] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95c1f61bee2d40849c44869e11134c4a in term 9.
20:07:48.954 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:48.953567 11148 leader_election.cc:304] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57af19dfdfce4211824a07be0443b43a, 95c1f61bee2d40849c44869e11134c4a; no voters: 3ab092e11f74445fbbe7a4c29511b76b
20:07:48.954 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:48.953852 11455 raft_consensus.cc:2804] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 10 FOLLOWER]: Leader pre-election won for term 11
20:07:48.954 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:48.954171 11455 raft_consensus.cc:493] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3ab092e11f74445fbbe7a4c29511b76b)
20:07:48.954 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:48.954248 11455 raft_consensus.cc:3060] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 10 FOLLOWER]: Advancing to term 11
20:07:48.955 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:48.955291 11455 raft_consensus.cc:515] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:48.955 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:48.955595 11455 leader_election.cc:290] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [CANDIDATE]: Term 11 election: Requested vote from peers 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709), 57af19dfdfce4211824a07be0443b43a (127.8.230.130:37897)
20:07:48.956 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:48.955947 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4035636acf142eeb16e984e2f510768" candidate_uuid: "95c1f61bee2d40849c44869e11134c4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 40 } } ignore_live_leader: false dest_uuid: "57af19dfdfce4211824a07be0443b43a"
20:07:48.956 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:48.956036 11396 raft_consensus.cc:3060] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 9 FOLLOWER]: Advancing to term 11
20:07:48.956 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:48.956658 11147 leader_election.cc:336] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709): Network error: Client connection negotiation failed: client connection to 127.8.230.129:42709: connect: Connection refused (error 111)
20:07:48.957 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:48.957293 11396 raft_consensus.cc:2468] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95c1f61bee2d40849c44869e11134c4a in term 11.
20:07:48.957 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:48.957543 11148 leader_election.cc:304] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57af19dfdfce4211824a07be0443b43a, 95c1f61bee2d40849c44869e11134c4a; no voters: 3ab092e11f74445fbbe7a4c29511b76b
20:07:48.957 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:48.957773 11455 raft_consensus.cc:2804] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 11 FOLLOWER]: Leader election won for term 11
20:07:48.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:48.958031 11455 raft_consensus.cc:697] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [term 11 LEADER]: Becoming Leader. State: Replica: 95c1f61bee2d40849c44869e11134c4a, State: Running, Role: LEADER
20:07:48.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:48.958122 11455 consensus_queue.cc:237] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 10.40, Last appended by leader: 40, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:48.959 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:48.959317  9301 catalog_manager.cc:5696] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a reported cstate change: term changed from 10 to 11, leader changed from 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129) to 95c1f61bee2d40849c44869e11134c4a (127.8.230.131). New cstate: current_term: 11 leader_uuid: "95c1f61bee2d40849c44869e11134c4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } health_report { overall_health: HEALTHY } } }
20:07:49.460 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:49.460671 11147 consensus_peers.cc:597] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a -> Peer 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709): Couldn't send request to peer 3ab092e11f74445fbbe7a4c29511b76b. Status: Network error: Client connection negotiation failed: client connection to 127.8.230.129:42709: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
20:07:49.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:49.467259 11396 raft_consensus.cc:1275] T a4035636acf142eeb16e984e2f510768 P 57af19dfdfce4211824a07be0443b43a [term 11 FOLLOWER]: Refusing update from remote peer 95c1f61bee2d40849c44869e11134c4a: Log matching property violated. Preceding OpId in replica: term: 9 index: 36. Preceding OpId from leader: term: 11 index: 41. (index mismatch)
20:07:49.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:49.467736 11472 consensus_queue.cc:1048] T a4035636acf142eeb16e984e2f510768 P 95c1f61bee2d40849c44869e11134c4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 41, Last known committed idx: 36, Time since last communication: 0.000s
20:07:49.937 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:2538) Learned about tablet a4035636acf142eeb16e984e2f510768 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181' with partition [<start>, <end>)
20:07:49.937 [DEBUG - kudu-nio-2] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=a4035636acf142eeb16e984e2f510768@[3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709),57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897),95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)[L]]}]
20:07:49.937 [DEBUG - kudu-nio-2] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=a4035636acf142eeb16e984e2f510768, attempt=14, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=2344), Traces: [0ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [2ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [3ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [21ms] refreshing cache from master, [21ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [23ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [25ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [26ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [27ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [44ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [45ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [47ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [47ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [62ms] refreshing cache from master, [62ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [65ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [67ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [70ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [70ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [161ms] refreshing cache from master, [162ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [164ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [168ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [170ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [170ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [322ms] refreshing cache from master, [322ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [324ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [326ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [328ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [328ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [502ms] refreshing cache from master, [502ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [504ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, [507ms] sending RPC to server 3ab092e11f74445fbbe7a4c29511b76b, [508ms] received response from server 3ab092e11f74445fbbe7a4c29511b76b: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [508ms] delaying RPC due to: Network error: Failed to connect to peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709): Connection refused: /127.8.230.129:42709, [2342ms] refreshing cache from master, [2342ms] Sub RPC GetTableLocations: sending RPC to server master-127.8.230.188:39477, [2344ms] Sub RPC GetTableLocations: received response from server master-127.8.230.188:39477: OK, deferred=Deferred@1028955216(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$243/0x00007f1e703371b0@22938cbe -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1204981044 after retry RPC@943109479) -> (continuation of Deferred@254270202 after retry RPC@37215220) -> (continuation of Deferred@713787927 after retry RPC@949862862) -> (continuation of Deferred@1456586163 after retry RPC@9737494) -> (continuation of Deferred@348646365 after retry RPC@819982454) -> (continuation of Deferred@172700534 after retry RPC@1171795465), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6243bb02 -> wakeup thread Time-limited test -> (continuation of Deferred@1204981044 after retry RPC@943109479) -> (continuation of Deferred@254270202 after retry RPC@37215220) -> (continuation of Deferred@713787927 after retry RPC@949862862) -> (continuation of Deferred@1456586163 after retry RPC@9737494) -> (continuation of Deferred@348646365 after retry RPC@819982454) -> (continuation of Deferred@172700534 after retry RPC@1171795465))) after lookup
20:07:49.938 [DEBUG - kudu-nio-2] (Connection.java:748) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] connecting to peer
20:07:49.939 [DEBUG - kudu-nio-6] (Connection.java:757) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] Successfully connected to peer
20:07:49.953 [DEBUG - kudu-nio-6] (Negotiator.java:919) Authenticated connection [id: 0x0889ae5f, L:/127.0.0.1:40402 - R:/127.8.230.131:37545] using TOKEN/null
20:07:49.965 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.8.230.129:42709
20:07:49.965 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

20:07:49.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:49.966347  9114 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:49.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskx5o66C/build/release/bin/kudu
20:07:49.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal
20:07:49.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data
20:07:49.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
20:07:49.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
20:07:49.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
20:07:49.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:49.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:49.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:49.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/logs
20:07:49.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data/info.pb
20:07:49.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:49.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
20:07:49.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
20:07:49.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
20:07:49.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
20:07:49.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:49.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:49.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:49.967 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
20:07:49.967 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
20:07:49.967 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.129:42709
20:07:49.967 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.8.230.129
20:07:49.967 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:49.967 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=44243
20:07:49.967 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.129
20:07:49.967 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:49.967 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:49.967 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:49.967 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
20:07:50.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:50.056190 11486 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
20:07:50.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:50.056416 11486 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
20:07:50.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:50.056439 11486 flags.cc:432] Enabled unsafe flag: --never_fsync=true
20:07:50.058 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:50.058068 11486 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
20:07:50.058 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:50.058139 11486 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.230.129
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.059995 11486 tablet_server_runner.cc:78] Tablet server non-default flags:
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.230.148:37709
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.230.129:42709
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data/info.pb
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.230.129
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=44243
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.8.230.190:34269,127.8.230.189:44351,127.8.230.188:39477
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.11486
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.8.230.129
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/logs
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 66827b15b158c37363c37baa160f29a0ca1f1e63
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type RELEASE
20:07:50.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 18 May 2026 19:43:16 UTC on e7f111948823
20:07:50.061 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 11961
20:07:50.061 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.060276 11486 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
20:07:50.061 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.060480 11486 file_cache.cc:492] Constructed file cache file cache with capacity 419430
20:07:50.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:50.063050 11494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:50.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:50.063154 11491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:50.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260518 20:07:50.063194 11492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
20:07:50.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.063552 11486 server_base.cc:1061] running on GCE node
20:07:50.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.063736 11486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
20:07:50.064 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.063932 11486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
20:07:50.067 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.067209 11486 hybrid_clock.cc:648] HybridClock initialized: now 1779134870066820 us; error 424 us; skew 500 ppm
20:07:50.068 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.068558 11486 webserver.cc:533] Webserver started at http://127.8.230.129:44243/ using document root <none> and password file <none>
20:07:50.068 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.068833 11486 fs_manager.cc:362] Metadata directory not provided
20:07:50.069 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.068881 11486 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
20:07:50.070 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.070202 11486 fs_manager.cc:714] Time spent opening directory manager: real 0.001s	user 0.001s	sys 0.000s
20:07:50.071 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.070981 11500 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
20:07:50.071 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.071292 11486 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
20:07:50.071 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.071409 11486 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data,/tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal
20:07:50.071 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "3ab092e11f74445fbbe7a4c29511b76b"
20:07:50.071 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-05-18 20:07:21 on dist-test-slave-mzrs"
20:07:50.072 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.071821 11486 fs_report.cc:389] FS layout report
20:07:50.072 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
20:07:50.072 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal
20:07:50.072 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/wal
20:07:50.072 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data/data
20:07:50.072 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
20:07:50.072 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
20:07:50.072 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
20:07:50.072 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
20:07:50.072 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
20:07:50.072 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
20:07:50.072 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
20:07:50.072 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
20:07:50.072 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
20:07:50.072 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
20:07:50.072 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
20:07:50.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.100832 11486 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
20:07:50.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.101253 11486 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
20:07:50.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.101402 11486 kserver.cc:163] Server-wide thread pool size limit: 3276
20:07:50.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.101716 11486 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
20:07:50.102 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.102393 11509 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
20:07:50.103 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.103603 11486 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
20:07:50.104 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.103726 11486 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.002s	user 0.000s	sys 0.000s
20:07:50.104 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.103895 11486 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
20:07:50.105 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.105070 11486 ts_tablet_manager.cc:616] Registered 1 tablets
20:07:50.105 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.105156 11486 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
20:07:50.105 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.105248 11509 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Bootstrap starting.
20:07:50.114 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.113930 11486 rpc_server.cc:307] RPC server started. Bound to: 127.8.230.129:42709
20:07:50.114 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.114045 11616 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.230.129:42709 every 8 connection(s)
20:07:50.114 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.114456 11486 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskx5o66C/test-tmp/mini-kudu-cluster18058925015283208717/ts-0/data/info.pb
20:07:50.118 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.117934 11509 log.cc:826] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Log is configured to *not* fsync() on all Append() calls
20:07:50.121 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.121167 11617 heartbeater.cc:344] Connected to a master server at 127.8.230.188:39477
20:07:50.121 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.121168 11619 heartbeater.cc:344] Connected to a master server at 127.8.230.189:44351
20:07:50.121 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.121311 11617 heartbeater.cc:461] Registering TS with master...
20:07:50.121 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.121311 11619 heartbeater.cc:461] Registering TS with master...
20:07:50.121 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.121554 11619 heartbeater.cc:507] Master 127.8.230.189:44351 requested a full tablet report, sending...
20:07:50.122 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.121555 11617 heartbeater.cc:507] Master 127.8.230.188:39477 requested a full tablet report, sending...
20:07:50.122 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.122370  9228 ts_manager.cc:194] Re-registered known tserver with Master: 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709)
20:07:50.122 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.122496  9301 ts_manager.cc:194] Re-registered known tserver with Master: 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709)
20:07:50.122 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.122665  9114 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskx5o66C/build/release/bin/kudu as pid 11486
20:07:50.123 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
20:07:50.123 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.122997  9301 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.8.230.129:52957
20:07:50.123 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.123179 11618 heartbeater.cc:344] Connected to a master server at 127.8.230.190:34269
20:07:50.123 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.123248 11618 heartbeater.cc:461] Registering TS with master...
20:07:50.123 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.123354 11618 heartbeater.cc:507] Master 127.8.230.190:34269 requested a full tablet report, sending...
20:07:50.124 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.123878  9154 ts_manager.cc:194] Re-registered known tserver with Master: 3ab092e11f74445fbbe7a4c29511b76b (127.8.230.129:42709)
20:07:50.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.125573 11509 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Bootstrap replayed 1/1 log segments. Stats: ops{read=40 overwritten=0 applied=40 ignored=0} inserts{seen=30 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
20:07:50.126 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.125924 11509 tablet_bootstrap.cc:492] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Bootstrap complete.
20:07:50.127 [DEBUG - kudu-nio-6] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 33 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181, tablet=a4035636acf142eeb16e984e2f510768, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
20:07:50.127 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.127040 11509 ts_tablet_manager.cc:1403] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Time spent bootstrapping tablet: real 0.022s	user 0.004s	sys 0.015s
20:07:50.128 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.128274 11509 raft_consensus.cc:359] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 10 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:50.128 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.128567 11509 raft_consensus.cc:740] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [term 10 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ab092e11f74445fbbe7a4c29511b76b, State: Initialized, Role: FOLLOWER
20:07:50.128 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.128680 11509 consensus_queue.cc:260] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 40, Last appended: 10.40, Last appended by leader: 40, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab092e11f74445fbbe7a4c29511b76b" member_type: VOTER last_known_addr { host: "127.8.230.129" port: 42709 } } peers { permanent_uuid: "57af19dfdfce4211824a07be0443b43a" member_type: VOTER last_known_addr { host: "127.8.230.130" port: 37897 } } peers { permanent_uuid: "95c1f61bee2d40849c44869e11134c4a" member_type: VOTER last_known_addr { host: "127.8.230.131" port: 37545 } }
20:07:50.129 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.128973 11509 ts_tablet_manager.cc:1434] T a4035636acf142eeb16e984e2f510768 P 3ab092e11f74445fbbe7a4c29511b76b: Time spent starting tablet: real 0.002s	user 0.000s	sys 0.003s
20:07:50.129 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.129034 11617 heartbeater.cc:499] Master 127.8.230.188:39477 was elected leader, sending a full tablet report...
20:07:50.133 [DEBUG - kudu-nio-6] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 33 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1779134842181, tablet=a4035636acf142eeb16e984e2f510768, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
20:07:50.138 [DEBUG - main] (Connection.java:575) [peer master-127.8.230.190:34269(127.8.230.190:34269)] disconnecting while in state READY
20:07:50.139 [DEBUG - main] (Connection.java:575) [peer 57af19dfdfce4211824a07be0443b43a(127.8.230.130:37897)] disconnecting while in state TERMINATED
20:07:50.140 [DEBUG - main] (Connection.java:575) [peer master-127.8.230.189:44351(127.8.230.189:44351)] disconnecting while in state READY
20:07:50.140 [DEBUG - main] (Connection.java:575) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] disconnecting while in state READY
20:07:50.140 [DEBUG - main] (Connection.java:575) [peer master-127.8.230.188:39477(127.8.230.188:39477)] disconnecting while in state READY
20:07:50.140 [DEBUG - main] (Connection.java:575) [peer 3ab092e11f74445fbbe7a4c29511b76b(127.8.230.129:42709)] disconnecting while in state TERMINATED
20:07:50.143 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.230.189:44351(127.8.230.189:44351)] handling channelInactive
20:07:50.143 [DEBUG - kudu-nio-6] (Connection.java:239) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] handling channelInactive
20:07:50.143 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.230.189:44351(127.8.230.189:44351)] cleaning up while in state READY due to: connection closed
20:07:50.143 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 95c1f61bee2d40849c44869e11134c4a(127.8.230.131:37545)] cleaning up while in state READY due to: connection closed
20:07:50.143 [DEBUG - kudu-nio-2] (Connection.java:239) [peer master-127.8.230.188:39477(127.8.230.188:39477)] handling channelInactive
20:07:50.144 [DEBUG - kudu-nio-2] (Connection.java:709) [peer master-127.8.230.188:39477(127.8.230.188:39477)] cleaning up while in state READY due to: connection closed
20:07:50.144 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2704) Releasing all remaining resources
20:07:50.149 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.8.230.190:34269(127.8.230.190:34269)] handling channelInactive
20:07:50.150 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.8.230.190:34269(127.8.230.190:34269)] cleaning up while in state READY due to: connection closed
20:07:50.152 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.148967  9114 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskx5o66C/build/release/bin/kudu with pid 11486
20:07:50.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.156841  9114 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskx5o66C/build/release/bin/kudu with pid 11313
20:07:50.163 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.162582  9114 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskx5o66C/build/release/bin/kudu with pid 11132
20:07:50.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.169237  9114 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskx5o66C/build/release/bin/kudu with pid 9125
20:07:50.175 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.175603  9114 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskx5o66C/build/release/bin/kudu with pid 9194
20:07:50.181 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260518 20:07:50.181107  9114 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskx5o66C/build/release/bin/kudu with pid 9267
20:07:50.213 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-05-18T20:07:50Z chronyd exiting

Time: 30.808

OK (1 test)

