JUnit version 4.13.2
.02:03:49.638 [DEBUG - main] (RandomUtils.java:49) Using random seed: 1786500229629
02:03:49.696 [INFO - main] (RetryRule.java:138) Creating RetryStatement with result reporter and retry count of 0 (no retries)
02:03:49.762 [INFO - main] (KuduTestHarness.java:155) Creating a new MiniKuduCluster...
02:03:49.774 [INFO - main] (TempDirUtils.java:71) Using the temp directory defined by TEST_TMPDIR: /tmp/dist-test-taskzRoOVp/test-tmp
02:03:49.779 [INFO - main] (KuduBinaryLocator.java:83) Using Kudu binary directory specified by system property 'kuduBinDir': ../../../build/tsan/bin
02:03:49.782 [INFO - main] (MiniKuduCluster.java:217) Starting process: [/tmp/dist-test-taskzRoOVp/java/build/dist-test/../../../build/tsan/bin/kudu, test, mini_cluster, --serialization=pb]
02:03:49.971 [DEBUG - main] (MiniKuduCluster.java:185) Request: create_cluster {
  num_masters: 3
  num_tservers: 3
  enable_kerberos: false
  cluster_root: "/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585"
  hms_mode: NONE
  mini_kdc_options {
  }
  principal: "kudu"
  mini_oidc_options {
  }
}

02:03:50.182 [DEBUG - main] (MiniKuduCluster.java:195) Response: 
02:03:50.183 [DEBUG - main] (MiniKuduCluster.java:185) Request: start_cluster {
}

02:03:50.190 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:50.189640 16653 mini_oidc.cc:150] Starting JWKS server
02:03:50.194 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:50.193828 16653 webserver.cc:533] Webserver started at http://127.0.0.1:45909/ using document root <none> and password file <none>
02:03:50.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:50.194506 16653 mini_oidc.cc:186] Starting OIDC Discovery server
02:03:50.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:50.196853 16653 webserver.cc:533] Webserver started at http://127.0.0.1:46533/ using document root <none> and password file <none>
02:03:50.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-08-12T02:03:50Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
02:03:50.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-08-12T02:03:50Z Disabled control of system clock
02:03:50.271 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:50.270329 16653 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskzRoOVp/build/tsan/bin/kudu
02:03:50.271 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskzRoOVp/build/tsan/bin/kudu
02:03:50.272 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-0/wal
02:03:50.272 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-0/data
02:03:50.272 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
02:03:50.273 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
02:03:50.273 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
02:03:50.273 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:03:50.274 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:03:50.274 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:03:50.274 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-0/logs
02:03:50.275 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-0/data/info.pb
02:03:50.275 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:03:50.276 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
02:03:50.277 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
02:03:50.277 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
02:03:50.277 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
02:03:50.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:03:50.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:03:50.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:03:50.279 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
02:03:50.279 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
02:03:50.279 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
02:03:50.279 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
02:03:50.280 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.16.67.126:34295
02:03:50.280 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.16.67.126
02:03:50.280 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
02:03:50.280 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.16.67.84:39899
02:03:50.281 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:03:50.281 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:03:50.282 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:03:50.282 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
02:03:50.283 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.16.67.126:34295,127.16.67.125:32941,127.16.67.124:33409
02:03:50.283 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
02:03:50.283 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:45909/jwks.json with env {}
02:03:50.607 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:50.606523 16667 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
02:03:50.607 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:50.607146 16667 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
02:03:50.608 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:50.607592 16667 flags.cc:432] Enabled unsafe flag: --never_fsync=true
02:03:50.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:50.643226 16667 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
02:03:50.644 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:50.643539 16667 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
02:03:50.644 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:50.643770 16667 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
02:03:50.644 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:50.644071 16667 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
02:03:50.644 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:50.644343 16667 flags.cc:432] Enabled experimental flag: --enable_jwt_token_auth=true
02:03:50.645 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:50.644562 16667 flags.cc:432] Enabled experimental flag: --jwks_url=http://localhost:45909/jwks.json
02:03:50.684 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:50.683629 16667 master_runner.cc:387] Master server non-default flags:
02:03:50.684 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-0/logs
02:03:50.684 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:03:50.685 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
02:03:50.685 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:03:50.685 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.16.67.84:39899
02:03:50.686 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:03:50.686 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:03:50.686 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-0/data
02:03:50.686 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-0/wal
02:03:50.686 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
02:03:50.687 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.16.67.126:34295,127.16.67.125:32941,127.16.67.124:33409
02:03:50.687 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:03:50.687 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:03:50.687 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
02:03:50.688 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.16.67.126:34295
02:03:50.688 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
02:03:50.688 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
02:03:50.688 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
02:03:50.688 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:45909/jwks.json
02:03:50.689 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:03:50.689 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:03:50.689 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-0/data/info.pb
02:03:50.689 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.16.67.126
02:03:50.689 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
02:03:50.689 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
02:03:50.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:03:50.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
02:03:50.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
02:03:50.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:03:50.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
02:03:50.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
02:03:50.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
02:03:50.692 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 1d9153c248733781bf69132f2d094d9d3c92b2bc
02:03:50.692 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
02:03:50.692 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 12 Aug 2026 01:45:55 UTC on e7f111948823
02:03:50.692 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 13299
02:03:50.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
02:03:50.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:50.685319 16667 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
02:03:50.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:50.687963 16667 file_cache.cc:504] Constructed file cache file cache with capacity 419430
02:03:50.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:50.706270 16674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:03:52.031 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:50.708114 16678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:03:52.033 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:52.032439 16680 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1323 milliseconds
02:03:52.033 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:52.033010 16682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:03:52.034 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.033989 16667 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
02:03:52.037 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.037040 16667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
02:03:52.041 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.040385 16667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
02:03:52.042 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.041960 16667 hybrid_clock.cc:648] HybridClock initialized: now 1786500232041848 us; error 162 us; skew 500 ppm
02:03:52.043 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.043151 16667 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
02:03:52.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.050238 16667 webserver.cc:533] Webserver started at http://127.16.67.126:41163/ using document root <none> and password file <none>
02:03:52.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.051352 16667 fs_manager.cc:362] Metadata directory not provided
02:03:52.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.051553 16667 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
02:03:52.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.051951 16667 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
02:03:52.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.056457 16667 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-0/data/instance:
02:03:52.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "657ce041f6d24ed9a826f7aab333f9f5"
02:03:52.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-12 02:03:52 on dist-test-slave-h6n0"
02:03:52.058 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.057837 16667 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-0/wal/instance:
02:03:52.058 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "657ce041f6d24ed9a826f7aab333f9f5"
02:03:52.058 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-12 02:03:52 on dist-test-slave-h6n0"
02:03:52.067 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.066149 16667 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.001s
02:03:52.072 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.072448 16693 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
02:03:52.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.074469 16667 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.002s	sys 0.004s
02:03:52.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.075223 16667 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-0/data,/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-0/wal
02:03:52.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "657ce041f6d24ed9a826f7aab333f9f5"
02:03:52.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-12 02:03:52 on dist-test-slave-h6n0"
02:03:52.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.075664 16667 fs_report.cc:389] FS layout report
02:03:52.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
02:03:52.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-0/wal
02:03:52.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-0/wal
02:03:52.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-0/data/data
02:03:52.078 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
02:03:52.078 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
02:03:52.078 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
02:03:52.079 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
02:03:52.079 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
02:03:52.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
02:03:52.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
02:03:52.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
02:03:52.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
02:03:52.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
02:03:52.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
02:03:52.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.148655 16667 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
02:03:52.150 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.150498 16667 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
02:03:52.151 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.150957 16667 kserver.cc:163] Server-wide thread pool size limit: 3276
02:03:52.235 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.235214 16667 rpc_server.cc:307] RPC server started. Bound to: 127.16.67.126:34295
02:03:52.236 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.235288 16770 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.67.126:34295 every 8 connection(s)
02:03:52.238 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.238197 16667 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-0/data/info.pb
02:03:52.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.242268 16653 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskzRoOVp/build/tsan/bin/kudu as pid 16667
02:03:52.243 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.242964 16653 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-0/wal/instance
02:03:52.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.249182 16771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
02:03:52.250 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.249511 16653 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskzRoOVp/build/tsan/bin/kudu
02:03:52.250 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskzRoOVp/build/tsan/bin/kudu
02:03:52.250 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-1/wal
02:03:52.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-1/data
02:03:52.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
02:03:52.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
02:03:52.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
02:03:52.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:03:52.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:03:52.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:03:52.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-1/logs
02:03:52.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-1/data/info.pb
02:03:52.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:03:52.253 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
02:03:52.253 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
02:03:52.253 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
02:03:52.253 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
02:03:52.253 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:03:52.253 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:03:52.253 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:03:52.254 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
02:03:52.254 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
02:03:52.254 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
02:03:52.254 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
02:03:52.254 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.16.67.125:32941
02:03:52.254 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.16.67.125
02:03:52.254 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
02:03:52.254 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.16.67.84:39899
02:03:52.254 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:03:52.255 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:03:52.255 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:03:52.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
02:03:52.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.16.67.126:34295,127.16.67.125:32941,127.16.67.124:33409
02:03:52.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
02:03:52.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:45909/jwks.json with env {}
02:03:52.266 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.264081 16771 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.67.126" port: 34295 } has no permanent_uuid. Determining permanent_uuid...
02:03:52.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.295285 16771 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } has no permanent_uuid. Determining permanent_uuid...
02:03:52.298 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:52.298266 16694 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.67.125:32941: connect: Connection refused (error 111)
02:03:52.304 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:52.303639 16771 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.16.67.125:32941: Network error: Client connection negotiation failed: client connection to 127.16.67.125:32941: connect: Connection refused (error 111)
02:03:52.354 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.353235 16771 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } attempt: 1
02:03:52.358 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:52.357932 16771 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.16.67.125:32941: Network error: Client connection negotiation failed: client connection to 127.16.67.125:32941: connect: Connection refused (error 111)
02:03:52.427 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.426411 16771 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } attempt: 2
02:03:52.433 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:52.432523 16771 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.16.67.125:32941: Network error: Client connection negotiation failed: client connection to 127.16.67.125:32941: connect: Connection refused (error 111)
02:03:52.524 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.524102 16771 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } attempt: 3
02:03:52.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:52.528584 16771 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.16.67.125:32941: Network error: Client connection negotiation failed: client connection to 127.16.67.125:32941: connect: Connection refused (error 111)
02:03:52.616 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:52.615854 16776 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
02:03:52.617 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:52.617091 16776 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
02:03:52.618 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:52.618687 16776 flags.cc:432] Enabled unsafe flag: --never_fsync=true
02:03:52.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:52.653611 16776 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
02:03:52.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:52.653923 16776 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
02:03:52.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:52.654129 16776 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
02:03:52.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:52.654338 16776 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
02:03:52.655 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:52.654578 16776 flags.cc:432] Enabled experimental flag: --enable_jwt_token_auth=true
02:03:52.655 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:52.654815 16776 flags.cc:432] Enabled experimental flag: --jwks_url=http://localhost:45909/jwks.json
02:03:52.672 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.672071 16771 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } attempt: 4
02:03:52.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:52.677956 16771 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.16.67.125:32941: Network error: Client connection negotiation failed: client connection to 127.16.67.125:32941: connect: Connection refused (error 111)
02:03:52.705 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.705013 16776 master_runner.cc:387] Master server non-default flags:
02:03:52.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-1/logs
02:03:52.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:03:52.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
02:03:52.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:03:52.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.16.67.84:39899
02:03:52.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:03:52.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:03:52.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-1/data
02:03:52.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-1/wal
02:03:52.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
02:03:52.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.16.67.126:34295,127.16.67.125:32941,127.16.67.124:33409
02:03:52.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:03:52.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:03:52.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
02:03:52.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.16.67.125:32941
02:03:52.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
02:03:52.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
02:03:52.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
02:03:52.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:45909/jwks.json
02:03:52.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:03:52.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:03:52.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-1/data/info.pb
02:03:52.710 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.16.67.125
02:03:52.710 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
02:03:52.710 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
02:03:52.710 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:03:52.710 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
02:03:52.711 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
02:03:52.711 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:03:52.711 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
02:03:52.711 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
02:03:52.711 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
02:03:52.711 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 1d9153c248733781bf69132f2d094d9d3c92b2bc
02:03:52.712 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
02:03:52.712 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 12 Aug 2026 01:45:55 UTC on e7f111948823
02:03:52.712 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 13299
02:03:52.712 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
02:03:52.712 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.706856 16776 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
02:03:52.712 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.709105 16776 file_cache.cc:504] Constructed file cache file cache with capacity 419430
02:03:52.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:52.724817 16788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:03:52.979 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:52.977452 16771 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } attempt: 5
02:03:52.985 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:52.984895 16771 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.16.67.125:32941: Network error: Client connection negotiation failed: client connection to 127.16.67.125:32941: connect: Connection refused (error 111)
02:03:53.533 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:53.532483 16771 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } attempt: 6
02:03:53.545 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:53.544392 16771 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.16.67.125:32941: Network error: Client connection negotiation failed: client connection to 127.16.67.125:32941: connect: Connection refused (error 111)
02:03:53.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:52.728165 16789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:03:53.983 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:53.982506 16790 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
02:03:53.985 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:53.984534 16791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:03:53.988 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:53.987897 16776 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.262s	user 0.392s	sys 0.869s
02:03:53.988 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:53.988173 16776 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.263s	user 0.392s	sys 0.869s
02:03:53.988 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:53.988435 16776 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
02:03:53.989 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:53.989599 16776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
02:03:53.999 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:53.999147 16776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
02:03:54.002 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.001832 16776 hybrid_clock.cc:648] HybridClock initialized: now 1786500234001748 us; error 88 us; skew 500 ppm
02:03:54.003 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.002987 16776 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
02:03:54.012 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.012213 16776 webserver.cc:533] Webserver started at http://127.16.67.125:40205/ using document root <none> and password file <none>
02:03:54.013 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.013402 16776 fs_manager.cc:362] Metadata directory not provided
02:03:54.014 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.013674 16776 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
02:03:54.014 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.014313 16776 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
02:03:54.020 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.019342 16776 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-1/data/instance:
02:03:54.020 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "fd29b0c9750c4be7acec1a78c70549a6"
02:03:54.020 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-12 02:03:54 on dist-test-slave-h6n0"
02:03:54.021 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.021118 16776 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-1/wal/instance:
02:03:54.021 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "fd29b0c9750c4be7acec1a78c70549a6"
02:03:54.021 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-12 02:03:54 on dist-test-slave-h6n0"
02:03:54.032 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.031747 16776 fs_manager.cc:696] Time spent creating directory manager: real 0.010s	user 0.007s	sys 0.004s
02:03:54.041 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.040464 16803 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
02:03:54.043 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.043211 16776 fs_manager.cc:730] Time spent opening block manager: real 0.007s	user 0.002s	sys 0.003s
02:03:54.044 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.043710 16776 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-1/data,/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-1/wal
02:03:54.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "fd29b0c9750c4be7acec1a78c70549a6"
02:03:54.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-12 02:03:54 on dist-test-slave-h6n0"
02:03:54.046 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.044449 16776 fs_report.cc:389] FS layout report
02:03:54.046 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
02:03:54.046 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-1/wal
02:03:54.046 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-1/wal
02:03:54.047 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-1/data/data
02:03:54.047 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
02:03:54.047 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
02:03:54.047 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
02:03:54.047 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
02:03:54.048 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
02:03:54.048 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
02:03:54.048 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
02:03:54.048 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
02:03:54.048 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
02:03:54.048 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
02:03:54.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
02:03:54.128 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.127826 16776 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
02:03:54.130 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.129644 16776 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
02:03:54.130 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.130365 16776 kserver.cc:163] Server-wide thread pool size limit: 3276
02:03:54.209 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.209380 16776 rpc_server.cc:307] RPC server started. Bound to: 127.16.67.125:32941
02:03:54.210 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.209484 16886 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.67.125:32941 every 8 connection(s)
02:03:54.212 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.212538 16776 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-1/data/info.pb
02:03:54.218 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.214707 16653 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskzRoOVp/build/tsan/bin/kudu as pid 16776
02:03:54.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.215246 16653 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-1/wal/instance
02:03:54.220 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.219874 16653 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskzRoOVp/build/tsan/bin/kudu
02:03:54.220 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskzRoOVp/build/tsan/bin/kudu
02:03:54.220 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-2/wal
02:03:54.221 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-2/data
02:03:54.221 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
02:03:54.221 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
02:03:54.221 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
02:03:54.221 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:03:54.221 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:03:54.221 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:03:54.222 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-2/logs
02:03:54.222 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-2/data/info.pb
02:03:54.222 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:03:54.222 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
02:03:54.222 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
02:03:54.222 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
02:03:54.223 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
02:03:54.223 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:03:54.223 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:03:54.223 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:03:54.223 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
02:03:54.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
02:03:54.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
02:03:54.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
02:03:54.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.16.67.124:33409
02:03:54.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.16.67.124
02:03:54.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
02:03:54.225 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.16.67.84:39899
02:03:54.225 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:03:54.225 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:03:54.225 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:03:54.225 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
02:03:54.225 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.16.67.126:34295,127.16.67.125:32941,127.16.67.124:33409
02:03:54.225 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
02:03:54.226 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:45909/jwks.json with env {}
02:03:54.226 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.220538 16887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
02:03:54.238 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.236227 16887 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.67.126" port: 34295 } has no permanent_uuid. Determining permanent_uuid...
02:03:54.261 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.260887 16887 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } has no permanent_uuid. Determining permanent_uuid...
02:03:54.286 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.285679 16887 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } has no permanent_uuid. Determining permanent_uuid...
02:03:54.289 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:54.288565 16807 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.67.124:33409: connect: Connection refused (error 111)
02:03:54.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:54.293264 16887 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.16.67.124:33409: Network error: Client connection negotiation failed: client connection to 127.16.67.124:33409: connect: Connection refused (error 111)
02:03:54.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.342774 16887 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } attempt: 1
02:03:54.349 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:54.348812 16887 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.16.67.124:33409: Network error: Client connection negotiation failed: client connection to 127.16.67.124:33409: connect: Connection refused (error 111)
02:03:54.417 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.417241 16887 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } attempt: 2
02:03:54.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:54.422031 16887 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.16.67.124:33409: Network error: Client connection negotiation failed: client connection to 127.16.67.124:33409: connect: Connection refused (error 111)
02:03:54.514 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.513446 16887 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } attempt: 3
02:03:54.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:54.518268 16887 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.16.67.124:33409: Network error: Client connection negotiation failed: client connection to 127.16.67.124:33409: connect: Connection refused (error 111)
02:03:54.580 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:54.579808 16889 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
02:03:54.580 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:54.580530 16889 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
02:03:54.581 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:54.581213 16889 flags.cc:432] Enabled unsafe flag: --never_fsync=true
02:03:54.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.604996 16771 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } attempt: 7
02:03:54.617 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.616907 16771 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } has no permanent_uuid. Determining permanent_uuid...
02:03:54.619 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:54.618856 16889 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
02:03:54.619 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:54.619253 16889 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
02:03:54.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:54.619685 16889 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
02:03:54.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:54.620102 16889 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
02:03:54.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:54.620733 16889 flags.cc:432] Enabled experimental flag: --enable_jwt_token_auth=true
02:03:54.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:54.620998 16889 flags.cc:432] Enabled experimental flag: --jwks_url=http://localhost:45909/jwks.json
02:03:54.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:54.622920 16771 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.16.67.124:33409: Network error: Client connection negotiation failed: client connection to 127.16.67.124:33409: connect: Connection refused (error 111)
02:03:54.661 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.660528 16889 master_runner.cc:387] Master server non-default flags:
02:03:54.661 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-2/logs
02:03:54.661 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:03:54.661 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
02:03:54.661 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:03:54.662 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.16.67.84:39899
02:03:54.662 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:03:54.662 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:03:54.663 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-2/data
02:03:54.663 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-2/wal
02:03:54.663 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
02:03:54.663 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.16.67.126:34295,127.16.67.125:32941,127.16.67.124:33409
02:03:54.664 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:03:54.665 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:03:54.665 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
02:03:54.665 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.16.67.124:33409
02:03:54.665 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
02:03:54.665 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
02:03:54.665 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
02:03:54.665 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:45909/jwks.json
02:03:54.666 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:03:54.666 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:03:54.666 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-2/data/info.pb
02:03:54.666 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.16.67.124
02:03:54.666 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
02:03:54.667 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
02:03:54.667 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:03:54.667 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
02:03:54.667 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
02:03:54.667 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:03:54.667 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
02:03:54.667 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
02:03:54.667 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
02:03:54.667 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 1d9153c248733781bf69132f2d094d9d3c92b2bc
02:03:54.667 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
02:03:54.668 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 12 Aug 2026 01:45:55 UTC on e7f111948823
02:03:54.668 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 13299
02:03:54.668 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
02:03:54.668 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.661767 16887 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } attempt: 4
02:03:54.668 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.662376 16889 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
02:03:54.668 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.664695 16889 file_cache.cc:504] Constructed file cache file cache with capacity 419430
02:03:54.668 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:54.667236 16887 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.16.67.124:33409: Network error: Client connection negotiation failed: client connection to 127.16.67.124:33409: connect: Connection refused (error 111)
02:03:54.680 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:54.679095 16906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:03:54.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.681306 16771 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } attempt: 1
02:03:54.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:54.689334 16771 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.16.67.124:33409: Network error: Client connection negotiation failed: client connection to 127.16.67.124:33409: connect: Connection refused (error 111)
02:03:54.771 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.770879 16771 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } attempt: 2
02:03:54.779 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:54.778790 16771 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.16.67.124:33409: Network error: Client connection negotiation failed: client connection to 127.16.67.124:33409: connect: Connection refused (error 111)
02:03:54.865 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.864367 16771 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } attempt: 3
02:03:54.873 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:54.873214 16771 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.16.67.124:33409: Network error: Client connection negotiation failed: client connection to 127.16.67.124:33409: connect: Connection refused (error 111)
02:03:54.968 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:54.966699 16887 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } attempt: 5
02:03:54.976 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:54.975625 16887 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.16.67.124:33409: Network error: Client connection negotiation failed: client connection to 127.16.67.124:33409: connect: Connection refused (error 111)
02:03:55.018 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:55.013765 16771 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } attempt: 4
02:03:55.024 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:55.023631 16771 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.16.67.124:33409: Network error: Client connection negotiation failed: client connection to 127.16.67.124:33409: connect: Connection refused (error 111)
02:03:55.308 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:55.307180 16771 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } attempt: 5
02:03:55.315 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:55.315160 16771 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.16.67.124:33409: Network error: Client connection negotiation failed: client connection to 127.16.67.124:33409: connect: Connection refused (error 111)
02:03:55.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:55.523391 16887 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } attempt: 6
02:03:55.563 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:55.563108 16887 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.16.67.124:33409: Network error: Client connection negotiation failed: client connection to 127.16.67.124:33409: connect: Connection refused (error 111)
02:03:55.869 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:55.867695 16771 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } attempt: 6
02:03:55.895 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:55.895167 16771 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.16.67.124:33409: Network error: Client connection negotiation failed: client connection to 127.16.67.124:33409: connect: Connection refused (error 111)
02:03:56.308 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:56.082939 16905 debug-util.cc:398] Leaking SignalData structure 0x7b080002ee00 after lost signal to thread 16889
02:03:56.309 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:56.308681 16889 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.629s	user 0.595s	sys 0.985s
02:03:56.309 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:56.309171 16889 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.630s	user 0.595s	sys 0.985s
02:03:56.310 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:54.680795 16907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:03:56.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:56.311040 16911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:03:56.314 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:56.314258 16909 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1630 milliseconds
02:03:56.314 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.314263 16889 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
02:03:56.316 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.315726 16889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
02:03:56.318 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.318063 16889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
02:03:56.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.319401 16889 hybrid_clock.cc:648] HybridClock initialized: now 1786500236319378 us; error 52 us; skew 500 ppm
02:03:56.320 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.320238 16889 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
02:03:56.327 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.326843 16889 webserver.cc:533] Webserver started at http://127.16.67.124:36221/ using document root <none> and password file <none>
02:03:56.328 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.327754 16889 fs_manager.cc:362] Metadata directory not provided
02:03:56.328 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.327947 16889 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
02:03:56.328 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.328416 16889 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
02:03:56.333 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.332888 16889 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-2/data/instance:
02:03:56.333 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "90cc59c231544922bc3df97e621bc023"
02:03:56.333 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-12 02:03:56 on dist-test-slave-h6n0"
02:03:56.334 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.334159 16889 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-2/wal/instance:
02:03:56.334 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "90cc59c231544922bc3df97e621bc023"
02:03:56.334 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-12 02:03:56 on dist-test-slave-h6n0"
02:03:56.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.342710 16889 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.005s	sys 0.001s
02:03:56.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.349696 16928 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
02:03:56.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.351023 16889 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.007s	sys 0.000s
02:03:56.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.351322 16889 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-2/data,/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-2/wal
02:03:56.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "90cc59c231544922bc3df97e621bc023"
02:03:56.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-12 02:03:56 on dist-test-slave-h6n0"
02:03:56.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.351627 16889 fs_report.cc:389] FS layout report
02:03:56.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
02:03:56.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-2/wal
02:03:56.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-2/wal
02:03:56.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-2/data/data
02:03:56.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
02:03:56.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
02:03:56.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
02:03:56.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
02:03:56.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
02:03:56.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
02:03:56.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
02:03:56.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
02:03:56.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
02:03:56.353 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
02:03:56.353 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
02:03:56.403 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.402796 16889 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
02:03:56.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.404385 16889 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
02:03:56.405 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.404861 16889 kserver.cc:163] Server-wide thread pool size limit: 3276
02:03:56.487 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.486509 16889 rpc_server.cc:307] RPC server started. Bound to: 127.16.67.124:33409
02:03:56.487 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.486617 17003 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.67.124:33409 every 8 connection(s)
02:03:56.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.490031 16889 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-2/data/info.pb
02:03:56.494 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.493695 16653 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskzRoOVp/build/tsan/bin/kudu as pid 16889
02:03:56.494 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.494045 16653 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/master-2/wal/instance
02:03:56.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.496769 17007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
02:03:56.520 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.516129 17007 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.67.126" port: 34295 } has no permanent_uuid. Determining permanent_uuid...
02:03:56.544 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.543310 17007 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } has no permanent_uuid. Determining permanent_uuid...
02:03:56.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.554584 17007 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } has no permanent_uuid. Determining permanent_uuid...
02:03:56.574 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.574115 17007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023: Bootstrap starting.
02:03:56.581 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.581625 17007 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023: Neither blocks nor log segments found. Creating new log.
02:03:56.584 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.583742 17007 log.cc:826] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023: Log is configured to *not* fsync() on all Append() calls
02:03:56.588 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.588476 17007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023: No bootstrap required, opened a new log
02:03:56.608 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.608170 17007 raft_consensus.cc:359] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657ce041f6d24ed9a826f7aab333f9f5" member_type: VOTER last_known_addr { host: "127.16.67.126" port: 34295 } } peers { permanent_uuid: "fd29b0c9750c4be7acec1a78c70549a6" member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } } peers { permanent_uuid: "90cc59c231544922bc3df97e621bc023" member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } }
02:03:56.609 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.608842 17007 raft_consensus.cc:385] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
02:03:56.609 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.609035 17007 raft_consensus.cc:740] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90cc59c231544922bc3df97e621bc023, State: Initialized, Role: FOLLOWER
02:03:56.610 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.609704 17007 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657ce041f6d24ed9a826f7aab333f9f5" member_type: VOTER last_known_addr { host: "127.16.67.126" port: 34295 } } peers { permanent_uuid: "fd29b0c9750c4be7acec1a78c70549a6" member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } } peers { permanent_uuid: "90cc59c231544922bc3df97e621bc023" member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } }
02:03:56.613 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.612778 17019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657ce041f6d24ed9a826f7aab333f9f5" member_type: VOTER last_known_addr { host: "127.16.67.126" port: 34295 } } peers { permanent_uuid: "fd29b0c9750c4be7acec1a78c70549a6" member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } } peers { permanent_uuid: "90cc59c231544922bc3df97e621bc023" member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } } }
02:03:56.614 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.614225 17019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [sys.catalog]: This master's current role is: FOLLOWER
02:03:56.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.614647 17007 sys_catalog.cc:565] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [sys.catalog]: configured and running, proceeding with master startup.
02:03:56.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.623816 16887 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } attempt: 7
02:03:56.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:56.648902 17036 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
02:03:56.650 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:56.649814 17036 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
02:03:56.653 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.652865 16887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd29b0c9750c4be7acec1a78c70549a6: Bootstrap starting.
02:03:56.661 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.661361 16887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd29b0c9750c4be7acec1a78c70549a6: Neither blocks nor log segments found. Creating new log.
02:03:56.662 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:56.661396 16936 tablet.cc:2406] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023: Can't schedule compaction. Clean time has not been advanced past its initial value.
02:03:56.672 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.672519 16887 log.cc:826] T 00000000000000000000000000000000 P fd29b0c9750c4be7acec1a78c70549a6: Log is configured to *not* fsync() on all Append() calls
02:03:56.673 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.646605 16889 master_runner.cc:187] Error getting master registration for 127.16.67.126:34295: OK, instance_id { permanent_uuid: "657ce041f6d24ed9a826f7aab333f9f5" instance_seqno: 1786500232209913 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
02:03:56.673 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.673395 16889 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
02:03:56.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.678186 16887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd29b0c9750c4be7acec1a78c70549a6: No bootstrap required, opened a new log
02:03:56.701 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.700207 16887 raft_consensus.cc:359] T 00000000000000000000000000000000 P fd29b0c9750c4be7acec1a78c70549a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657ce041f6d24ed9a826f7aab333f9f5" member_type: VOTER last_known_addr { host: "127.16.67.126" port: 34295 } } peers { permanent_uuid: "fd29b0c9750c4be7acec1a78c70549a6" member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } } peers { permanent_uuid: "90cc59c231544922bc3df97e621bc023" member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } }
02:03:56.701 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.700907 16887 raft_consensus.cc:385] T 00000000000000000000000000000000 P fd29b0c9750c4be7acec1a78c70549a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
02:03:56.701 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.701195 16887 raft_consensus.cc:740] T 00000000000000000000000000000000 P fd29b0c9750c4be7acec1a78c70549a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd29b0c9750c4be7acec1a78c70549a6, State: Initialized, Role: FOLLOWER
02:03:56.702 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.702209 16887 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd29b0c9750c4be7acec1a78c70549a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657ce041f6d24ed9a826f7aab333f9f5" member_type: VOTER last_known_addr { host: "127.16.67.126" port: 34295 } } peers { permanent_uuid: "fd29b0c9750c4be7acec1a78c70549a6" member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } } peers { permanent_uuid: "90cc59c231544922bc3df97e621bc023" member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } }
02:03:56.705 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.704841 17039 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd29b0c9750c4be7acec1a78c70549a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657ce041f6d24ed9a826f7aab333f9f5" member_type: VOTER last_known_addr { host: "127.16.67.126" port: 34295 } } peers { permanent_uuid: "fd29b0c9750c4be7acec1a78c70549a6" member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } } peers { permanent_uuid: "90cc59c231544922bc3df97e621bc023" member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } } }
02:03:56.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.706722 16887 sys_catalog.cc:565] T 00000000000000000000000000000000 P fd29b0c9750c4be7acec1a78c70549a6 [sys.catalog]: configured and running, proceeding with master startup.
02:03:56.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.705948 17039 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd29b0c9750c4be7acec1a78c70549a6 [sys.catalog]: This master's current role is: FOLLOWER
02:03:56.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:56.741021 17054 catalog_manager.cc:1594] T 00000000000000000000000000000000 P fd29b0c9750c4be7acec1a78c70549a6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
02:03:56.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:56.741909 17054 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
02:03:56.772 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.737077 16776 master_runner.cc:187] Error getting master registration for 127.16.67.126:34295: OK, instance_id { permanent_uuid: "657ce041f6d24ed9a826f7aab333f9f5" instance_seqno: 1786500232209913 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
02:03:56.772 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.772042 16776 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
02:03:56.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.800168 17019 raft_consensus.cc:493] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
02:03:56.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.800768 17019 raft_consensus.cc:515] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657ce041f6d24ed9a826f7aab333f9f5" member_type: VOTER last_known_addr { host: "127.16.67.126" port: 34295 } } peers { permanent_uuid: "fd29b0c9750c4be7acec1a78c70549a6" member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } } peers { permanent_uuid: "90cc59c231544922bc3df97e621bc023" member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } }
02:03:56.806 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.805601 17019 leader_election.cc:290] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657ce041f6d24ed9a826f7aab333f9f5 (127.16.67.126:34295), fd29b0c9750c4be7acec1a78c70549a6 (127.16.67.125:32941)
02:03:56.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.806686 16850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "90cc59c231544922bc3df97e621bc023" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd29b0c9750c4be7acec1a78c70549a6" is_pre_election: true
02:03:56.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.807623 16850 raft_consensus.cc:2468] T 00000000000000000000000000000000 P fd29b0c9750c4be7acec1a78c70549a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90cc59c231544922bc3df97e621bc023 in term 0.
02:03:56.810 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.809473 16929 leader_election.cc:304] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90cc59c231544922bc3df97e621bc023, fd29b0c9750c4be7acec1a78c70549a6; no voters: 
02:03:56.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.810977 17019 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [term 0 FOLLOWER]: Leader pre-election won for term 1
02:03:56.812 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.812255 17019 raft_consensus.cc:493] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
02:03:56.813 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.813165 17019 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [term 0 FOLLOWER]: Advancing to term 1
02:03:56.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.823082 17019 raft_consensus.cc:515] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657ce041f6d24ed9a826f7aab333f9f5" member_type: VOTER last_known_addr { host: "127.16.67.126" port: 34295 } } peers { permanent_uuid: "fd29b0c9750c4be7acec1a78c70549a6" member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } } peers { permanent_uuid: "90cc59c231544922bc3df97e621bc023" member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } }
02:03:56.834 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.827726 16850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "90cc59c231544922bc3df97e621bc023" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd29b0c9750c4be7acec1a78c70549a6"
02:03:56.834 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.828298 16850 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fd29b0c9750c4be7acec1a78c70549a6 [term 0 FOLLOWER]: Advancing to term 1
02:03:56.836 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.836114 16850 raft_consensus.cc:2468] T 00000000000000000000000000000000 P fd29b0c9750c4be7acec1a78c70549a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90cc59c231544922bc3df97e621bc023 in term 1.
02:03:56.837 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.837196 17019 leader_election.cc:290] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [CANDIDATE]: Term 1 election: Requested vote from peers 657ce041f6d24ed9a826f7aab333f9f5 (127.16.67.126:34295), fd29b0c9750c4be7acec1a78c70549a6 (127.16.67.125:32941)
02:03:56.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.837188 16929 leader_election.cc:304] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90cc59c231544922bc3df97e621bc023, fd29b0c9750c4be7acec1a78c70549a6; no voters: 
02:03:56.843 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.842846 17019 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [term 1 FOLLOWER]: Leader election won for term 1
02:03:56.844 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.844229 17019 raft_consensus.cc:697] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [term 1 LEADER]: Becoming Leader. State: Replica: 90cc59c231544922bc3df97e621bc023, State: Running, Role: LEADER
02:03:56.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.846014 17019 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657ce041f6d24ed9a826f7aab333f9f5" member_type: VOTER last_known_addr { host: "127.16.67.126" port: 34295 } } peers { permanent_uuid: "fd29b0c9750c4be7acec1a78c70549a6" member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } } peers { permanent_uuid: "90cc59c231544922bc3df97e621bc023" member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } }
02:03:56.868 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.866987 17019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90cc59c231544922bc3df97e621bc023. Latest consensus state: current_term: 1 leader_uuid: "90cc59c231544922bc3df97e621bc023" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657ce041f6d24ed9a826f7aab333f9f5" member_type: VOTER last_known_addr { host: "127.16.67.126" port: 34295 } } peers { permanent_uuid: "fd29b0c9750c4be7acec1a78c70549a6" member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } } peers { permanent_uuid: "90cc59c231544922bc3df97e621bc023" member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } } }
02:03:56.868 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.867704 17019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [sys.catalog]: This master's current role is: LEADER
02:03:56.872 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.871429 17060 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
02:03:56.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.880255 17060 catalog_manager.cc:1520] Initializing Kudu cluster ID...
02:03:56.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:56.889317 16810 tablet.cc:2406] T 00000000000000000000000000000000 P fd29b0c9750c4be7acec1a78c70549a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
02:03:56.908 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.907903 16850 raft_consensus.cc:1275] T 00000000000000000000000000000000 P fd29b0c9750c4be7acec1a78c70549a6 [term 1 FOLLOWER]: Refusing update from remote peer 90cc59c231544922bc3df97e621bc023: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
02:03:56.912 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.910501 17057 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd29b0c9750c4be7acec1a78c70549a6" member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
02:03:56.929 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.928647 16771 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } attempt: 7
02:03:56.954 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.950109 17039 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd29b0c9750c4be7acec1a78c70549a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90cc59c231544922bc3df97e621bc023. Latest consensus state: current_term: 1 leader_uuid: "90cc59c231544922bc3df97e621bc023" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657ce041f6d24ed9a826f7aab333f9f5" member_type: VOTER last_known_addr { host: "127.16.67.126" port: 34295 } } peers { permanent_uuid: "fd29b0c9750c4be7acec1a78c70549a6" member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } } peers { permanent_uuid: "90cc59c231544922bc3df97e621bc023" member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } } }
02:03:56.955 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.955196 17039 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd29b0c9750c4be7acec1a78c70549a6 [sys.catalog]: This master's current role is: FOLLOWER
02:03:56.995 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.994302 17039 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd29b0c9750c4be7acec1a78c70549a6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "90cc59c231544922bc3df97e621bc023" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657ce041f6d24ed9a826f7aab333f9f5" member_type: VOTER last_known_addr { host: "127.16.67.126" port: 34295 } } peers { permanent_uuid: "fd29b0c9750c4be7acec1a78c70549a6" member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } } peers { permanent_uuid: "90cc59c231544922bc3df97e621bc023" member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } } }
02:03:56.996 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.995092 17039 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd29b0c9750c4be7acec1a78c70549a6 [sys.catalog]: This master's current role is: FOLLOWER
02:03:56.998 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:56.994927 16771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 657ce041f6d24ed9a826f7aab333f9f5: Bootstrap starting.
02:03:57.014 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.008848 17060 catalog_manager.cc:1383] Generated new cluster ID: 7c58821f37764f73a5a4a450ade0d080
02:03:57.014 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.009151 17060 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
02:03:57.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.015007 16771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 657ce041f6d24ed9a826f7aab333f9f5: Neither blocks nor log segments found. Creating new log.
02:03:57.018 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.017717 16771 log.cc:826] T 00000000000000000000000000000000 P 657ce041f6d24ed9a826f7aab333f9f5: Log is configured to *not* fsync() on all Append() calls
02:03:57.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.021955 17057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "90cc59c231544922bc3df97e621bc023" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657ce041f6d24ed9a826f7aab333f9f5" member_type: VOTER last_known_addr { host: "127.16.67.126" port: 34295 } } peers { permanent_uuid: "fd29b0c9750c4be7acec1a78c70549a6" member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } } peers { permanent_uuid: "90cc59c231544922bc3df97e621bc023" member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } } }
02:03:57.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.022784 17057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [sys.catalog]: This master's current role is: LEADER
02:03:57.031 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.031219 16771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 657ce041f6d24ed9a826f7aab333f9f5: No bootstrap required, opened a new log
02:03:57.065 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.063555 16771 raft_consensus.cc:359] T 00000000000000000000000000000000 P 657ce041f6d24ed9a826f7aab333f9f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657ce041f6d24ed9a826f7aab333f9f5" member_type: VOTER last_known_addr { host: "127.16.67.126" port: 34295 } } peers { permanent_uuid: "fd29b0c9750c4be7acec1a78c70549a6" member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } } peers { permanent_uuid: "90cc59c231544922bc3df97e621bc023" member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } }
02:03:57.066 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.065981 16771 raft_consensus.cc:385] T 00000000000000000000000000000000 P 657ce041f6d24ed9a826f7aab333f9f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
02:03:57.067 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.066598 16771 raft_consensus.cc:740] T 00000000000000000000000000000000 P 657ce041f6d24ed9a826f7aab333f9f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657ce041f6d24ed9a826f7aab333f9f5, State: Initialized, Role: FOLLOWER
02:03:57.071 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.067957 16771 consensus_queue.cc:260] T 00000000000000000000000000000000 P 657ce041f6d24ed9a826f7aab333f9f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657ce041f6d24ed9a826f7aab333f9f5" member_type: VOTER last_known_addr { host: "127.16.67.126" port: 34295 } } peers { permanent_uuid: "fd29b0c9750c4be7acec1a78c70549a6" member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } } peers { permanent_uuid: "90cc59c231544922bc3df97e621bc023" member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } }
02:03:57.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.074647 17060 catalog_manager.cc:1406] Generated new certificate authority record
02:03:57.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.079878 17060 catalog_manager.cc:1540] Loading token signing keys...
02:03:57.090 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.087711 16728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "90cc59c231544922bc3df97e621bc023" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657ce041f6d24ed9a826f7aab333f9f5"
02:03:57.090 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.088701 16733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "90cc59c231544922bc3df97e621bc023" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657ce041f6d24ed9a826f7aab333f9f5" is_pre_election: true
02:03:57.094 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.090925 16728 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 657ce041f6d24ed9a826f7aab333f9f5 [term 0 FOLLOWER]: Advancing to term 1
02:03:57.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.098248 16728 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 657ce041f6d24ed9a826f7aab333f9f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90cc59c231544922bc3df97e621bc023 in term 1.
02:03:57.103 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.102643 16771 sys_catalog.cc:565] T 00000000000000000000000000000000 P 657ce041f6d24ed9a826f7aab333f9f5 [sys.catalog]: configured and running, proceeding with master startup.
02:03:57.106 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.103318 17071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 657ce041f6d24ed9a826f7aab333f9f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657ce041f6d24ed9a826f7aab333f9f5" member_type: VOTER last_known_addr { host: "127.16.67.126" port: 34295 } } peers { permanent_uuid: "fd29b0c9750c4be7acec1a78c70549a6" member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } } peers { permanent_uuid: "90cc59c231544922bc3df97e621bc023" member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } } }
02:03:57.108 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.107661 17071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 657ce041f6d24ed9a826f7aab333f9f5 [sys.catalog]: This master's current role is: FOLLOWER
02:03:57.115 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.112615 16727 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 657ce041f6d24ed9a826f7aab333f9f5 [term 1 FOLLOWER]: Refusing update from remote peer 90cc59c231544922bc3df97e621bc023: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
02:03:57.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.117323 17019 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657ce041f6d24ed9a826f7aab333f9f5" member_type: VOTER last_known_addr { host: "127.16.67.126" port: 34295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
02:03:57.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.119747 17060 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023: Generated new TSK 0
02:03:57.121 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.121146 17060 catalog_manager.cc:1550] Initializing in-progress tserver states...
02:03:57.152 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.151834 16653 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskzRoOVp/build/tsan/bin/kudu
02:03:57.152 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskzRoOVp/build/tsan/bin/kudu
02:03:57.152 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-0/wal
02:03:57.152 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-0/data
02:03:57.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
02:03:57.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
02:03:57.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
02:03:57.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:03:57.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:03:57.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:03:57.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-0/logs
02:03:57.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-0/data/info.pb
02:03:57.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:03:57.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
02:03:57.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
02:03:57.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
02:03:57.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
02:03:57.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:03:57.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:03:57.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:03:57.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
02:03:57.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
02:03:57.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.16.67.65:0
02:03:57.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.16.67.65
02:03:57.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.16.67.65
02:03:57.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
02:03:57.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.16.67.126:34295,127.16.67.125:32941,127.16.67.124:33409
02:03:57.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.16.67.84:39899
02:03:57.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:03:57.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:03:57.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
02:03:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.250674 17071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 657ce041f6d24ed9a826f7aab333f9f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90cc59c231544922bc3df97e621bc023. Latest consensus state: current_term: 1 leader_uuid: "90cc59c231544922bc3df97e621bc023" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657ce041f6d24ed9a826f7aab333f9f5" member_type: VOTER last_known_addr { host: "127.16.67.126" port: 34295 } } peers { permanent_uuid: "fd29b0c9750c4be7acec1a78c70549a6" member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } } peers { permanent_uuid: "90cc59c231544922bc3df97e621bc023" member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } } }
02:03:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.251052 17084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 657ce041f6d24ed9a826f7aab333f9f5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "90cc59c231544922bc3df97e621bc023" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657ce041f6d24ed9a826f7aab333f9f5" member_type: VOTER last_known_addr { host: "127.16.67.126" port: 34295 } } peers { permanent_uuid: "fd29b0c9750c4be7acec1a78c70549a6" member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } } peers { permanent_uuid: "90cc59c231544922bc3df97e621bc023" member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } } }
02:03:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.251441 17071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 657ce041f6d24ed9a826f7aab333f9f5 [sys.catalog]: This master's current role is: FOLLOWER
02:03:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.251744 17084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 657ce041f6d24ed9a826f7aab333f9f5 [sys.catalog]: This master's current role is: FOLLOWER
02:03:57.288 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.286751 17100 catalog_manager.cc:1295] Loaded cluster ID: 7c58821f37764f73a5a4a450ade0d080
02:03:57.289 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.288937 17100 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 657ce041f6d24ed9a826f7aab333f9f5: loading cluster ID for follower catalog manager: success
02:03:57.296 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.296306 17100 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 657ce041f6d24ed9a826f7aab333f9f5: acquiring CA information for follower catalog manager: success
02:03:57.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.311985 17100 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 657ce041f6d24ed9a826f7aab333f9f5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
02:03:57.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.495463 17019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "90cc59c231544922bc3df97e621bc023" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657ce041f6d24ed9a826f7aab333f9f5" member_type: VOTER last_known_addr { host: "127.16.67.126" port: 34295 } } peers { permanent_uuid: "fd29b0c9750c4be7acec1a78c70549a6" member_type: VOTER last_known_addr { host: "127.16.67.125" port: 32941 } } peers { permanent_uuid: "90cc59c231544922bc3df97e621bc023" member_type: VOTER last_known_addr { host: "127.16.67.124" port: 33409 } } }
02:03:57.498 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.496236 17019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90cc59c231544922bc3df97e621bc023 [sys.catalog]: This master's current role is: LEADER
02:03:57.749 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.748705 17054 catalog_manager.cc:1295] Loaded cluster ID: 7c58821f37764f73a5a4a450ade0d080
02:03:57.749 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.749481 17054 catalog_manager.cc:1588] T 00000000000000000000000000000000 P fd29b0c9750c4be7acec1a78c70549a6: loading cluster ID for follower catalog manager: success
02:03:57.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.757125 17054 catalog_manager.cc:1610] T 00000000000000000000000000000000 P fd29b0c9750c4be7acec1a78c70549a6: acquiring CA information for follower catalog manager: success
02:03:57.776 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:57.776135 17054 catalog_manager.cc:1638] T 00000000000000000000000000000000 P fd29b0c9750c4be7acec1a78c70549a6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
02:03:57.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:57.898126 17083 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
02:03:57.899 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:57.899417 17083 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
02:03:57.900 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:57.900476 17083 flags.cc:432] Enabled unsafe flag: --never_fsync=true
02:03:57.975 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:57.974633 17083 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
02:03:57.976 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:57.976667 17083 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.67.65
02:03:58.058 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:58.058002 17083 tablet_server_runner.cc:78] Tablet server non-default flags:
02:03:58.058 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-0/logs
02:03:58.059 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:03:58.059 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
02:03:58.059 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:03:58.059 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.16.67.84:39899
02:03:58.059 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:03:58.059 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:03:58.059 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-0/data
02:03:58.059 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-0/wal
02:03:58.059 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:03:58.059 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:03:58.059 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.16.67.65:0
02:03:58.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
02:03:58.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:03:58.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:03:58.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-0/data/info.pb
02:03:58.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.16.67.65
02:03:58.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
02:03:58.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.16.67.126:34295,127.16.67.125:32941,127.16.67.124:33409
02:03:58.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
02:03:58.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:03:58.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
02:03:58.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
02:03:58.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:03:58.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.16.67.65
02:03:58.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
02:03:58.061 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
02:03:58.061 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
02:03:58.061 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 1d9153c248733781bf69132f2d094d9d3c92b2bc
02:03:58.061 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
02:03:58.061 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 12 Aug 2026 01:45:55 UTC on e7f111948823
02:03:58.061 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 13299
02:03:58.061 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
02:03:58.070 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:58.070415 17083 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
02:03:58.073 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:03:58.073348 17083 file_cache.cc:504] Constructed file cache file cache with capacity 419430
02:03:58.112 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:58.111778 17110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:04:00.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:59.506728 17109 debug-util.cc:398] Leaking SignalData structure 0x7b080002ede0 after lost signal to thread 17083
02:04:00.887 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:00.879035 17109 kernel_stack_watchdog.cc:198] Thread 17083 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:642 for 387ms:
02:04:00.887 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Kernel stack:
02:04:00.887 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) (could not read kernel stack)
02:04:00.887 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) User stack:
02:04:00.887 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) <Timed out: thread did not respond: maybe it is blocking signals>
02:04:00.887 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:00.882318 17083 thread.cc:641] GCE (cloud detector) Time spent creating pthread: real 2.766s	user 0.919s	sys 1.577s
02:04:00.888 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:00.882827 17083 thread.cc:608] GCE (cloud detector) Time spent starting thread: real 2.767s	user 0.919s	sys 1.577s
02:04:00.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:03:58.120462 17111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:04:00.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:00.903265 17117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:04:00.911 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:00.910903 17083 server_base.cc:1061] running on GCE node
02:04:00.913 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:00.912562 17083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
02:04:00.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:00.916688 17083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
02:04:00.918 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:00.918193 17083 hybrid_clock.cc:648] HybridClock initialized: now 1786500240917998 us; error 185 us; skew 500 ppm
02:04:00.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:00.919456 17083 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
02:04:00.935 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:00.935427 17083 webserver.cc:533] Webserver started at http://127.16.67.65:41931/ using document root <none> and password file <none>
02:04:00.937 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:00.936722 17083 fs_manager.cc:362] Metadata directory not provided
02:04:00.937 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:00.937019 17083 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
02:04:00.937 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:00.937621 17083 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
02:04:00.945 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:00.944844 17083 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-0/data/instance:
02:04:00.945 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "9e0ec91b729b44319839f751e036d8c1"
02:04:00.945 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-12 02:04:00 on dist-test-slave-h6n0"
02:04:00.946 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:00.946594 17083 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-0/wal/instance:
02:04:00.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "9e0ec91b729b44319839f751e036d8c1"
02:04:00.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-12 02:04:00 on dist-test-slave-h6n0"
02:04:00.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:00.957898 17083 fs_manager.cc:696] Time spent creating directory manager: real 0.010s	user 0.003s	sys 0.006s
02:04:00.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:00.966173 17125 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
02:04:00.967 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:00.967597 17083 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.003s	sys 0.000s
02:04:00.968 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:00.968050 17083 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-0/data,/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-0/wal
02:04:00.968 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "9e0ec91b729b44319839f751e036d8c1"
02:04:00.968 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-12 02:04:00 on dist-test-slave-h6n0"
02:04:00.969 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:00.968524 17083 fs_report.cc:389] FS layout report
02:04:00.969 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
02:04:00.969 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-0/wal
02:04:00.969 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-0/wal
02:04:00.969 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-0/data/data
02:04:00.969 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
02:04:00.969 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
02:04:00.969 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
02:04:00.969 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
02:04:00.969 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
02:04:00.969 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
02:04:00.970 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
02:04:00.970 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
02:04:00.970 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
02:04:00.970 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
02:04:00.970 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
02:04:01.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.051317 17083 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
02:04:01.053 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.053396 17083 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
02:04:01.054 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.053966 17083 kserver.cc:163] Server-wide thread pool size limit: 3276
02:04:01.058 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.057703 17083 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
02:04:01.064 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.063851 17083 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
02:04:01.064 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.064136 17083 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
02:04:01.064 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.064448 17083 ts_tablet_manager.cc:616] Registered 0 tablets
02:04:01.065 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.064647 17083 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
02:04:01.305 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.304679 17083 rpc_server.cc:307] RPC server started. Bound to: 127.16.67.65:36583
02:04:01.305 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.304877 17312 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.67.65:36583 every 8 connection(s)
02:04:01.307 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.307638 17083 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-0/data/info.pb
02:04:01.314 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.313534 16653 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskzRoOVp/build/tsan/bin/kudu as pid 17083
02:04:01.314 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.314356 16653 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-0/wal/instance
02:04:01.342 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.342227 16653 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskzRoOVp/build/tsan/bin/kudu
02:04:01.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskzRoOVp/build/tsan/bin/kudu
02:04:01.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-1/wal
02:04:01.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-1/data
02:04:01.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
02:04:01.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
02:04:01.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
02:04:01.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:04:01.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:04:01.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:04:01.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-1/logs
02:04:01.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-1/data/info.pb
02:04:01.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:04:01.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
02:04:01.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
02:04:01.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
02:04:01.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
02:04:01.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:04:01.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:04:01.344 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:04:01.344 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
02:04:01.344 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
02:04:01.344 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.16.67.66:0
02:04:01.344 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.16.67.66
02:04:01.344 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.16.67.66
02:04:01.344 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
02:04:01.344 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.16.67.126:34295,127.16.67.125:32941,127.16.67.124:33409
02:04:01.344 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.16.67.84:39899
02:04:01.344 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:04:01.344 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:04:01.344 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
02:04:01.402 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.401535 17316 heartbeater.cc:344] Connected to a master server at 127.16.67.125:32941
02:04:01.402 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.402086 17316 heartbeater.cc:461] Registering TS with master...
02:04:01.405 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.404743 17316 heartbeater.cc:507] Master 127.16.67.125:32941 requested a full tablet report, sending...
02:04:01.407 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.407467 17314 heartbeater.cc:344] Connected to a master server at 127.16.67.124:33409
02:04:01.408 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.407864 17314 heartbeater.cc:461] Registering TS with master...
02:04:01.409 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.408841 17314 heartbeater.cc:507] Master 127.16.67.124:33409 requested a full tablet report, sending...
02:04:01.412 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.411442 16826 ts_manager.cc:194] Registered new tserver with Master: 9e0ec91b729b44319839f751e036d8c1 (127.16.67.65:36583)
02:04:01.412 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.412055 16951 ts_manager.cc:194] Registered new tserver with Master: 9e0ec91b729b44319839f751e036d8c1 (127.16.67.65:36583)
02:04:01.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.415329 16951 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.16.67.65:52559
02:04:01.416 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.416230 17317 heartbeater.cc:344] Connected to a master server at 127.16.67.126:34295
02:04:01.416 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.416584 17317 heartbeater.cc:461] Registering TS with master...
02:04:01.417 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.417523 17317 heartbeater.cc:507] Master 127.16.67.126:34295 requested a full tablet report, sending...
02:04:01.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.421216 16716 ts_manager.cc:194] Registered new tserver with Master: 9e0ec91b729b44319839f751e036d8c1 (127.16.67.65:36583)
02:04:01.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:01.793787 17326 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
02:04:01.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:01.794615 17326 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
02:04:01.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:01.795498 17326 flags.cc:432] Enabled unsafe flag: --never_fsync=true
02:04:01.836 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:01.836377 17326 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
02:04:01.837 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:01.837657 17326 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.67.66
02:04:01.876 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.875965 17326 tablet_server_runner.cc:78] Tablet server non-default flags:
02:04:01.876 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-1/logs
02:04:01.876 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:04:01.876 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
02:04:01.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:04:01.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.16.67.84:39899
02:04:01.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:04:01.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:04:01.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-1/data
02:04:01.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-1/wal
02:04:01.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:04:01.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:04:01.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.16.67.66:0
02:04:01.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
02:04:01.878 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:04:01.878 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:04:01.878 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-1/data/info.pb
02:04:01.878 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.16.67.66
02:04:01.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
02:04:01.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.16.67.126:34295,127.16.67.125:32941,127.16.67.124:33409
02:04:01.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
02:04:01.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:04:01.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
02:04:01.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
02:04:01.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:04:01.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.16.67.66
02:04:01.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
02:04:01.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
02:04:01.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
02:04:01.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 1d9153c248733781bf69132f2d094d9d3c92b2bc
02:04:01.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
02:04:01.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 12 Aug 2026 01:45:55 UTC on e7f111948823
02:04:01.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 13299
02:04:01.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
02:04:01.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.877873 17326 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
02:04:01.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:01.880651 17326 file_cache.cc:504] Constructed file cache file cache with capacity 419430
02:04:01.896 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:01.895339 17342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:04:02.427 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:02.425040 17314 heartbeater.cc:499] Master 127.16.67.124:33409 was elected leader, sending a full tablet report...
02:04:03.332 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:01.896286 17351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:04:03.332 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:03.332360 17326 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.437s	user 0.428s	sys 0.974s
02:04:03.333 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:03.333168 17326 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.438s	user 0.429s	sys 0.974s
02:04:03.333 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:03.299448 17341 debug-util.cc:398] Leaking SignalData structure 0x7b080002ede0 after lost signal to thread 17326
02:04:03.335 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:03.334640 17353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:04:03.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:03.338225 17352 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1440 milliseconds
02:04:03.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.338254 17326 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
02:04:03.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.339560 17326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
02:04:03.342 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.341732 17326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
02:04:03.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.343142 17326 hybrid_clock.cc:648] HybridClock initialized: now 1786500243343100 us; error 40 us; skew 500 ppm
02:04:03.344 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.344156 17326 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
02:04:03.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.351658 17326 webserver.cc:533] Webserver started at http://127.16.67.66:35661/ using document root <none> and password file <none>
02:04:03.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.352692 17326 fs_manager.cc:362] Metadata directory not provided
02:04:03.353 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.352950 17326 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
02:04:03.353 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.353555 17326 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
02:04:03.359 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.358521 17326 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-1/data/instance:
02:04:03.359 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "f02b99900ae14094a50f21ad3593d551"
02:04:03.359 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-12 02:04:03 on dist-test-slave-h6n0"
02:04:03.360 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.359949 17326 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-1/wal/instance:
02:04:03.360 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "f02b99900ae14094a50f21ad3593d551"
02:04:03.360 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-12 02:04:03 on dist-test-slave-h6n0"
02:04:03.367 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.367532 17326 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.005s	sys 0.002s
02:04:03.373 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.373443 17367 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
02:04:03.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.374781 17326 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.001s
02:04:03.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.375109 17326 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-1/data,/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-1/wal
02:04:03.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "f02b99900ae14094a50f21ad3593d551"
02:04:03.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-12 02:04:03 on dist-test-slave-h6n0"
02:04:03.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.375403 17326 fs_report.cc:389] FS layout report
02:04:03.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
02:04:03.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-1/wal
02:04:03.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-1/wal
02:04:03.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-1/data/data
02:04:03.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
02:04:03.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
02:04:03.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
02:04:03.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
02:04:03.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
02:04:03.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
02:04:03.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
02:04:03.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
02:04:03.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
02:04:03.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
02:04:03.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
02:04:03.439 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.438875 17326 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
02:04:03.441 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.440901 17326 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
02:04:03.441 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.441321 17326 kserver.cc:163] Server-wide thread pool size limit: 3276
02:04:03.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.444023 17326 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
02:04:03.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.448824 17326 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
02:04:03.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.449059 17326 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
02:04:03.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.449296 17326 ts_tablet_manager.cc:616] Registered 0 tablets
02:04:03.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.449468 17326 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
02:04:03.673 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.673034 17326 rpc_server.cc:307] RPC server started. Bound to: 127.16.67.66:46107
02:04:03.673 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.673357 17552 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.67.66:46107 every 8 connection(s)
02:04:03.676 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.675809 17326 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-1/data/info.pb
02:04:03.684 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.683337 16653 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskzRoOVp/build/tsan/bin/kudu as pid 17326
02:04:03.684 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.683969 16653 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-1/wal/instance
02:04:03.701 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.700999 16653 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskzRoOVp/build/tsan/bin/kudu
02:04:03.701 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskzRoOVp/build/tsan/bin/kudu
02:04:03.701 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-2/wal
02:04:03.702 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-2/data
02:04:03.702 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
02:04:03.702 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
02:04:03.702 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
02:04:03.702 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:04:03.702 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:04:03.702 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:04:03.702 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-2/logs
02:04:03.702 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-2/data/info.pb
02:04:03.703 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:04:03.703 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
02:04:03.703 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
02:04:03.703 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
02:04:03.703 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
02:04:03.703 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:04:03.703 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:04:03.703 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:04:03.703 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
02:04:03.704 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
02:04:03.704 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.16.67.67:0
02:04:03.704 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.16.67.67
02:04:03.704 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.16.67.67
02:04:03.704 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
02:04:03.704 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.16.67.126:34295,127.16.67.125:32941,127.16.67.124:33409
02:04:03.704 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.16.67.84:39899
02:04:03.704 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:04:03.704 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:04:03.704 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
02:04:03.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.742324 17555 heartbeater.cc:344] Connected to a master server at 127.16.67.124:33409
02:04:03.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.742942 17555 heartbeater.cc:461] Registering TS with master...
02:04:03.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.743248 17556 heartbeater.cc:344] Connected to a master server at 127.16.67.125:32941
02:04:03.744 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.743585 17556 heartbeater.cc:461] Registering TS with master...
02:04:03.746 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.744593 17555 heartbeater.cc:507] Master 127.16.67.124:33409 requested a full tablet report, sending...
02:04:03.746 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.744946 17556 heartbeater.cc:507] Master 127.16.67.125:32941 requested a full tablet report, sending...
02:04:03.746 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.745961 17557 heartbeater.cc:344] Connected to a master server at 127.16.67.126:34295
02:04:03.746 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.746506 17557 heartbeater.cc:461] Registering TS with master...
02:04:03.750 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.748409 17557 heartbeater.cc:507] Master 127.16.67.126:34295 requested a full tablet report, sending...
02:04:03.750 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.748991 16826 ts_manager.cc:194] Registered new tserver with Master: f02b99900ae14094a50f21ad3593d551 (127.16.67.66:46107)
02:04:03.750 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.749722 16951 ts_manager.cc:194] Registered new tserver with Master: f02b99900ae14094a50f21ad3593d551 (127.16.67.66:46107)
02:04:03.752 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.752301 16951 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.16.67.66:44981
02:04:03.753 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:03.752718 16716 ts_manager.cc:194] Registered new tserver with Master: f02b99900ae14094a50f21ad3593d551 (127.16.67.66:46107)
02:04:04.102 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:04.102083 17573 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
02:04:04.102 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:04.102684 17573 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
02:04:04.103 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:04.103231 17573 flags.cc:432] Enabled unsafe flag: --never_fsync=true
02:04:04.138 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:04.137912 17573 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
02:04:04.139 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:04.138805 17573 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.67.67
02:04:04.176 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:04.176136 17573 tablet_server_runner.cc:78] Tablet server non-default flags:
02:04:04.176 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-2/logs
02:04:04.176 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:04:04.176 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
02:04:04.176 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:04:04.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.16.67.84:39899
02:04:04.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:04:04.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:04:04.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-2/data
02:04:04.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-2/wal
02:04:04.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:04:04.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:04:04.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.16.67.67:0
02:04:04.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
02:04:04.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:04:04.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:04:04.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-2/data/info.pb
02:04:04.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.16.67.67
02:04:04.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
02:04:04.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.16.67.126:34295,127.16.67.125:32941,127.16.67.124:33409
02:04:04.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
02:04:04.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:04:04.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
02:04:04.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
02:04:04.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:04:04.178 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.16.67.67
02:04:04.178 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
02:04:04.178 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
02:04:04.178 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
02:04:04.178 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 1d9153c248733781bf69132f2d094d9d3c92b2bc
02:04:04.178 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
02:04:04.178 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 12 Aug 2026 01:45:55 UTC on e7f111948823
02:04:04.178 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 13299
02:04:04.178 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
02:04:04.178 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:04.177511 17573 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
02:04:04.180 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:04.179548 17573 file_cache.cc:504] Constructed file cache file cache with capacity 419430
02:04:04.193 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:04.193452 17583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:04:04.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:04.757104 17555 heartbeater.cc:499] Master 127.16.67.124:33409 was elected leader, sending a full tablet report...
02:04:05.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:04.196066 17585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:04:05.533 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:05.533006 17587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:04:05.535 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:05.535521 17586 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1335 milliseconds
02:04:05.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.535619 17573 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
02:04:05.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.536957 17573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
02:04:05.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.539795 17573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
02:04:05.541 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.541240 17573 hybrid_clock.cc:648] HybridClock initialized: now 1786500245541188 us; error 53 us; skew 500 ppm
02:04:05.542 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.542037 17573 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
02:04:05.549 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.548842 17573 webserver.cc:533] Webserver started at http://127.16.67.67:34365/ using document root <none> and password file <none>
02:04:05.550 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.549841 17573 fs_manager.cc:362] Metadata directory not provided
02:04:05.550 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.550043 17573 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
02:04:05.550 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.550446 17573 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
02:04:05.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.554869 17573 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-2/data/instance:
02:04:05.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "91b849f0528142ed8048da0214d55cce"
02:04:05.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-12 02:04:05 on dist-test-slave-h6n0"
02:04:05.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.556586 17573 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-2/wal/instance:
02:04:05.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "91b849f0528142ed8048da0214d55cce"
02:04:05.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-12 02:04:05 on dist-test-slave-h6n0"
02:04:05.568 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.567726 17573 fs_manager.cc:696] Time spent creating directory manager: real 0.010s	user 0.007s	sys 0.004s
02:04:05.574 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.574424 17601 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
02:04:05.575 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.575639 17573 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.001s
02:04:05.576 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.576066 17573 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-2/data,/tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-2/wal
02:04:05.576 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "91b849f0528142ed8048da0214d55cce"
02:04:05.576 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-12 02:04:05 on dist-test-slave-h6n0"
02:04:05.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.576467 17573 fs_report.cc:389] FS layout report
02:04:05.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
02:04:05.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-2/wal
02:04:05.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-2/wal
02:04:05.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-2/data/data
02:04:05.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
02:04:05.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
02:04:05.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
02:04:05.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
02:04:05.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
02:04:05.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
02:04:05.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
02:04:05.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
02:04:05.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
02:04:05.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
02:04:05.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
02:04:05.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.637646 17573 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
02:04:05.639 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.639408 17573 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
02:04:05.640 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.639957 17573 kserver.cc:163] Server-wide thread pool size limit: 3276
02:04:05.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.642869 17573 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
02:04:05.648 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.647644 17573 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
02:04:05.648 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.647969 17573 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
02:04:05.648 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.648360 17573 ts_tablet_manager.cc:616] Registered 0 tablets
02:04:05.648 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.648571 17573 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
02:04:05.861 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.860811 17573 rpc_server.cc:307] RPC server started. Bound to: 127.16.67.67:44863
02:04:05.861 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.860942 17783 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.67.67:44863 every 8 connection(s)
02:04:05.864 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.863749 17573 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-2/data/info.pb
02:04:05.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.873661 16653 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskzRoOVp/build/tsan/bin/kudu as pid 17573
02:04:05.878 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.874392 16653 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskzRoOVp/test-tmp/mini-kudu-cluster10133144334745337585/ts-2/wal/instance
02:04:05.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.929955 17788 heartbeater.cc:344] Connected to a master server at 127.16.67.125:32941
02:04:05.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.930883 17788 heartbeater.cc:461] Registering TS with master...
02:04:05.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.932502 17788 heartbeater.cc:507] Master 127.16.67.125:32941 requested a full tablet report, sending...
02:04:05.937 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.936861 17784 heartbeater.cc:344] Connected to a master server at 127.16.67.124:33409
02:04:05.937 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.937218 17784 heartbeater.cc:461] Registering TS with master...
02:04:05.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.938117 17784 heartbeater.cc:507] Master 127.16.67.124:33409 requested a full tablet report, sending...
02:04:05.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.938683 16826 ts_manager.cc:194] Registered new tserver with Master: 91b849f0528142ed8048da0214d55cce (127.16.67.67:44863)
02:04:05.946 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.944264 16951 ts_manager.cc:194] Registered new tserver with Master: 91b849f0528142ed8048da0214d55cce (127.16.67.67:44863)
02:04:05.946 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.944442 17789 heartbeater.cc:344] Connected to a master server at 127.16.67.126:34295
02:04:05.946 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.944936 17789 heartbeater.cc:461] Registering TS with master...
02:04:05.946 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.946393 16951 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.16.67.67:33223
02:04:05.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.946161 17789 heartbeater.cc:507] Master 127.16.67.126:34295 requested a full tablet report, sending...
02:04:05.952 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.952078 16716 ts_manager.cc:194] Registered new tserver with Master: 91b849f0528142ed8048da0214d55cce (127.16.67.67:44863)
02:04:05.968 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:05.967875 16653 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
02:04:05.981 [DEBUG - main] (MiniKuduCluster.java:195) Response: 
02:04:05.983 [DEBUG - main] (MiniKuduCluster.java:185) Request: get_masters {
}

02:04:05.993 [DEBUG - main] (MiniKuduCluster.java:195) Response: get_masters {
  masters {
    id {
      type: MASTER
      index: 0
    }
    bound_rpc_address {
      host: "127.16.67.126"
      port: 34295
    }
    bound_http_address {
      host: "127.16.67.126"
      port: 41163
    }
  }
  masters {
    id {
      type: MASTER
      index: 1
    }
    bound_rpc_address {
      host: "127.16.67.125"
      port: 32941
    }
    bound_http_address {
      host: "127.16.67.125"
      port: 40205
    }
  }
  masters {
    id {
      type: MASTER
      index: 2
    }
    bound_rpc_address {
      host: "127.16.67.124"
      port: 33409
    }
    bound_http_address {
      host: "127.16.67.124"
      port: 36221
    }
  }
}

02:04:05.997 [DEBUG - main] (MiniKuduCluster.java:185) Request: get_tservers {
}

02:04:05.999 [DEBUG - main] (MiniKuduCluster.java:195) Response: get_tservers {
  tservers {
    id {
      type: TSERVER
      index: 0
    }
    bound_rpc_address {
      host: "127.16.67.65"
      port: 36583
    }
    bound_http_address {
      host: "127.16.67.65"
      port: 41931
    }
  }
  tservers {
    id {
      type: TSERVER
      index: 1
    }
    bound_rpc_address {
      host: "127.16.67.66"
      port: 46107
    }
    bound_http_address {
      host: "127.16.67.66"
      port: 35661
    }
  }
  tservers {
    id {
      type: TSERVER
      index: 2
    }
    bound_rpc_address {
      host: "127.16.67.67"
      port: 44863
    }
    bound_http_address {
      host: "127.16.67.67"
      port: 34365
    }
  }
}

02:04:05.999 [INFO - main] (KuduTestHarness.java:157) Creating a new Kudu client...
02:04:06.173 [DEBUG - main] (CompositeStrategy.java:59) Unable to load modern Subject methods, relying only on legacy methods
java.lang.NoSuchMethodException: javax.security.auth.Subject.current()
	at java.base/java.lang.Class.getDeclaredMethod(Class.java:2677)
	at org.apache.kudu.client.internals.ModernStrategy.<init>(ModernStrategy.java:51)
	at org.apache.kudu.client.internals.CompositeStrategy.<init>(CompositeStrategy.java:52)
	at org.apache.kudu.client.internals.CompositeStrategy.<clinit>(CompositeStrategy.java:41)
	at org.apache.kudu.client.internals.SecurityManagerCompatibility.get(SecurityManagerCompatibility.java:43)
	at org.apache.kudu.client.SecurityContext.setupSubject(SecurityContext.java:155)
	at org.apache.kudu.client.SecurityContext.<init>(SecurityContext.java:138)
	at org.apache.kudu.client.AsyncKuduClient.<init>(AsyncKuduClient.java:431)
	at org.apache.kudu.client.AsyncKuduClient$AsyncKuduClientBuilder.build(AsyncKuduClient.java:3109)
	at org.apache.kudu.test.KuduTestHarness.before(KuduTestHarness.java:161)
	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:50)
	at org.apache.kudu.test.junit.RetryRule$RetryStatement.doOneAttemptAndReport(RetryRule.java:184)
	at org.apache.kudu.test.junit.RetryRule$RetryStatement.evaluate(RetryRule.java:244)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:27)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
	at org.junit.runner.JUnitCore.runMain(JUnitCore.java:77)
	at org.junit.runner.JUnitCore.main(JUnitCore.java:36)
02:04:06.234 [DEBUG - main] (SecurityUtil.java:122) Could not login via JAAS. Using no credentials: Unable to obtain Principal Name for authentication 
02:04:06.474 [DEBUG - main] (Connection.java:748) [peer master-127.16.67.125:32941(127.16.67.125:32941)] connecting to peer
02:04:06.543 [DEBUG - main] (Connection.java:748) [peer master-127.16.67.124:33409(127.16.67.124:33409)] connecting to peer
02:04:06.544 [DEBUG - main] (Connection.java:748) [peer master-127.16.67.126:34295(127.16.67.126:34295)] connecting to peer
02:04:06.568 [DEBUG - kudu-nio-2] (Connection.java:757) [peer master-127.16.67.126:34295(127.16.67.126:34295)] Successfully connected to peer
02:04:06.571 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.16.67.125:32941(127.16.67.125:32941)] Successfully connected to peer
02:04:06.574 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.16.67.124:33409(127.16.67.124:33409)] Successfully connected to peer
02:04:06.673 [DEBUG - kudu-nio-2] (Negotiator.java:528) SASL mechanism PLAIN chosen for peer 127.16.67.126
02:04:06.673 [DEBUG - kudu-nio-1] (Negotiator.java:528) SASL mechanism PLAIN chosen for peer 127.16.67.124
02:04:06.685 [DEBUG - kudu-nio-0] (Negotiator.java:528) SASL mechanism PLAIN chosen for peer 127.16.67.125
02:04:06.917 [DEBUG - kudu-nio-0] (Negotiator.java:918) Authenticated connection [id: 0x384d6a0d, L:/127.0.0.1:60482 - R:/127.16.67.125:32941] using SASL/PLAIN
02:04:06.917 [DEBUG - kudu-nio-2] (Negotiator.java:918) Authenticated connection [id: 0x34f14c7b, L:/127.0.0.1:36942 - R:/127.16.67.126:34295] using SASL/PLAIN
02:04:06.917 [DEBUG - kudu-nio-1] (Negotiator.java:918) Authenticated connection [id: 0xdf6d5620, L:/127.0.0.1:43742 - R:/127.16.67.124:33409] using SASL/PLAIN
02:04:06.953 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:06.952842 17784 heartbeater.cc:499] Master 127.16.67.124:33409 was elected leader, sending a full tablet report...
02:04:06.958 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet Kudu Master for table 'Kudu Master' with partition [<start>, <end>)
02:04:06.960 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=CreateTable, tablet=null, attempt=1, TimeoutTracker(timeout=50000, elapsed=531), Traces: [0ms] refreshing cache from master, [29ms] Sub RPC ConnectToMaster: sending RPC to server master-127.16.67.125:32941, [109ms] Sub RPC ConnectToMaster: sending RPC to server master-127.16.67.124:33409, [110ms] Sub RPC ConnectToMaster: sending RPC to server master-127.16.67.126:34295, [500ms] Sub RPC ConnectToMaster: received response from server master-127.16.67.126:34295: OK, [500ms] Sub RPC ConnectToMaster: received response from server master-127.16.67.125:32941: OK, [500ms] Sub RPC ConnectToMaster: received response from server master-127.16.67.124:33409: OK, deferred=null) after lookup
02:04:07.013 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.011407 16949 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43742:
02:04:07.013 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) name: "org.apache.kudu.client.TestLeaderFailover-1786500228891"
02:04:07.014 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) schema {
02:04:07.014 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   columns {
02:04:07.014 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     name: "key"
02:04:07.014 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     type: INT32
02:04:07.014 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_key: true
02:04:07.014 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_nullable: false
02:04:07.014 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     cfile_block_size: 0
02:04:07.014 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     immutable: false
02:04:07.014 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_auto_incrementing: false
02:04:07.014 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
02:04:07.014 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   columns {
02:04:07.014 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     name: "column1_i"
02:04:07.014 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     type: INT32
02:04:07.014 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_key: false
02:04:07.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_nullable: false
02:04:07.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     cfile_block_size: 0
02:04:07.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     immutable: false
02:04:07.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_auto_incrementing: false
02:04:07.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
02:04:07.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   columns {
02:04:07.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     name: "column2_i"
02:04:07.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     type: INT32
02:04:07.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_key: false
02:04:07.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_nullable: false
02:04:07.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     cfile_block_size: 0
02:04:07.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     immutable: false
02:04:07.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_auto_incrementing: false
02:04:07.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
02:04:07.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   columns {
02:04:07.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     name: "column3_s"
02:04:07.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     type: STRING
02:04:07.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_key: false
02:04:07.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_nullable: true
02:04:07.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     encoding: DICT_ENCODING
02:04:07.016 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     compression: LZ4
02:04:07.016 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     cfile_block_size: 4096
02:04:07.016 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     immutable: false
02:04:07.016 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_auto_incrementing: false
02:04:07.016 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
02:04:07.016 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   columns {
02:04:07.016 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     name: "column4_b"
02:04:07.016 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     type: BOOL
02:04:07.016 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_key: false
02:04:07.016 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_nullable: false
02:04:07.016 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     cfile_block_size: 0
02:04:07.016 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     immutable: false
02:04:07.016 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_auto_incrementing: false
02:04:07.016 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
02:04:07.016 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) }
02:04:07.016 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) partition_schema {
02:04:07.016 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   range_schema {
02:04:07.016 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     columns {
02:04:07.017 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)       name: "key"
02:04:07.017 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     }
02:04:07.017 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
02:04:07.017 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) }
02:04:07.017 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:07.017303 16949 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table org.apache.kudu.client.TestLeaderFailover-1786500228891 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
02:04:07.107 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:894) Opened table 10913ac85c0e481cabbf77bf00667b76
02:04:07.162 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.161048 17680 tablet_service.cc:1511] Processing CreateTablet for tablet 66d46d031bb04cfebcc1389692a56af1 (DEFAULT_TABLE table=org.apache.kudu.client.TestLeaderFailover-1786500228891 [id=10913ac85c0e481cabbf77bf00667b76]), partition=RANGE (key) PARTITION UNBOUNDED
02:04:07.163 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.162914 17680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66d46d031bb04cfebcc1389692a56af1. 1 dirs total, 0 dirs full, 0 dirs failed
02:04:07.170 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.169610 17452 tablet_service.cc:1511] Processing CreateTablet for tablet 66d46d031bb04cfebcc1389692a56af1 (DEFAULT_TABLE table=org.apache.kudu.client.TestLeaderFailover-1786500228891 [id=10913ac85c0e481cabbf77bf00667b76]), partition=RANGE (key) PARTITION UNBOUNDED
02:04:07.171 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.171499 17452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66d46d031bb04cfebcc1389692a56af1. 1 dirs total, 0 dirs full, 0 dirs failed
02:04:07.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.176084 17203 tablet_service.cc:1511] Processing CreateTablet for tablet 66d46d031bb04cfebcc1389692a56af1 (DEFAULT_TABLE table=org.apache.kudu.client.TestLeaderFailover-1786500228891 [id=10913ac85c0e481cabbf77bf00667b76]), partition=RANGE (key) PARTITION UNBOUNDED
02:04:07.178 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.178220 17203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66d46d031bb04cfebcc1389692a56af1. 1 dirs total, 0 dirs full, 0 dirs failed
02:04:07.199 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.198717 17844 tablet_bootstrap.cc:492] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce: Bootstrap starting.
02:04:07.200 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.200194 17845 tablet_bootstrap.cc:492] T 66d46d031bb04cfebcc1389692a56af1 P f02b99900ae14094a50f21ad3593d551: Bootstrap starting.
02:04:07.208 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.208509 17844 tablet_bootstrap.cc:654] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce: Neither blocks nor log segments found. Creating new log.
02:04:07.209 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.209519 17845 tablet_bootstrap.cc:654] T 66d46d031bb04cfebcc1389692a56af1 P f02b99900ae14094a50f21ad3593d551: Neither blocks nor log segments found. Creating new log.
02:04:07.213 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.212397 17844 log.cc:826] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce: Log is configured to *not* fsync() on all Append() calls
02:04:07.215 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.215154 17845 log.cc:826] T 66d46d031bb04cfebcc1389692a56af1 P f02b99900ae14094a50f21ad3593d551: Log is configured to *not* fsync() on all Append() calls
02:04:07.223 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.222592 17846 tablet_bootstrap.cc:492] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1: Bootstrap starting.
02:04:07.229 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.228657 17844 tablet_bootstrap.cc:492] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce: No bootstrap required, opened a new log
02:04:07.230 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.230072 17844 ts_tablet_manager.cc:1403] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce: Time spent bootstrapping tablet: real 0.036s	user 0.021s	sys 0.006s
02:04:07.237 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.236699 17846 tablet_bootstrap.cc:654] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1: Neither blocks nor log segments found. Creating new log.
02:04:07.240 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.239832 17846 log.cc:826] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1: Log is configured to *not* fsync() on all Append() calls
02:04:07.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.251771 17845 tablet_bootstrap.cc:492] T 66d46d031bb04cfebcc1389692a56af1 P f02b99900ae14094a50f21ad3593d551: No bootstrap required, opened a new log
02:04:07.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.252341 17846 tablet_bootstrap.cc:492] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1: No bootstrap required, opened a new log
02:04:07.253 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.252767 17845 ts_tablet_manager.cc:1403] T 66d46d031bb04cfebcc1389692a56af1 P f02b99900ae14094a50f21ad3593d551: Time spent bootstrapping tablet: real 0.053s	user 0.017s	sys 0.014s
02:04:07.253 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.252921 17846 ts_tablet_manager.cc:1403] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1: Time spent bootstrapping tablet: real 0.031s	user 0.007s	sys 0.018s
02:04:07.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.286262 17845 raft_consensus.cc:359] T 66d46d031bb04cfebcc1389692a56af1 P f02b99900ae14094a50f21ad3593d551 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91b849f0528142ed8048da0214d55cce" member_type: VOTER last_known_addr { host: "127.16.67.67" port: 44863 } } peers { permanent_uuid: "f02b99900ae14094a50f21ad3593d551" member_type: VOTER last_known_addr { host: "127.16.67.66" port: 46107 } } peers { permanent_uuid: "9e0ec91b729b44319839f751e036d8c1" member_type: VOTER last_known_addr { host: "127.16.67.65" port: 36583 } }
02:04:07.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.287539 17845 raft_consensus.cc:385] T 66d46d031bb04cfebcc1389692a56af1 P f02b99900ae14094a50f21ad3593d551 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
02:04:07.288 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.288100 17845 raft_consensus.cc:740] T 66d46d031bb04cfebcc1389692a56af1 P f02b99900ae14094a50f21ad3593d551 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f02b99900ae14094a50f21ad3593d551, State: Initialized, Role: FOLLOWER
02:04:07.291 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.290226 17845 consensus_queue.cc:260] T 66d46d031bb04cfebcc1389692a56af1 P f02b99900ae14094a50f21ad3593d551 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91b849f0528142ed8048da0214d55cce" member_type: VOTER last_known_addr { host: "127.16.67.67" port: 44863 } } peers { permanent_uuid: "f02b99900ae14094a50f21ad3593d551" member_type: VOTER last_known_addr { host: "127.16.67.66" port: 46107 } } peers { permanent_uuid: "9e0ec91b729b44319839f751e036d8c1" member_type: VOTER last_known_addr { host: "127.16.67.65" port: 36583 } }
02:04:07.291 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.290278 17846 raft_consensus.cc:359] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91b849f0528142ed8048da0214d55cce" member_type: VOTER last_known_addr { host: "127.16.67.67" port: 44863 } } peers { permanent_uuid: "f02b99900ae14094a50f21ad3593d551" member_type: VOTER last_known_addr { host: "127.16.67.66" port: 46107 } } peers { permanent_uuid: "9e0ec91b729b44319839f751e036d8c1" member_type: VOTER last_known_addr { host: "127.16.67.65" port: 36583 } }
02:04:07.291 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.291250 17846 raft_consensus.cc:385] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
02:04:07.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.291590 17846 raft_consensus.cc:740] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0ec91b729b44319839f751e036d8c1, State: Initialized, Role: FOLLOWER
02:04:07.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.292793 17846 consensus_queue.cc:260] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91b849f0528142ed8048da0214d55cce" member_type: VOTER last_known_addr { host: "127.16.67.67" port: 44863 } } peers { permanent_uuid: "f02b99900ae14094a50f21ad3593d551" member_type: VOTER last_known_addr { host: "127.16.67.66" port: 46107 } } peers { permanent_uuid: "9e0ec91b729b44319839f751e036d8c1" member_type: VOTER last_known_addr { host: "127.16.67.65" port: 36583 } }
02:04:07.318 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.317956 17851 raft_consensus.cc:493] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
02:04:07.318 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.318542 17845 ts_tablet_manager.cc:1434] T 66d46d031bb04cfebcc1389692a56af1 P f02b99900ae14094a50f21ad3593d551: Time spent starting tablet: real 0.065s	user 0.026s	sys 0.018s
02:04:07.320 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.311642 17844 raft_consensus.cc:359] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91b849f0528142ed8048da0214d55cce" member_type: VOTER last_known_addr { host: "127.16.67.67" port: 44863 } } peers { permanent_uuid: "f02b99900ae14094a50f21ad3593d551" member_type: VOTER last_known_addr { host: "127.16.67.66" port: 46107 } } peers { permanent_uuid: "9e0ec91b729b44319839f751e036d8c1" member_type: VOTER last_known_addr { host: "127.16.67.65" port: 36583 } }
02:04:07.320 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.320230 17844 raft_consensus.cc:385] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
02:04:07.320 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.320568 17844 raft_consensus.cc:740] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b849f0528142ed8048da0214d55cce, State: Initialized, Role: FOLLOWER
02:04:07.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.324361 17844 consensus_queue.cc:260] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91b849f0528142ed8048da0214d55cce" member_type: VOTER last_known_addr { host: "127.16.67.67" port: 44863 } } peers { permanent_uuid: "f02b99900ae14094a50f21ad3593d551" member_type: VOTER last_known_addr { host: "127.16.67.66" port: 46107 } } peers { permanent_uuid: "9e0ec91b729b44319839f751e036d8c1" member_type: VOTER last_known_addr { host: "127.16.67.65" port: 36583 } }
02:04:07.329 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.329149 17846 ts_tablet_manager.cc:1434] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1: Time spent starting tablet: real 0.076s	user 0.036s	sys 0.014s
02:04:07.333 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.318735 17851 raft_consensus.cc:515] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91b849f0528142ed8048da0214d55cce" member_type: VOTER last_known_addr { host: "127.16.67.67" port: 44863 } } peers { permanent_uuid: "f02b99900ae14094a50f21ad3593d551" member_type: VOTER last_known_addr { host: "127.16.67.66" port: 46107 } } peers { permanent_uuid: "9e0ec91b729b44319839f751e036d8c1" member_type: VOTER last_known_addr { host: "127.16.67.65" port: 36583 } }
02:04:07.342 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.341517 17851 leader_election.cc:290] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91b849f0528142ed8048da0214d55cce (127.16.67.67:44863), f02b99900ae14094a50f21ad3593d551 (127.16.67.66:46107)
02:04:07.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.381371 17844 ts_tablet_manager.cc:1434] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce: Time spent starting tablet: real 0.151s	user 0.043s	sys 0.014s
02:04:07.385 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:07.384429 17318 tablet.cc:2406] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
02:04:07.397 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:07.396341 17791 tablet.cc:2406] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce: Can't schedule compaction. Clean time has not been advanced past its initial value.
02:04:07.403 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.401373 17479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66d46d031bb04cfebcc1389692a56af1" candidate_uuid: "9e0ec91b729b44319839f751e036d8c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f02b99900ae14094a50f21ad3593d551" is_pre_election: true
02:04:07.403 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.402453 17479 raft_consensus.cc:2468] T 66d46d031bb04cfebcc1389692a56af1 P f02b99900ae14094a50f21ad3593d551 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0ec91b729b44319839f751e036d8c1 in term 0.
02:04:07.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.402387 17715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66d46d031bb04cfebcc1389692a56af1" candidate_uuid: "9e0ec91b729b44319839f751e036d8c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b849f0528142ed8048da0214d55cce" is_pre_election: true
02:04:07.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.403437 17715 raft_consensus.cc:2468] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0ec91b729b44319839f751e036d8c1 in term 0.
02:04:07.406 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.404735 17127 leader_election.cc:304] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e0ec91b729b44319839f751e036d8c1, f02b99900ae14094a50f21ad3593d551; no voters: 
02:04:07.407 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.406947 17851 raft_consensus.cc:2804] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
02:04:07.408 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.407600 17851 raft_consensus.cc:493] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
02:04:07.408 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.408106 17851 raft_consensus.cc:3060] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1 [term 0 FOLLOWER]: Advancing to term 1
02:04:07.416 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.415738 17851 raft_consensus.cc:515] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91b849f0528142ed8048da0214d55cce" member_type: VOTER last_known_addr { host: "127.16.67.67" port: 44863 } } peers { permanent_uuid: "f02b99900ae14094a50f21ad3593d551" member_type: VOTER last_known_addr { host: "127.16.67.66" port: 46107 } } peers { permanent_uuid: "9e0ec91b729b44319839f751e036d8c1" member_type: VOTER last_known_addr { host: "127.16.67.65" port: 36583 } }
02:04:07.418 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.417776 17851 leader_election.cc:290] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1 [CANDIDATE]: Term 1 election: Requested vote from peers 91b849f0528142ed8048da0214d55cce (127.16.67.67:44863), f02b99900ae14094a50f21ad3593d551 (127.16.67.66:46107)
02:04:07.419 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.418761 17715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66d46d031bb04cfebcc1389692a56af1" candidate_uuid: "9e0ec91b729b44319839f751e036d8c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b849f0528142ed8048da0214d55cce"
02:04:07.419 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.418955 17479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66d46d031bb04cfebcc1389692a56af1" candidate_uuid: "9e0ec91b729b44319839f751e036d8c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f02b99900ae14094a50f21ad3593d551"
02:04:07.419 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.419441 17715 raft_consensus.cc:3060] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce [term 0 FOLLOWER]: Advancing to term 1
02:04:07.420 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.419867 17479 raft_consensus.cc:3060] T 66d46d031bb04cfebcc1389692a56af1 P f02b99900ae14094a50f21ad3593d551 [term 0 FOLLOWER]: Advancing to term 1
02:04:07.427 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.426467 17715 raft_consensus.cc:2468] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0ec91b729b44319839f751e036d8c1 in term 1.
02:04:07.427 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.427445 17479 raft_consensus.cc:2468] T 66d46d031bb04cfebcc1389692a56af1 P f02b99900ae14094a50f21ad3593d551 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0ec91b729b44319839f751e036d8c1 in term 1.
02:04:07.428 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.427902 17127 leader_election.cc:304] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b849f0528142ed8048da0214d55cce, 9e0ec91b729b44319839f751e036d8c1; no voters: 
02:04:07.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.428972 17851 raft_consensus.cc:2804] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1 [term 1 FOLLOWER]: Leader election won for term 1
02:04:07.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.429479 17851 raft_consensus.cc:697] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1 [term 1 LEADER]: Becoming Leader. State: Replica: 9e0ec91b729b44319839f751e036d8c1, State: Running, Role: LEADER
02:04:07.431 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.430637 17851 consensus_queue.cc:237] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91b849f0528142ed8048da0214d55cce" member_type: VOTER last_known_addr { host: "127.16.67.67" port: 44863 } } peers { permanent_uuid: "f02b99900ae14094a50f21ad3593d551" member_type: VOTER last_known_addr { host: "127.16.67.66" port: 46107 } } peers { permanent_uuid: "9e0ec91b729b44319839f751e036d8c1" member_type: VOTER last_known_addr { host: "127.16.67.65" port: 36583 } }
02:04:07.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.443194 16949 catalog_manager.cc:5719] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e0ec91b729b44319839f751e036d8c1 (127.16.67.65). New cstate: current_term: 1 leader_uuid: "9e0ec91b729b44319839f751e036d8c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91b849f0528142ed8048da0214d55cce" member_type: VOTER last_known_addr { host: "127.16.67.67" port: 44863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f02b99900ae14094a50f21ad3593d551" member_type: VOTER last_known_addr { host: "127.16.67.66" port: 46107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0ec91b729b44319839f751e036d8c1" member_type: VOTER last_known_addr { host: "127.16.67.65" port: 36583 } health_report { overall_health: HEALTHY } } }
02:04:07.486 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:07.485437 17572 tablet.cc:2406] T 66d46d031bb04cfebcc1389692a56af1 P f02b99900ae14094a50f21ad3593d551: Can't schedule compaction. Clean time has not been advanced past its initial value.
02:04:07.600 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:894) Opened table 10913ac85c0e481cabbf77bf00667b76
02:04:07.623 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 66d46d031bb04cfebcc1389692a56af1 for table 'org.apache.kudu.client.TestLeaderFailover-1786500228891' with partition [<start>, <end>)
02:04:07.630 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299993, tablet=66d46d031bb04cfebcc1389692a56af1@[91b849f0528142ed8048da0214d55cce(127.16.67.67:44863),9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583)[L],f02b99900ae14094a50f21ad3593d551(127.16.67.66:46107)]}]
02:04:07.632 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=null, attempt=1, sequence_id=-1, TimeoutTracker(timeout=30000, elapsed=21), Traces: [0ms] refreshing cache from master, [1ms] Sub RPC GetTableLocations: sending RPC to server master-127.16.67.124:33409, [11ms] Sub RPC GetTableLocations: received response from server master-127.16.67.124:33409: OK, deferred=null) after lookup
02:04:07.655 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583)] connecting to peer
02:04:07.658 [DEBUG - kudu-nio-3] (Connection.java:757) [peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583)] Successfully connected to peer
02:04:07.724 [DEBUG - kudu-nio-3] (Negotiator.java:918) Authenticated connection [id: 0x8c2c534e, L:/127.0.0.1:49312 - R:/127.16.67.65:36583] using TOKEN/null
02:04:07.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.739421 17479 raft_consensus.cc:1275] T 66d46d031bb04cfebcc1389692a56af1 P f02b99900ae14094a50f21ad3593d551 [term 1 FOLLOWER]: Refusing update from remote peer 9e0ec91b729b44319839f751e036d8c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
02:04:07.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.740809 17715 raft_consensus.cc:1275] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce [term 1 FOLLOWER]: Refusing update from remote peer 9e0ec91b729b44319839f751e036d8c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
02:04:07.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.741489 17851 consensus_queue.cc:1048] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f02b99900ae14094a50f21ad3593d551" member_type: VOTER last_known_addr { host: "127.16.67.66" port: 46107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
02:04:07.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.742620 17855 consensus_queue.cc:1048] T 66d46d031bb04cfebcc1389692a56af1 P 9e0ec91b729b44319839f751e036d8c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b849f0528142ed8048da0214d55cce" member_type: VOTER last_known_addr { host: "127.16.67.67" port: 44863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
02:04:07.781 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.781093 17870 mvcc.cc:204] Tried to move back new op lower bound from 7317505014721998848 to 7317505013485387776. Current Snapshot: MvccSnapshot[applied={T|T < 7317505014721998848}]
02:04:07.973 [DEBUG - kudu-nio-3] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 3 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestLeaderFailover-1786500228891, tablet=66d46d031bb04cfebcc1389692a56af1, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
02:04:07.975 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.16.67.65:36583
02:04:07.977 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

02:04:07.978 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:07.977825 16653 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskzRoOVp/build/tsan/bin/kudu with pid 17083
02:04:08.035 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
02:04:08.036 [DEBUG - kudu-nio-3] (Connection.java:239) [peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583)] handling channelInactive
02:04:08.037 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583)] cleaning up while in state READY due to: connection closed
02:04:08.037 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583) for tablet 66d46d031bb04cfebcc1389692a56af1: connection closed
02:04:08.056 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 66d46d031bb04cfebcc1389692a56af1 for table 'org.apache.kudu.client.TestLeaderFailover-1786500228891' with partition [<start>, <end>)
02:04:08.056 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=66d46d031bb04cfebcc1389692a56af1@[91b849f0528142ed8048da0214d55cce(127.16.67.67:44863),9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583)[L],f02b99900ae14094a50f21ad3593d551(127.16.67.66:46107)]}]
02:04:08.057 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=66d46d031bb04cfebcc1389692a56af1, attempt=2, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=20), Traces: [0ms] sending RPC to server 9e0ec91b729b44319839f751e036d8c1, [1ms] received response from server 9e0ec91b729b44319839f751e036d8c1: Network error: connection closed, [1ms] delaying RPC due to: Network error: connection closed, [14ms] refreshing cache from master, [15ms] Sub RPC GetTableLocations: sending RPC to server master-127.16.67.124:33409, [19ms] Sub RPC GetTableLocations: received response from server master-127.16.67.124:33409: OK, deferred=Deferred@241725940(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$227/0x00007f37ec338000@6a954c0b -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2c8a739a -> wakeup thread Time-limited test)) after lookup
02:04:08.058 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583)] connecting to peer
02:04:08.061 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583
02:04:08.061 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583)] cleaning up while in state CONNECTING due to: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583
02:04:08.061 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583) for tablet 66d46d031bb04cfebcc1389692a56af1: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583
02:04:08.076 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 66d46d031bb04cfebcc1389692a56af1 for table 'org.apache.kudu.client.TestLeaderFailover-1786500228891' with partition [<start>, <end>)
02:04:08.077 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=66d46d031bb04cfebcc1389692a56af1@[91b849f0528142ed8048da0214d55cce(127.16.67.67:44863),9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583)[L],f02b99900ae14094a50f21ad3593d551(127.16.67.66:46107)]}]
02:04:08.077 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=66d46d031bb04cfebcc1389692a56af1, attempt=4, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=41), Traces: [0ms] sending RPC to server 9e0ec91b729b44319839f751e036d8c1, [1ms] received response from server 9e0ec91b729b44319839f751e036d8c1: Network error: connection closed, [1ms] delaying RPC due to: Network error: connection closed, [14ms] refreshing cache from master, [15ms] Sub RPC GetTableLocations: sending RPC to server master-127.16.67.124:33409, [19ms] Sub RPC GetTableLocations: received response from server master-127.16.67.124:33409: OK, [21ms] sending RPC to server 9e0ec91b729b44319839f751e036d8c1, [25ms] received response from server 9e0ec91b729b44319839f751e036d8c1: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [25ms] delaying RPC due to: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [34ms] refreshing cache from master, [34ms] Sub RPC GetTableLocations: sending RPC to server master-127.16.67.124:33409, [40ms] Sub RPC GetTableLocations: received response from server master-127.16.67.124:33409: OK, deferred=Deferred@241725940(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$227/0x00007f37ec338000@6a954c0b -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1548372956 after retry RPC@1437078598), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2c8a739a -> wakeup thread Time-limited test -> (continuation of Deferred@1548372956 after retry RPC@1437078598))) after lookup
02:04:08.078 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583)] connecting to peer
02:04:08.081 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583
02:04:08.081 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583)] cleaning up while in state CONNECTING due to: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583
02:04:08.081 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583) for tablet 66d46d031bb04cfebcc1389692a56af1: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583
02:04:08.116 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 66d46d031bb04cfebcc1389692a56af1 for table 'org.apache.kudu.client.TestLeaderFailover-1786500228891' with partition [<start>, <end>)
02:04:08.116 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=66d46d031bb04cfebcc1389692a56af1@[91b849f0528142ed8048da0214d55cce(127.16.67.67:44863),9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583)[L],f02b99900ae14094a50f21ad3593d551(127.16.67.66:46107)]}]
02:04:08.117 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=66d46d031bb04cfebcc1389692a56af1, attempt=6, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=80), Traces: [0ms] sending RPC to server 9e0ec91b729b44319839f751e036d8c1, [1ms] received response from server 9e0ec91b729b44319839f751e036d8c1: Network error: connection closed, [1ms] delaying RPC due to: Network error: connection closed, [14ms] refreshing cache from master, [15ms] Sub RPC GetTableLocations: sending RPC to server master-127.16.67.124:33409, [19ms] Sub RPC GetTableLocations: received response from server master-127.16.67.124:33409: OK, [21ms] sending RPC to server 9e0ec91b729b44319839f751e036d8c1, [25ms] received response from server 9e0ec91b729b44319839f751e036d8c1: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [25ms] delaying RPC due to: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [34ms] refreshing cache from master, [34ms] Sub RPC GetTableLocations: sending RPC to server master-127.16.67.124:33409, [40ms] Sub RPC GetTableLocations: received response from server master-127.16.67.124:33409: OK, [42ms] sending RPC to server 9e0ec91b729b44319839f751e036d8c1, [45ms] received response from server 9e0ec91b729b44319839f751e036d8c1: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [45ms] delaying RPC due to: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [74ms] refreshing cache from master, [74ms] Sub RPC GetTableLocations: sending RPC to server master-127.16.67.124:33409, [79ms] Sub RPC GetTableLocations: received response from server master-127.16.67.124:33409: OK, deferred=Deferred@241725940(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$227/0x00007f37ec338000@6a954c0b -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1548372956 after retry RPC@1437078598) -> (continuation of Deferred@1626623337 after retry RPC@886926193), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2c8a739a -> wakeup thread Time-limited test -> (continuation of Deferred@1548372956 after retry RPC@1437078598) -> (continuation of Deferred@1626623337 after retry RPC@886926193))) after lookup
02:04:08.118 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583)] connecting to peer
02:04:08.120 [INFO - kudu-nio-6] (Connection.java:429) Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583
02:04:08.120 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583)] cleaning up while in state CONNECTING due to: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583
02:04:08.120 [INFO - kudu-nio-6] (AsyncKuduClient.java:2355) Invalidating location 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583) for tablet 66d46d031bb04cfebcc1389692a56af1: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583
02:04:08.156 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 66d46d031bb04cfebcc1389692a56af1 for table 'org.apache.kudu.client.TestLeaderFailover-1786500228891' with partition [<start>, <end>)
02:04:08.157 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=66d46d031bb04cfebcc1389692a56af1@[91b849f0528142ed8048da0214d55cce(127.16.67.67:44863),9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583)[L],f02b99900ae14094a50f21ad3593d551(127.16.67.66:46107)]}]
02:04:08.157 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=66d46d031bb04cfebcc1389692a56af1, attempt=8, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=121), Traces: [0ms] sending RPC to server 9e0ec91b729b44319839f751e036d8c1, [1ms] received response from server 9e0ec91b729b44319839f751e036d8c1: Network error: connection closed, [1ms] delaying RPC due to: Network error: connection closed, [14ms] refreshing cache from master, [15ms] Sub RPC GetTableLocations: sending RPC to server master-127.16.67.124:33409, [19ms] Sub RPC GetTableLocations: received response from server master-127.16.67.124:33409: OK, [21ms] sending RPC to server 9e0ec91b729b44319839f751e036d8c1, [25ms] received response from server 9e0ec91b729b44319839f751e036d8c1: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [25ms] delaying RPC due to: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [34ms] refreshing cache from master, [34ms] Sub RPC GetTableLocations: sending RPC to server master-127.16.67.124:33409, [40ms] Sub RPC GetTableLocations: received response from server master-127.16.67.124:33409: OK, [42ms] sending RPC to server 9e0ec91b729b44319839f751e036d8c1, [45ms] received response from server 9e0ec91b729b44319839f751e036d8c1: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [45ms] delaying RPC due to: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [74ms] refreshing cache from master, [74ms] Sub RPC GetTableLocations: sending RPC to server master-127.16.67.124:33409, [79ms] Sub RPC GetTableLocations: received response from server master-127.16.67.124:33409: OK, [81ms] sending RPC to server 9e0ec91b729b44319839f751e036d8c1, [84ms] received response from server 9e0ec91b729b44319839f751e036d8c1: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [84ms] delaying RPC due to: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [114ms] refreshing cache from master, [114ms] Sub RPC GetTableLocations: sending RPC to server master-127.16.67.124:33409, [120ms] Sub RPC GetTableLocations: received response from server master-127.16.67.124:33409: OK, deferred=Deferred@241725940(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$227/0x00007f37ec338000@6a954c0b -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1548372956 after retry RPC@1437078598) -> (continuation of Deferred@1626623337 after retry RPC@886926193) -> (continuation of Deferred@1914200216 after retry RPC@300694895), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2c8a739a -> wakeup thread Time-limited test -> (continuation of Deferred@1548372956 after retry RPC@1437078598) -> (continuation of Deferred@1626623337 after retry RPC@886926193) -> (continuation of Deferred@1914200216 after retry RPC@300694895))) after lookup
02:04:08.158 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583)] connecting to peer
02:04:08.159 [INFO - kudu-nio-7] (Connection.java:429) Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583
02:04:08.160 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583)] cleaning up while in state CONNECTING due to: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583
02:04:08.160 [INFO - kudu-nio-7] (AsyncKuduClient.java:2355) Invalidating location 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583) for tablet 66d46d031bb04cfebcc1389692a56af1: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583
02:04:08.257 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 66d46d031bb04cfebcc1389692a56af1 for table 'org.apache.kudu.client.TestLeaderFailover-1786500228891' with partition [<start>, <end>)
02:04:08.257 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=66d46d031bb04cfebcc1389692a56af1@[91b849f0528142ed8048da0214d55cce(127.16.67.67:44863),9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583)[L],f02b99900ae14094a50f21ad3593d551(127.16.67.66:46107)]}]
02:04:08.258 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=66d46d031bb04cfebcc1389692a56af1, attempt=10, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=221), Traces: [0ms] sending RPC to server 9e0ec91b729b44319839f751e036d8c1, [1ms] received response from server 9e0ec91b729b44319839f751e036d8c1: Network error: connection closed, [1ms] delaying RPC due to: Network error: connection closed, [14ms] refreshing cache from master, [15ms] Sub RPC GetTableLocations: sending RPC to server master-127.16.67.124:33409, [19ms] Sub RPC GetTableLocations: received response from server master-127.16.67.124:33409: OK, [21ms] sending RPC to server 9e0ec91b729b44319839f751e036d8c1, [25ms] received response from server 9e0ec91b729b44319839f751e036d8c1: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [25ms] delaying RPC due to: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [34ms] refreshing cache from master, [34ms] Sub RPC GetTableLocations: sending RPC to server master-127.16.67.124:33409, [40ms] Sub RPC GetTableLocations: received response from server master-127.16.67.124:33409: OK, [42ms] sending RPC to server 9e0ec91b729b44319839f751e036d8c1, [45ms] received response from server 9e0ec91b729b44319839f751e036d8c1: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [45ms] delaying RPC due to: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [74ms] refreshing cache from master, [74ms] Sub RPC GetTableLocations: sending RPC to server master-127.16.67.124:33409, [79ms] Sub RPC GetTableLocations: received response from server master-127.16.67.124:33409: OK, [81ms] sending RPC to server 9e0ec91b729b44319839f751e036d8c1, [84ms] received response from server 9e0ec91b729b44319839f751e036d8c1: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [84ms] delaying RPC due to: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [114ms] refreshing cache from master, [114ms] Sub RPC GetTableLocations: sending RPC to server master-127.16.67.124:33409, [120ms] Sub RPC GetTableLocations: received response from server master-127.16.67.124:33409: OK, [121ms] sending RPC to server 9e0ec91b729b44319839f751e036d8c1, [124ms] received response from server 9e0ec91b729b44319839f751e036d8c1: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [124ms] delaying RPC due to: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [215ms] refreshing cache from master, [215ms] Sub RPC GetTableLocations: sending RPC to server master-127.16.67.124:33409, [220ms] Sub RPC GetTableLocations: received response from server master-127.16.67.124:33409: OK, deferred=Deferred@241725940(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$227/0x00007f37ec338000@6a954c0b -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1548372956 after retry RPC@1437078598) -> (continuation of Deferred@1626623337 after retry RPC@886926193) -> (continuation of Deferred@1914200216 after retry RPC@300694895) -> (continuation of Deferred@2130266701 after retry RPC@1939786471), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2c8a739a -> wakeup thread Time-limited test -> (continuation of Deferred@1548372956 after retry RPC@1437078598) -> (continuation of Deferred@1626623337 after retry RPC@886926193) -> (continuation of Deferred@1914200216 after retry RPC@300694895) -> (continuation of Deferred@2130266701 after retry RPC@1939786471))) after lookup
02:04:08.259 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583)] connecting to peer
02:04:08.260 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583
02:04:08.260 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583)] cleaning up while in state CONNECTING due to: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583
02:04:08.260 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583) for tablet 66d46d031bb04cfebcc1389692a56af1: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583
02:04:09.412 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.411700 17894 raft_consensus.cc:493] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0ec91b729b44319839f751e036d8c1)
02:04:09.412 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.412221 17894 raft_consensus.cc:515] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91b849f0528142ed8048da0214d55cce" member_type: VOTER last_known_addr { host: "127.16.67.67" port: 44863 } } peers { permanent_uuid: "f02b99900ae14094a50f21ad3593d551" member_type: VOTER last_known_addr { host: "127.16.67.66" port: 46107 } } peers { permanent_uuid: "9e0ec91b729b44319839f751e036d8c1" member_type: VOTER last_known_addr { host: "127.16.67.65" port: 36583 } }
02:04:09.414 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.413810 17895 raft_consensus.cc:493] T 66d46d031bb04cfebcc1389692a56af1 P f02b99900ae14094a50f21ad3593d551 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0ec91b729b44319839f751e036d8c1)
02:04:09.414 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.414248 17895 raft_consensus.cc:515] T 66d46d031bb04cfebcc1389692a56af1 P f02b99900ae14094a50f21ad3593d551 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91b849f0528142ed8048da0214d55cce" member_type: VOTER last_known_addr { host: "127.16.67.67" port: 44863 } } peers { permanent_uuid: "f02b99900ae14094a50f21ad3593d551" member_type: VOTER last_known_addr { host: "127.16.67.66" port: 46107 } } peers { permanent_uuid: "9e0ec91b729b44319839f751e036d8c1" member_type: VOTER last_known_addr { host: "127.16.67.65" port: 36583 } }
02:04:09.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.415367 17894 leader_election.cc:290] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f02b99900ae14094a50f21ad3593d551 (127.16.67.66:46107), 9e0ec91b729b44319839f751e036d8c1 (127.16.67.65:36583)
02:04:09.426 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:09.426102 17610 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.67.65:36583: connect: Connection refused (error 111)
02:04:09.428 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.428321 17895 leader_election.cc:290] T 66d46d031bb04cfebcc1389692a56af1 P f02b99900ae14094a50f21ad3593d551 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91b849f0528142ed8048da0214d55cce (127.16.67.67:44863), 9e0ec91b729b44319839f751e036d8c1 (127.16.67.65:36583)
02:04:09.434 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:09.433562 17373 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.67.65:36583: connect: Connection refused (error 111)
02:04:09.435 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:09.435377 17610 leader_election.cc:336] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9e0ec91b729b44319839f751e036d8c1 (127.16.67.65:36583): Network error: Client connection negotiation failed: client connection to 127.16.67.65:36583: connect: Connection refused (error 111)
02:04:09.439 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.439108 17479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66d46d031bb04cfebcc1389692a56af1" candidate_uuid: "91b849f0528142ed8048da0214d55cce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f02b99900ae14094a50f21ad3593d551" is_pre_election: true
02:04:09.440 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.439751 17479 raft_consensus.cc:2468] T 66d46d031bb04cfebcc1389692a56af1 P f02b99900ae14094a50f21ad3593d551 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91b849f0528142ed8048da0214d55cce in term 1.
02:04:09.440 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:09.440408 17373 leader_election.cc:336] T 66d46d031bb04cfebcc1389692a56af1 P f02b99900ae14094a50f21ad3593d551 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9e0ec91b729b44319839f751e036d8c1 (127.16.67.65:36583): Network error: Client connection negotiation failed: client connection to 127.16.67.65:36583: connect: Connection refused (error 111)
02:04:09.441 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.441447 17602 leader_election.cc:304] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91b849f0528142ed8048da0214d55cce, f02b99900ae14094a50f21ad3593d551; no voters: 9e0ec91b729b44319839f751e036d8c1
02:04:09.442 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.442258 17894 raft_consensus.cc:2804] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce [term 1 FOLLOWER]: Leader pre-election won for term 2
02:04:09.443 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.442636 17894 raft_consensus.cc:493] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e0ec91b729b44319839f751e036d8c1)
02:04:09.443 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.443004 17894 raft_consensus.cc:3060] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce [term 1 FOLLOWER]: Advancing to term 2
02:04:09.447 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.446882 17715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66d46d031bb04cfebcc1389692a56af1" candidate_uuid: "f02b99900ae14094a50f21ad3593d551" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91b849f0528142ed8048da0214d55cce" is_pre_election: true
02:04:09.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.448768 17894 raft_consensus.cc:515] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91b849f0528142ed8048da0214d55cce" member_type: VOTER last_known_addr { host: "127.16.67.67" port: 44863 } } peers { permanent_uuid: "f02b99900ae14094a50f21ad3593d551" member_type: VOTER last_known_addr { host: "127.16.67.66" port: 46107 } } peers { permanent_uuid: "9e0ec91b729b44319839f751e036d8c1" member_type: VOTER last_known_addr { host: "127.16.67.65" port: 36583 } }
02:04:09.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.449842 17715 raft_consensus.cc:2393] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f02b99900ae14094a50f21ad3593d551 in current term 2: Already voted for candidate 91b849f0528142ed8048da0214d55cce in this term.
02:04:09.451 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.450809 17894 leader_election.cc:290] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce [CANDIDATE]: Term 2 election: Requested vote from peers f02b99900ae14094a50f21ad3593d551 (127.16.67.66:46107), 9e0ec91b729b44319839f751e036d8c1 (127.16.67.65:36583)
02:04:09.452 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.451546 17479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66d46d031bb04cfebcc1389692a56af1" candidate_uuid: "91b849f0528142ed8048da0214d55cce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f02b99900ae14094a50f21ad3593d551"
02:04:09.452 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.452157 17479 raft_consensus.cc:3060] T 66d46d031bb04cfebcc1389692a56af1 P f02b99900ae14094a50f21ad3593d551 [term 1 FOLLOWER]: Advancing to term 2
02:04:09.454 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.453732 17369 leader_election.cc:304] T 66d46d031bb04cfebcc1389692a56af1 P f02b99900ae14094a50f21ad3593d551 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f02b99900ae14094a50f21ad3593d551; no voters: 91b849f0528142ed8048da0214d55cce, 9e0ec91b729b44319839f751e036d8c1
02:04:09.455 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:09.455045 17610 leader_election.cc:336] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9e0ec91b729b44319839f751e036d8c1 (127.16.67.65:36583): Network error: Client connection negotiation failed: client connection to 127.16.67.65:36583: connect: Connection refused (error 111)
02:04:09.459 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.458964 17479 raft_consensus.cc:2468] T 66d46d031bb04cfebcc1389692a56af1 P f02b99900ae14094a50f21ad3593d551 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91b849f0528142ed8048da0214d55cce in term 2.
02:04:09.459 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.459394 17895 raft_consensus.cc:2749] T 66d46d031bb04cfebcc1389692a56af1 P f02b99900ae14094a50f21ad3593d551 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
02:04:09.460 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.460029 17602 leader_election.cc:304] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91b849f0528142ed8048da0214d55cce, f02b99900ae14094a50f21ad3593d551; no voters: 9e0ec91b729b44319839f751e036d8c1
02:04:09.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.461057 17894 raft_consensus.cc:2804] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce [term 2 FOLLOWER]: Leader election won for term 2
02:04:09.463 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.463222 17894 raft_consensus.cc:697] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce [term 2 LEADER]: Becoming Leader. State: Replica: 91b849f0528142ed8048da0214d55cce, State: Running, Role: LEADER
02:04:09.464 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.464068 17894 consensus_queue.cc:237] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91b849f0528142ed8048da0214d55cce" member_type: VOTER last_known_addr { host: "127.16.67.67" port: 44863 } } peers { permanent_uuid: "f02b99900ae14094a50f21ad3593d551" member_type: VOTER last_known_addr { host: "127.16.67.66" port: 46107 } } peers { permanent_uuid: "9e0ec91b729b44319839f751e036d8c1" member_type: VOTER last_known_addr { host: "127.16.67.65" port: 36583 } }
02:04:09.475 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.474037 16949 catalog_manager.cc:5719] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce reported cstate change: term changed from 1 to 2, leader changed from 9e0ec91b729b44319839f751e036d8c1 (127.16.67.65) to 91b849f0528142ed8048da0214d55cce (127.16.67.67). New cstate: current_term: 2 leader_uuid: "91b849f0528142ed8048da0214d55cce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91b849f0528142ed8048da0214d55cce" member_type: VOTER last_known_addr { host: "127.16.67.67" port: 44863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f02b99900ae14094a50f21ad3593d551" member_type: VOTER last_known_addr { host: "127.16.67.66" port: 46107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0ec91b729b44319839f751e036d8c1" member_type: VOTER last_known_addr { host: "127.16.67.65" port: 36583 } health_report { overall_health: UNKNOWN } } }
02:04:09.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260812 02:04:09.882920 17610 consensus_peers.cc:597] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce -> Peer 9e0ec91b729b44319839f751e036d8c1 (127.16.67.65:36583): Couldn't send request to peer 9e0ec91b729b44319839f751e036d8c1. Status: Network error: Client connection negotiation failed: client connection to 127.16.67.65:36583: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
02:04:09.945 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.944633 17479 raft_consensus.cc:1275] T 66d46d031bb04cfebcc1389692a56af1 P f02b99900ae14094a50f21ad3593d551 [term 2 FOLLOWER]: Refusing update from remote peer 91b849f0528142ed8048da0214d55cce: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
02:04:09.946 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:09.946099 17894 consensus_queue.cc:1048] T 66d46d031bb04cfebcc1389692a56af1 P 91b849f0528142ed8048da0214d55cce [LEADER]: Connected to new peer: Peer: permanent_uuid: "f02b99900ae14094a50f21ad3593d551" member_type: VOTER last_known_addr { host: "127.16.67.66" port: 46107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
02:04:10.075 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 66d46d031bb04cfebcc1389692a56af1 for table 'org.apache.kudu.client.TestLeaderFailover-1786500228891' with partition [<start>, <end>)
02:04:10.076 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=66d46d031bb04cfebcc1389692a56af1@[91b849f0528142ed8048da0214d55cce(127.16.67.67:44863)[L],9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583),f02b99900ae14094a50f21ad3593d551(127.16.67.66:46107)]}]
02:04:10.077 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=66d46d031bb04cfebcc1389692a56af1, attempt=12, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=2040), Traces: [0ms] sending RPC to server 9e0ec91b729b44319839f751e036d8c1, [1ms] received response from server 9e0ec91b729b44319839f751e036d8c1: Network error: connection closed, [1ms] delaying RPC due to: Network error: connection closed, [14ms] refreshing cache from master, [15ms] Sub RPC GetTableLocations: sending RPC to server master-127.16.67.124:33409, [19ms] Sub RPC GetTableLocations: received response from server master-127.16.67.124:33409: OK, [21ms] sending RPC to server 9e0ec91b729b44319839f751e036d8c1, [25ms] received response from server 9e0ec91b729b44319839f751e036d8c1: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [25ms] delaying RPC due to: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [34ms] refreshing cache from master, [34ms] Sub RPC GetTableLocations: sending RPC to server master-127.16.67.124:33409, [40ms] Sub RPC GetTableLocations: received response from server master-127.16.67.124:33409: OK, [42ms] sending RPC to server 9e0ec91b729b44319839f751e036d8c1, [45ms] received response from server 9e0ec91b729b44319839f751e036d8c1: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [45ms] delaying RPC due to: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [74ms] refreshing cache from master, [74ms] Sub RPC GetTableLocations: sending RPC to server master-127.16.67.124:33409, [79ms] Sub RPC GetTableLocations: received response from server master-127.16.67.124:33409: OK, [81ms] sending RPC to server 9e0ec91b729b44319839f751e036d8c1, [84ms] received response from server 9e0ec91b729b44319839f751e036d8c1: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [84ms] delaying RPC due to: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [114ms] refreshing cache from master, [114ms] Sub RPC GetTableLocations: sending RPC to server master-127.16.67.124:33409, [120ms] Sub RPC GetTableLocations: received response from server master-127.16.67.124:33409: OK, [121ms] sending RPC to server 9e0ec91b729b44319839f751e036d8c1, [124ms] received response from server 9e0ec91b729b44319839f751e036d8c1: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [124ms] delaying RPC due to: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [215ms] refreshing cache from master, [215ms] Sub RPC GetTableLocations: sending RPC to server master-127.16.67.124:33409, [220ms] Sub RPC GetTableLocations: received response from server master-127.16.67.124:33409: OK, [222ms] sending RPC to server 9e0ec91b729b44319839f751e036d8c1, [224ms] received response from server 9e0ec91b729b44319839f751e036d8c1: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [224ms] delaying RPC due to: Network error: Failed to connect to peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583): Connection refused: /127.16.67.65:36583, [2034ms] refreshing cache from master, [2034ms] Sub RPC GetTableLocations: sending RPC to server master-127.16.67.124:33409, [2039ms] Sub RPC GetTableLocations: received response from server master-127.16.67.124:33409: OK, deferred=Deferred@241725940(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$227/0x00007f37ec338000@6a954c0b -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1548372956 after retry RPC@1437078598) -> (continuation of Deferred@1626623337 after retry RPC@886926193) -> (continuation of Deferred@1914200216 after retry RPC@300694895) -> (continuation of Deferred@2130266701 after retry RPC@1939786471) -> (continuation of Deferred@464448187 after retry RPC@509788891), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2c8a739a -> wakeup thread Time-limited test -> (continuation of Deferred@1548372956 after retry RPC@1437078598) -> (continuation of Deferred@1626623337 after retry RPC@886926193) -> (continuation of Deferred@1914200216 after retry RPC@300694895) -> (continuation of Deferred@2130266701 after retry RPC@1939786471) -> (continuation of Deferred@464448187 after retry RPC@509788891))) after lookup
02:04:10.078 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 91b849f0528142ed8048da0214d55cce(127.16.67.67:44863)] connecting to peer
02:04:10.079 [DEBUG - kudu-nio-1] (Connection.java:757) [peer 91b849f0528142ed8048da0214d55cce(127.16.67.67:44863)] Successfully connected to peer
02:04:10.109 [DEBUG - kudu-nio-1] (Negotiator.java:918) Authenticated connection [id: 0xf4178a03, L:/127.0.0.1:51484 - R:/127.16.67.67:44863] using TOKEN/null
02:04:10.177 [DEBUG - kudu-nio-1] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 6 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestLeaderFailover-1786500228891, tablet=66d46d031bb04cfebcc1389692a56af1, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
02:04:10.182 [DEBUG - main] (Connection.java:575) [peer 91b849f0528142ed8048da0214d55cce(127.16.67.67:44863)] disconnecting while in state READY
02:04:10.183 [DEBUG - main] (Connection.java:575) [peer master-127.16.67.126:34295(127.16.67.126:34295)] disconnecting while in state READY
02:04:10.183 [DEBUG - main] (Connection.java:575) [peer 9e0ec91b729b44319839f751e036d8c1(127.16.67.65:36583)] disconnecting while in state TERMINATED
02:04:10.185 [DEBUG - main] (Connection.java:575) [peer master-127.16.67.125:32941(127.16.67.125:32941)] disconnecting while in state READY
02:04:10.186 [DEBUG - kudu-nio-1] (Connection.java:239) [peer 91b849f0528142ed8048da0214d55cce(127.16.67.67:44863)] handling channelInactive
02:04:10.186 [DEBUG - kudu-nio-2] (Connection.java:239) [peer master-127.16.67.126:34295(127.16.67.126:34295)] handling channelInactive
02:04:10.186 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.16.67.125:32941(127.16.67.125:32941)] handling channelInactive
02:04:10.186 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 91b849f0528142ed8048da0214d55cce(127.16.67.67:44863)] cleaning up while in state READY due to: connection closed
02:04:10.186 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.16.67.125:32941(127.16.67.125:32941)] cleaning up while in state READY due to: connection closed
02:04:10.187 [DEBUG - main] (Connection.java:575) [peer master-127.16.67.124:33409(127.16.67.124:33409)] disconnecting while in state READY
02:04:10.187 [DEBUG - kudu-nio-2] (Connection.java:709) [peer master-127.16.67.126:34295(127.16.67.126:34295)] cleaning up while in state READY due to: connection closed
02:04:10.188 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2704) Releasing all remaining resources
02:04:10.190 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.16.67.124:33409(127.16.67.124:33409)] handling channelInactive
02:04:10.190 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.16.67.124:33409(127.16.67.124:33409)] cleaning up while in state READY due to: connection closed
02:04:10.194 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:10.191220 16653 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskzRoOVp/build/tsan/bin/kudu with pid 17326
02:04:10.243 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:10.241657 16653 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskzRoOVp/build/tsan/bin/kudu with pid 17573
02:04:10.335 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:10.335000 16653 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskzRoOVp/build/tsan/bin/kudu with pid 16667
02:04:10.373 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:10.372601 16653 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskzRoOVp/build/tsan/bin/kudu with pid 16776
02:04:10.408 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260812 02:04:10.408309 16653 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskzRoOVp/build/tsan/bin/kudu with pid 16889
02:04:11.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-08-12T02:04:11Z chronyd exiting

Time: 23.847

OK (1 test)

