Note: This is test shard 3 of 4.
[==========] Running 4 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 4 tests from HmsConfigurations/MasterFailoverTest
[ RUN      ] HmsConfigurations/MasterFailoverTest.TestPauseAfterCreateTableIssued/0
2026-08-12T06:36:26Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:36:26Z Disabled control of system clock
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260812 06:36:26.191761 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.254:43189
--webserver_interface=127.19.104.254
--webserver_port=0
--builtin_ntp_servers=127.19.104.212:40453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.104.254:43189,127.19.104.253:35395,127.19.104.252:37967
--leader_failure_max_missed_heartbeat_periods=1.0 with env {}
W20260812 06:36:26.300904 19884 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:36:26.301146 19884 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:36:26.301185 19884 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:36:26.304518 19884 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:36:26.304601 19884 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:36:26.304625 19884 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:36:26.304641 19884 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:36:26.308758 19884 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:40453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.104.254:43189,127.19.104.253:35395,127.19.104.252:37967
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.254:43189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.104.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.19884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:36:26.309866 19884 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:26.310958 19884 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:26.317036 19891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:26.317083 19893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:26.317086 19890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:26.317281 19884 server_base.cc:1061] running on GCE node
I20260812 06:36:26.317780 19884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:36:26.318719 19884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:36:26.319903 19884 hybrid_clock.cc:648] HybridClock initialized: now 1786516586319897 us; error 75 us; skew 500 ppm
I20260812 06:36:26.321920 19884 webserver.cc:533] Webserver started at http://127.19.104.254:45345/ using document root <none> and password file <none>
I20260812 06:36:26.322525 19884 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:26.322614 19884 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:26.322937 19884 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:26.324467 19884 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-0/data/instance:
uuid: "05e04be5fbf84feca8531c53facf72c7"
format_stamp: "Formatted at 2026-08-12 06:36:26 on dist-test-slave-8n49"
I20260812 06:36:26.325028 19884 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-0/wal/instance:
uuid: "05e04be5fbf84feca8531c53facf72c7"
format_stamp: "Formatted at 2026-08-12 06:36:26 on dist-test-slave-8n49"
I20260812 06:36:26.328330 19884 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 06:36:26.330673 19900 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:26.331696 19884 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:36:26.331842 19884 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-0/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-0/wal
uuid: "05e04be5fbf84feca8531c53facf72c7"
format_stamp: "Formatted at 2026-08-12 06:36:26 on dist-test-slave-8n49"
I20260812 06:36:26.331952 19884 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:26.348202 19884 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:26.349116 19884 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:26.349351 19884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:26.358120 19884 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.254:43189
I20260812 06:36:26.358148 19958 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.254:43189 every 8 connection(s)
I20260812 06:36:26.359175 19884 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-0/data/info.pb
I20260812 06:36:26.362257 19960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:26.366227 19960 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:36:26.368443 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 19884
I20260812 06:36:26.368614 19875 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-0/wal/instance
I20260812 06:36:26.370265 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.253:35395
--webserver_interface=127.19.104.253
--webserver_port=0
--builtin_ntp_servers=127.19.104.212:40453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.104.254:43189,127.19.104.253:35395,127.19.104.252:37967
--leader_failure_max_missed_heartbeat_periods=1.0 with env {}
I20260812 06:36:26.380735 19960 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } has no permanent_uuid. Determining permanent_uuid...
W20260812 06:36:26.381680 19904 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.104.253:35395: connect: Connection refused (error 111)
W20260812 06:36:26.382687 19960 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.253:35395: Network error: Client connection negotiation failed: client connection to 127.19.104.253:35395: connect: Connection refused (error 111)
I20260812 06:36:26.431937 19960 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } attempt: 1
W20260812 06:36:26.433727 19960 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.253:35395: Network error: Client connection negotiation failed: client connection to 127.19.104.253:35395: connect: Connection refused (error 111)
W20260812 06:36:26.479817 19963 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:36:26.480113 19963 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:36:26.480195 19963 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:36:26.483572 19963 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:36:26.483678 19963 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:36:26.483716 19963 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:36:26.483758 19963 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:36:26.487742 19963 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:40453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.104.254:43189,127.19.104.253:35395,127.19.104.252:37967
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.253:35395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.104.253
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.19963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:36:26.488919 19963 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:26.490041 19963 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:26.495616 19975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:26.495659 19974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:26.495646 19977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:26.495914 19963 server_base.cc:1061] running on GCE node
I20260812 06:36:26.496680 19963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:36:26.497645 19963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:36:26.498833 19963 hybrid_clock.cc:648] HybridClock initialized: now 1786516586498823 us; error 28 us; skew 500 ppm
I20260812 06:36:26.500797 19963 webserver.cc:533] Webserver started at http://127.19.104.253:45875/ using document root <none> and password file <none>
I20260812 06:36:26.501389 19963 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:26.501471 19963 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:26.501703 19963 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:26.501899 19960 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } attempt: 2
W20260812 06:36:26.503437 19960 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.253:35395: Network error: Client connection negotiation failed: client connection to 127.19.104.253:35395: connect: Connection refused (error 111)
I20260812 06:36:26.503737 19963 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-1/data/instance:
uuid: "92de3a4448694c9b945dc0beeb24c1d5"
format_stamp: "Formatted at 2026-08-12 06:36:26 on dist-test-slave-8n49"
I20260812 06:36:26.504236 19963 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-1/wal/instance:
uuid: "92de3a4448694c9b945dc0beeb24c1d5"
format_stamp: "Formatted at 2026-08-12 06:36:26 on dist-test-slave-8n49"
I20260812 06:36:26.507647 19963 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.006s	sys 0.000s
I20260812 06:36:26.509984 19985 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:26.511001 19963 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 06:36:26.511145 19963 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-1/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-1/wal
uuid: "92de3a4448694c9b945dc0beeb24c1d5"
format_stamp: "Formatted at 2026-08-12 06:36:26 on dist-test-slave-8n49"
I20260812 06:36:26.511265 19963 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:26.538914 19963 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:26.539642 19963 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:26.539855 19963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:26.549309 20042 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.253:35395 every 8 connection(s)
I20260812 06:36:26.549310 19963 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.253:35395
I20260812 06:36:26.550865 19963 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-1/data/info.pb
I20260812 06:36:26.554756 20043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:26.557909 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 19963
I20260812 06:36:26.558027 19875 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-1/wal/instance
I20260812 06:36:26.558851 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.252:37967
--webserver_interface=127.19.104.252
--webserver_port=0
--builtin_ntp_servers=127.19.104.212:40453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.104.254:43189,127.19.104.253:35395,127.19.104.252:37967
--leader_failure_max_missed_heartbeat_periods=1.0 with env {}
I20260812 06:36:26.558115 20043 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:36:26.567682 20043 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:36:26.573712 20043 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } has no permanent_uuid. Determining permanent_uuid...
W20260812 06:36:26.574553 19989 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.104.252:37967: connect: Connection refused (error 111)
W20260812 06:36:26.575399 20043 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.252:37967: Network error: Client connection negotiation failed: client connection to 127.19.104.252:37967: connect: Connection refused (error 111)
I20260812 06:36:26.594620 19960 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } attempt: 3
I20260812 06:36:26.599375 19960 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } has no permanent_uuid. Determining permanent_uuid...
W20260812 06:36:26.600831 19960 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.252:37967: Network error: Client connection negotiation failed: client connection to 127.19.104.252:37967: connect: Connection refused (error 111)
I20260812 06:36:26.624647 20043 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } attempt: 1
W20260812 06:36:26.626573 20043 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.252:37967: Network error: Client connection negotiation failed: client connection to 127.19.104.252:37967: connect: Connection refused (error 111)
I20260812 06:36:26.632053 19960 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } attempt: 1
W20260812 06:36:26.633731 19960 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.252:37967: Network error: Client connection negotiation failed: client connection to 127.19.104.252:37967: connect: Connection refused (error 111)
W20260812 06:36:26.672573 20046 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:36:26.673075 20046 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:36:26.673161 20046 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:36:26.676803 20046 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:36:26.676923 20046 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:36:26.676964 20046 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:36:26.677003 20046 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:36:26.681383 20046 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:40453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.104.254:43189,127.19.104.253:35395,127.19.104.252:37967
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.252:37967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.104.252
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.20046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:36:26.682889 20046 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:26.684129 20046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:26.690073 20055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:26.690101 20058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:26.690073 20056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:26.690452 20046 server_base.cc:1061] running on GCE node
I20260812 06:36:26.691135 20046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:36:26.691980 20046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:36:26.693181 20046 hybrid_clock.cc:648] HybridClock initialized: now 1786516586693129 us; error 69 us; skew 500 ppm
I20260812 06:36:26.694779 20043 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } attempt: 2
I20260812 06:36:26.695467 20046 webserver.cc:533] Webserver started at http://127.19.104.252:39173/ using document root <none> and password file <none>
I20260812 06:36:26.696102 20046 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:26.696165 20046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
W20260812 06:36:26.696458 20043 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.252:37967: Network error: Client connection negotiation failed: client connection to 127.19.104.252:37967: connect: Connection refused (error 111)
I20260812 06:36:26.696511 20046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:26.698124 20046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-2/data/instance:
uuid: "751096c394f2450a8af13ab6b6586523"
format_stamp: "Formatted at 2026-08-12 06:36:26 on dist-test-slave-8n49"
I20260812 06:36:26.698721 20046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-2/wal/instance:
uuid: "751096c394f2450a8af13ab6b6586523"
format_stamp: "Formatted at 2026-08-12 06:36:26 on dist-test-slave-8n49"
I20260812 06:36:26.702188 20046 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 06:36:26.704278 20066 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:26.705353 20046 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:36:26.705461 20046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-2/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-2/wal
uuid: "751096c394f2450a8af13ab6b6586523"
format_stamp: "Formatted at 2026-08-12 06:36:26 on dist-test-slave-8n49"
I20260812 06:36:26.705539 20046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:26.708926 19960 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } attempt: 2
W20260812 06:36:26.710438 19960 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.252:37967: Network error: Client connection negotiation failed: client connection to 127.19.104.252:37967: connect: Connection refused (error 111)
I20260812 06:36:26.743515 20046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:26.744261 20046 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:26.744468 20046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:26.752421 20126 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.252:37967 every 8 connection(s)
I20260812 06:36:26.752431 20046 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.252:37967
I20260812 06:36:26.753543 20046 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-2/data/info.pb
I20260812 06:36:26.756359 20127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:26.757243 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 20046
I20260812 06:36:26.757349 19875 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/master-2/wal/instance
I20260812 06:36:26.760066 20127 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:36:26.769652 20127 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:36:26.775080 20127 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:36:26.781915 20127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523: Bootstrap starting.
I20260812 06:36:26.784221 20127 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:26.785429 20127 log.cc:826] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523: Log is configured to *not* fsync() on all Append() calls
I20260812 06:36:26.787146 20127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523: No bootstrap required, opened a new log
I20260812 06:36:26.787647 20043 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } attempt: 3
I20260812 06:36:26.791769 20127 raft_consensus.cc:359] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:26.792030 20127 raft_consensus.cc:385] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:26.792176 20127 raft_consensus.cc:740] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 751096c394f2450a8af13ab6b6586523, State: Initialized, Role: FOLLOWER
I20260812 06:36:26.792739 20127 consensus_queue.cc:260] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:26.793278 20137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } } }
I20260812 06:36:26.793390 20137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:36:26.793982 20127 sys_catalog.cc:565] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:36:26.794595 20043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5: Bootstrap starting.
I20260812 06:36:26.797056 20043 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:26.797675 20043 log.cc:826] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5: Log is configured to *not* fsync() on all Append() calls
W20260812 06:36:26.799338 20151 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:36:26.799468 20151 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:36:26.797957 20046 master_runner.cc:187] Error getting master registration for 127.19.104.254:43189: OK, instance_id { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" instance_seqno: 1786516586355573 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 06:36:26.800262 20046 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260812 06:36:26.800384 20043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5: No bootstrap required, opened a new log
I20260812 06:36:26.804055 20043 raft_consensus.cc:359] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:26.804243 20043 raft_consensus.cc:385] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:26.804332 20043 raft_consensus.cc:740] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92de3a4448694c9b945dc0beeb24c1d5, State: Initialized, Role: FOLLOWER
I20260812 06:36:26.804858 20043 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:26.805475 20154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } } }
I20260812 06:36:26.805619 20154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:36:26.806301 20043 sys_catalog.cc:565] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:36:26.809623 19960 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } attempt: 3
W20260812 06:36:26.812156 20167 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:36:26.812289 20167 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:36:26.810796 19963 master_runner.cc:187] Error getting master registration for 127.19.104.254:43189: OK, instance_id { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" instance_seqno: 1786516586355573 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 06:36:26.813100 19963 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260812 06:36:26.819152 19960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7: Bootstrap starting.
I20260812 06:36:26.821977 19960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:26.822891 19960 log.cc:826] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7: Log is configured to *not* fsync() on all Append() calls
I20260812 06:36:26.824874 19960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7: No bootstrap required, opened a new log
I20260812 06:36:26.828624 19960 raft_consensus.cc:359] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:26.828856 19960 raft_consensus.cc:385] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:26.828936 19960 raft_consensus.cc:740] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05e04be5fbf84feca8531c53facf72c7, State: Initialized, Role: FOLLOWER
I20260812 06:36:26.829447 19960 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:26.829979 20170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } } }
I20260812 06:36:26.830160 20170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:36:26.830768 19960 sys_catalog.cc:565] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 06:36:26.838673 20181 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:36:26.838852 20181 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:36:26.839926 19884 master_runner.cc:264] No leader master found from master 05e04be5fbf84feca8531c53facf72c7
I20260812 06:36:26.839994 19884 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260812 06:36:26.850953 19906 tablet.cc:2406] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:36:26.906509 20170 raft_consensus.cc:493] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:26.906745 20170 raft_consensus.cc:515] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:26.908021 20170 leader_election.cc:290] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92de3a4448694c9b945dc0beeb24c1d5 (127.19.104.253:35395), 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967)
I20260812 06:36:26.909103 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92de3a4448694c9b945dc0beeb24c1d5" is_pre_election: true
I20260812 06:36:26.909415 20015 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05e04be5fbf84feca8531c53facf72c7 in term 0.
I20260812 06:36:26.910019 19904 leader_election.cc:304] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05e04be5fbf84feca8531c53facf72c7, 92de3a4448694c9b945dc0beeb24c1d5; no voters: 
I20260812 06:36:26.910703 20097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:36:26.910977 20097 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05e04be5fbf84feca8531c53facf72c7 in term 0.
I20260812 06:36:26.911280 20170 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:36:26.911352 20170 raft_consensus.cc:493] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:36:26.911396 20170 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:26.913156 20170 raft_consensus.cc:515] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:26.913578 20170 leader_election.cc:290] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 1 election: Requested vote from peers 92de3a4448694c9b945dc0beeb24c1d5 (127.19.104.253:35395), 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967)
I20260812 06:36:26.914085 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92de3a4448694c9b945dc0beeb24c1d5"
I20260812 06:36:26.914206 20015 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:26.915750 20015 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05e04be5fbf84feca8531c53facf72c7 in term 1.
I20260812 06:36:26.916535 20097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523"
I20260812 06:36:26.916721 20097 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:26.917829 20097 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05e04be5fbf84feca8531c53facf72c7 in term 1.
I20260812 06:36:26.918270 19904 leader_election.cc:304] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05e04be5fbf84feca8531c53facf72c7, 751096c394f2450a8af13ab6b6586523; no voters: 
I20260812 06:36:26.918663 20170 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:36:26.918943 20170 raft_consensus.cc:697] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 1 LEADER]: Becoming Leader. State: Replica: 05e04be5fbf84feca8531c53facf72c7, State: Running, Role: LEADER
I20260812 06:36:26.919337 20170 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:26.926241 20170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05e04be5fbf84feca8531c53facf72c7. Latest consensus state: current_term: 1 leader_uuid: "05e04be5fbf84feca8531c53facf72c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } } }
I20260812 06:36:26.926453 20170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [sys.catalog]: This master's current role is: LEADER
I20260812 06:36:26.926910 20186 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:36:26.927981 20186 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:36:26.935101 20097 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [term 1 FOLLOWER]: Refusing update from remote peer 05e04be5fbf84feca8531c53facf72c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:36:26.935902 20183 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:36:26.943427 20137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05e04be5fbf84feca8531c53facf72c7. Latest consensus state: current_term: 1 leader_uuid: "05e04be5fbf84feca8531c53facf72c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } } }
I20260812 06:36:26.943543 20137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:36:26.944991 20015 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 1 FOLLOWER]: Refusing update from remote peer 05e04be5fbf84feca8531c53facf72c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:36:26.945647 20183 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:36:26.950003 20154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05e04be5fbf84feca8531c53facf72c7. Latest consensus state: current_term: 1 leader_uuid: "05e04be5fbf84feca8531c53facf72c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } } }
I20260812 06:36:26.950147 20154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:36:26.954511 20183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "05e04be5fbf84feca8531c53facf72c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } } }
I20260812 06:36:26.954654 20183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [sys.catalog]: This master's current role is: LEADER
I20260812 06:36:26.955987 20154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "05e04be5fbf84feca8531c53facf72c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } } }
I20260812 06:36:26.956108 20154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:36:26.956586 20187 mvcc.cc:204] Tried to move back new op lower bound from 7317571940078399488 to 7317571940040691712. Current Snapshot: MvccSnapshot[applied={T|T < 7317571940078399488}]
I20260812 06:36:26.959681 20191 mvcc.cc:204] Tried to move back new op lower bound from 7317571940078399488 to 7317571940040691712. Current Snapshot: MvccSnapshot[applied={T|T < 7317571940078399488}]
I20260812 06:36:26.960640 20170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "05e04be5fbf84feca8531c53facf72c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } } }
I20260812 06:36:26.960762 20170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [sys.catalog]: This master's current role is: LEADER
I20260812 06:36:26.962118 20137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "05e04be5fbf84feca8531c53facf72c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } } }
I20260812 06:36:26.962239 20137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:36:26.965395 20189 mvcc.cc:204] Tried to move back new op lower bound from 7317571940078399488 to 7317571940040691712. Current Snapshot: MvccSnapshot[applied={T|T < 7317571940078399488}]
I20260812 06:36:26.968816 20186 catalog_manager.cc:1383] Generated new cluster ID: c2f38e66602a4a4385d5be08edc0add3
I20260812 06:36:26.968883 20186 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:36:27.019697 20186 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 06:36:27.021231 20186 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:36:27.036448 20186 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7: Generated new TSK 0
I20260812 06:36:27.037935 20186 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:36:27.047942 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.104.193:0
--local_ip_for_outbound_sockets=127.19.104.193
--webserver_interface=127.19.104.193
--webserver_port=0
--tserver_master_addrs=127.19.104.254:43189,127.19.104.253:35395,127.19.104.252:37967
--builtin_ntp_servers=127.19.104.212:40453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
W20260812 06:36:27.159042 20198 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:36:27.159248 20198 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:36:27.159297 20198 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:36:27.162700 20198 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:36:27.162846 20198 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.104.193
I20260812 06:36:27.166721 20198 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:40453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.104.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.104.193
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.19.104.254:43189,127.19.104.253:35395,127.19.104.252:37967
--never_fsync=true
--heap_profile_path=/tmp/kudu.20198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.104.193

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:36:27.167827 20198 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:27.169026 20198 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:27.175411 20206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:27.175439 20208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:27.175725 20198 server_base.cc:1061] running on GCE node
W20260812 06:36:27.175411 20205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:27.176384 20198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:36:27.177333 20198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:36:27.178575 20198 hybrid_clock.cc:648] HybridClock initialized: now 1786516587178497 us; error 94 us; skew 500 ppm
I20260812 06:36:27.181231 20198 webserver.cc:533] Webserver started at http://127.19.104.193:42745/ using document root <none> and password file <none>
I20260812 06:36:27.182085 20198 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:27.182170 20198 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:27.182430 20198 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:27.184785 20198 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-0/data/instance:
uuid: "52ce252555214626853d4de2f72c7b7c"
format_stamp: "Formatted at 2026-08-12 06:36:27 on dist-test-slave-8n49"
I20260812 06:36:27.185454 20198 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-0/wal/instance:
uuid: "52ce252555214626853d4de2f72c7b7c"
format_stamp: "Formatted at 2026-08-12 06:36:27 on dist-test-slave-8n49"
I20260812 06:36:27.190251 20198 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260812 06:36:27.193224 20216 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:27.194496 20198 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.004s	sys 0.000s
I20260812 06:36:27.194689 20198 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-0/wal
uuid: "52ce252555214626853d4de2f72c7b7c"
format_stamp: "Formatted at 2026-08-12 06:36:27 on dist-test-slave-8n49"
I20260812 06:36:27.194808 20198 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:27.219632 20198 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:27.220631 20198 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:27.220877 20198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:27.221690 20198 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:36:27.223467 20198 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:36:27.223539 20198 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:27.223605 20198 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:36:27.223642 20198 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:27.238122 20198 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.193:37403
I20260812 06:36:27.238150 20350 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.193:37403 every 8 connection(s)
I20260812 06:36:27.239589 20198 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-0/data/info.pb
I20260812 06:36:27.244804 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 20198
I20260812 06:36:27.244920 19875 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-0/wal/instance
I20260812 06:36:27.249611 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.104.194:0
--local_ip_for_outbound_sockets=127.19.104.194
--webserver_interface=127.19.104.194
--webserver_port=0
--tserver_master_addrs=127.19.104.254:43189,127.19.104.253:35395,127.19.104.252:37967
--builtin_ntp_servers=127.19.104.212:40453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260812 06:36:27.251672 20352 heartbeater.cc:344] Connected to a master server at 127.19.104.254:43189
I20260812 06:36:27.251947 20352 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:27.253152 20353 heartbeater.cc:344] Connected to a master server at 127.19.104.253:35395
I20260812 06:36:27.253237 20353 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:27.253368 20352 heartbeater.cc:507] Master 127.19.104.254:43189 requested a full tablet report, sending...
I20260812 06:36:27.253391 20353 heartbeater.cc:507] Master 127.19.104.253:35395 requested a full tablet report, sending...
I20260812 06:36:27.254909 20004 ts_manager.cc:194] Registered new tserver with Master: 52ce252555214626853d4de2f72c7b7c (127.19.104.193:37403)
I20260812 06:36:27.257162 19919 ts_manager.cc:194] Registered new tserver with Master: 52ce252555214626853d4de2f72c7b7c (127.19.104.193:37403)
I20260812 06:36:27.258926 19919 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.104.193:52151
I20260812 06:36:27.259545 20351 heartbeater.cc:344] Connected to a master server at 127.19.104.252:37967
I20260812 06:36:27.259694 20351 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:27.259955 20351 heartbeater.cc:507] Master 127.19.104.252:37967 requested a full tablet report, sending...
I20260812 06:36:27.261313 20084 ts_manager.cc:194] Registered new tserver with Master: 52ce252555214626853d4de2f72c7b7c (127.19.104.193:37403)
W20260812 06:36:27.365936 20359 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:36:27.366204 20359 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:36:27.366271 20359 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:36:27.371188 20359 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:36:27.371397 20359 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.104.194
I20260812 06:36:27.377117 20359 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:40453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.104.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.104.194
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.19.104.254:43189,127.19.104.253:35395,127.19.104.252:37967
--never_fsync=true
--heap_profile_path=/tmp/kudu.20359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.104.194

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:36:27.378851 20359 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:27.380586 20359 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:27.389297 20364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:27.389648 20369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:27.389773 20365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:27.390040 20359 server_base.cc:1061] running on GCE node
I20260812 06:36:27.390488 20359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:36:27.391243 20359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:36:27.392452 20359 hybrid_clock.cc:648] HybridClock initialized: now 1786516587392422 us; error 41 us; skew 500 ppm
I20260812 06:36:27.395057 20359 webserver.cc:533] Webserver started at http://127.19.104.194:46029/ using document root <none> and password file <none>
I20260812 06:36:27.395920 20359 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:27.396005 20359 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:27.396260 20359 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:27.398679 20359 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-1/data/instance:
uuid: "40f95405712443ac98dc145f98dd3293"
format_stamp: "Formatted at 2026-08-12 06:36:27 on dist-test-slave-8n49"
I20260812 06:36:27.399351 20359 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-1/wal/instance:
uuid: "40f95405712443ac98dc145f98dd3293"
format_stamp: "Formatted at 2026-08-12 06:36:27 on dist-test-slave-8n49"
I20260812 06:36:27.404768 20359 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.005s
I20260812 06:36:27.408895 20375 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:27.410014 20359 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 06:36:27.410113 20359 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-1/wal
uuid: "40f95405712443ac98dc145f98dd3293"
format_stamp: "Formatted at 2026-08-12 06:36:27 on dist-test-slave-8n49"
I20260812 06:36:27.410254 20359 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:27.431569 20359 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:27.432329 20359 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:27.432503 20359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:27.433205 20359 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:36:27.434221 20359 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:36:27.434270 20359 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:27.434309 20359 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:36:27.434330 20359 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:27.443882 20359 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.194:39609
I20260812 06:36:27.444059 20504 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.194:39609 every 8 connection(s)
I20260812 06:36:27.444868 20359 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-1/data/info.pb
I20260812 06:36:27.455199 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 20359
I20260812 06:36:27.455308 19875 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-1/wal/instance
I20260812 06:36:27.455937 20506 heartbeater.cc:344] Connected to a master server at 127.19.104.254:43189
I20260812 06:36:27.456230 20506 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:27.456970 20506 heartbeater.cc:507] Master 127.19.104.254:43189 requested a full tablet report, sending...
I20260812 06:36:27.457944 20507 heartbeater.cc:344] Connected to a master server at 127.19.104.253:35395
I20260812 06:36:27.458035 20507 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:27.458060 20505 heartbeater.cc:344] Connected to a master server at 127.19.104.252:37967
I20260812 06:36:27.458150 20505 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:27.458236 20507 heartbeater.cc:507] Master 127.19.104.253:35395 requested a full tablet report, sending...
I20260812 06:36:27.458349 19919 ts_manager.cc:194] Registered new tserver with Master: 40f95405712443ac98dc145f98dd3293 (127.19.104.194:39609)
I20260812 06:36:27.458369 20505 heartbeater.cc:507] Master 127.19.104.252:37967 requested a full tablet report, sending...
I20260812 06:36:27.459043 19919 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.104.194:33735
I20260812 06:36:27.459090 20004 ts_manager.cc:194] Registered new tserver with Master: 40f95405712443ac98dc145f98dd3293 (127.19.104.194:39609)
I20260812 06:36:27.459179 20083 ts_manager.cc:194] Registered new tserver with Master: 40f95405712443ac98dc145f98dd3293 (127.19.104.194:39609)
I20260812 06:36:27.460533 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.104.195:0
--local_ip_for_outbound_sockets=127.19.104.195
--webserver_interface=127.19.104.195
--webserver_port=0
--tserver_master_addrs=127.19.104.254:43189,127.19.104.253:35395,127.19.104.252:37967
--builtin_ntp_servers=127.19.104.212:40453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260812 06:36:27.536203 20516 raft_consensus.cc:493] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 05e04be5fbf84feca8531c53facf72c7)
I20260812 06:36:27.536356 20516 raft_consensus.cc:515] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:27.538378 19929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "751096c394f2450a8af13ab6b6586523" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "05e04be5fbf84feca8531c53facf72c7" is_pre_election: true
I20260812 06:36:27.538750 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "751096c394f2450a8af13ab6b6586523" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92de3a4448694c9b945dc0beeb24c1d5" is_pre_election: true
I20260812 06:36:27.538902 20015 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 751096c394f2450a8af13ab6b6586523 in term 1.
I20260812 06:36:27.539276 20070 leader_election.cc:304] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 751096c394f2450a8af13ab6b6586523, 92de3a4448694c9b945dc0beeb24c1d5; no voters: 05e04be5fbf84feca8531c53facf72c7
I20260812 06:36:27.539608 20516 leader_election.cc:290] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05e04be5fbf84feca8531c53facf72c7 (127.19.104.254:43189), 92de3a4448694c9b945dc0beeb24c1d5 (127.19.104.253:35395)
I20260812 06:36:27.539872 20516 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 06:36:27.539927 20516 raft_consensus.cc:493] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 05e04be5fbf84feca8531c53facf72c7)
I20260812 06:36:27.539978 20516 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:36:27.541392 20517 raft_consensus.cc:493] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 05e04be5fbf84feca8531c53facf72c7)
I20260812 06:36:27.541524 20517 raft_consensus.cc:515] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:27.542891 19929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92de3a4448694c9b945dc0beeb24c1d5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "05e04be5fbf84feca8531c53facf72c7" is_pre_election: true
I20260812 06:36:27.543339 20097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92de3a4448694c9b945dc0beeb24c1d5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:36:27.543550 20517 leader_election.cc:290] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05e04be5fbf84feca8531c53facf72c7 (127.19.104.254:43189), 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967)
I20260812 06:36:27.547516 20516 raft_consensus.cc:515] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:27.547837 20097 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92de3a4448694c9b945dc0beeb24c1d5 in current term 2: Already voted for candidate 751096c394f2450a8af13ab6b6586523 in this term.
I20260812 06:36:27.549109 19929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "751096c394f2450a8af13ab6b6586523" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "05e04be5fbf84feca8531c53facf72c7"
I20260812 06:36:27.549952 20516 leader_election.cc:290] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [CANDIDATE]: Term 2 election: Requested vote from peers 05e04be5fbf84feca8531c53facf72c7 (127.19.104.254:43189), 92de3a4448694c9b945dc0beeb24c1d5 (127.19.104.253:35395)
I20260812 06:36:27.550379 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "751096c394f2450a8af13ab6b6586523" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92de3a4448694c9b945dc0beeb24c1d5"
I20260812 06:36:27.550510 20015 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:36:27.551564 20015 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 751096c394f2450a8af13ab6b6586523 in term 2.
I20260812 06:36:27.551831 19989 leader_election.cc:304] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92de3a4448694c9b945dc0beeb24c1d5; no voters: 05e04be5fbf84feca8531c53facf72c7, 751096c394f2450a8af13ab6b6586523
I20260812 06:36:27.552481 20070 leader_election.cc:304] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 751096c394f2450a8af13ab6b6586523, 92de3a4448694c9b945dc0beeb24c1d5; no voters: 05e04be5fbf84feca8531c53facf72c7
I20260812 06:36:27.552903 20516 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 06:36:27.553051 20516 raft_consensus.cc:697] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [term 2 LEADER]: Becoming Leader. State: Replica: 751096c394f2450a8af13ab6b6586523, State: Running, Role: LEADER
I20260812 06:36:27.553421 20516 consensus_queue.cc:237] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:27.555015 20516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 751096c394f2450a8af13ab6b6586523. Latest consensus state: current_term: 2 leader_uuid: "751096c394f2450a8af13ab6b6586523" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } } }
I20260812 06:36:27.555188 20516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [sys.catalog]: This master's current role is: LEADER
I20260812 06:36:27.555588 20519 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:36:27.557056 20519 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:36:27.557788 20519 catalog_manager.cc:1295] Loaded cluster ID: c2f38e66602a4a4385d5be08edc0add3
I20260812 06:36:27.557902 20519 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:36:27.560194 20519 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:36:27.560966 20519 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523: Loaded TSK: 0
I20260812 06:36:27.562017 20519 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:36:27.562668 20517 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 06:36:27.578528 20515 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:36:27.578784 20515 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:36:27.578884 20515 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:36:27.582557 20515 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:36:27.582727 20515 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.104.195
I20260812 06:36:27.586947 20515 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:40453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.104.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.104.195
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.19.104.254:43189,127.19.104.253:35395,127.19.104.252:37967
--never_fsync=true
--heap_profile_path=/tmp/kudu.20515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.104.195

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:36:27.588269 20515 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:27.589494 20515 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:27.596249 20526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:27.596238 20527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:27.596539 20529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:27.596508 20515 server_base.cc:1061] running on GCE node
I20260812 06:36:27.597018 20515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:36:27.597632 20515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:36:27.598811 20515 hybrid_clock.cc:648] HybridClock initialized: now 1786516587598800 us; error 35 us; skew 500 ppm
I20260812 06:36:27.600664 20515 webserver.cc:533] Webserver started at http://127.19.104.195:40873/ using document root <none> and password file <none>
I20260812 06:36:27.601317 20515 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:27.601409 20515 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:27.601675 20515 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:27.603444 20515 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-2/data/instance:
uuid: "69b835823c91478787fbf47bbf635642"
format_stamp: "Formatted at 2026-08-12 06:36:27 on dist-test-slave-8n49"
I20260812 06:36:27.603956 20515 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-2/wal/instance:
uuid: "69b835823c91478787fbf47bbf635642"
format_stamp: "Formatted at 2026-08-12 06:36:27 on dist-test-slave-8n49"
I20260812 06:36:27.608083 20515 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.005s
I20260812 06:36:27.610769 20535 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:27.611861 20515 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:36:27.611987 20515 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-2/wal
uuid: "69b835823c91478787fbf47bbf635642"
format_stamp: "Formatted at 2026-08-12 06:36:27 on dist-test-slave-8n49"
I20260812 06:36:27.612100 20515 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:27.633045 20097 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [term 2 LEADER]: Rejecting Update request from peer 05e04be5fbf84feca8531c53facf72c7 for earlier term 1. Current term is 2. Ops: []
I20260812 06:36:27.633585 20541 consensus_queue.cc:1059] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 06:36:27.633914 20541 consensus_queue.cc:1243] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [LEADER]: Peer 751096c394f2450a8af13ab6b6586523 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260812 06:36:27.633997 20542 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 06:36:27.634054 20542 raft_consensus.cc:740] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 05e04be5fbf84feca8531c53facf72c7, State: Running, Role: LEADER
I20260812 06:36:27.634262 20542 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:27.634581 20542 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:36:27.642412 20515 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:27.643168 20515 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:27.643378 20515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:27.644057 20515 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:36:27.645228 20515 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:36:27.645310 20515 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:27.645382 20515 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:36:27.645433 20515 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:27.655403 20515 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.195:46055
I20260812 06:36:27.655431 20668 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.195:46055 every 8 connection(s)
I20260812 06:36:27.656597 20515 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-2/data/info.pb
I20260812 06:36:27.659308 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 20515
I20260812 06:36:27.659417 19875 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestPauseAfterCreateTableIssued_0.1786516586160649-19875-0/minicluster-data/ts-2/wal/instance
I20260812 06:36:27.667373 20669 heartbeater.cc:344] Connected to a master server at 127.19.104.252:37967
I20260812 06:36:27.667484 20671 heartbeater.cc:344] Connected to a master server at 127.19.104.253:35395
I20260812 06:36:27.667625 20669 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:27.667626 20671 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:27.668195 20669 heartbeater.cc:507] Master 127.19.104.252:37967 requested a full tablet report, sending...
I20260812 06:36:27.668196 20671 heartbeater.cc:507] Master 127.19.104.253:35395 requested a full tablet report, sending...
I20260812 06:36:27.668634 20670 heartbeater.cc:344] Connected to a master server at 127.19.104.254:43189
I20260812 06:36:27.668716 20670 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:27.668941 20670 heartbeater.cc:507] Master 127.19.104.254:43189 requested a full tablet report, sending...
I20260812 06:36:27.669348 20083 ts_manager.cc:194] Registered new tserver with Master: 69b835823c91478787fbf47bbf635642 (127.19.104.195:46055)
I20260812 06:36:27.669466 20004 ts_manager.cc:194] Registered new tserver with Master: 69b835823c91478787fbf47bbf635642 (127.19.104.195:46055)
I20260812 06:36:27.669664 19919 ts_manager.cc:194] Registered new tserver with Master: 69b835823c91478787fbf47bbf635642 (127.19.104.195:46055)
I20260812 06:36:27.670653 20083 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.104.195:44685
I20260812 06:36:27.680757 19875 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 06:36:27.695118 20083 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.test_pause_after_create_table_issued"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260812 06:36:27.696494 20083 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.test_pause_after_create_table_issued in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:36:27.700099 20015 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 2 FOLLOWER]: Refusing update from remote peer 751096c394f2450a8af13ab6b6586523: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20260812 06:36:27.700205 19929 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 2 FOLLOWER]: Refusing update from remote peer 751096c394f2450a8af13ab6b6586523: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20260812 06:36:27.700717 20516 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 06:36:27.700911 20518 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 06:36:27.703065 20517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 751096c394f2450a8af13ab6b6586523. Latest consensus state: current_term: 2 leader_uuid: "751096c394f2450a8af13ab6b6586523" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } } }
I20260812 06:36:27.703223 20517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:36:27.703528 20542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 751096c394f2450a8af13ab6b6586523. Latest consensus state: current_term: 2 leader_uuid: "751096c394f2450a8af13ab6b6586523" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } } }
I20260812 06:36:27.703642 20542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:36:27.703881 20518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "751096c394f2450a8af13ab6b6586523" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } } }
I20260812 06:36:27.703994 20518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [sys.catalog]: This master's current role is: LEADER
I20260812 06:36:27.704208 20517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "751096c394f2450a8af13ab6b6586523" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } } }
I20260812 06:36:27.704311 20517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:36:27.704733 20518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "751096c394f2450a8af13ab6b6586523" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } } }
I20260812 06:36:27.704844 20518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [sys.catalog]: This master's current role is: LEADER
I20260812 06:36:27.705971 20542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "751096c394f2450a8af13ab6b6586523" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } } }
I20260812 06:36:27.706094 20542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:36:27.708032 19875 master_failover-itest.cc:212] Pausing leader master
I20260812 06:36:27.763092 20352 heartbeater.cc:499] Master 127.19.104.254:43189 was elected leader, sending a full tablet report...
I20260812 06:36:27.813714 20167 catalog_manager.cc:1295] Loaded cluster ID: c2f38e66602a4a4385d5be08edc0add3
I20260812 06:36:27.813807 20167 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5: loading cluster ID for follower catalog manager: success
I20260812 06:36:27.815678 20167 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5: acquiring CA information for follower catalog manager: success
I20260812 06:36:27.816907 20167 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:36:27.840384 20181 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:36:27.961440 20506 heartbeater.cc:499] Master 127.19.104.254:43189 was elected leader, sending a full tablet report...
I20260812 06:36:28.172448 20669 heartbeater.cc:499] Master 127.19.104.252:37967 was elected leader, sending a full tablet report...
I20260812 06:36:28.211118 20696 raft_consensus.cc:493] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 751096c394f2450a8af13ab6b6586523)
I20260812 06:36:28.211107 20695 raft_consensus.cc:493] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 751096c394f2450a8af13ab6b6586523)
I20260812 06:36:28.211289 20696 raft_consensus.cc:515] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:28.211300 20695 raft_consensus.cc:515] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:28.211953 20696 leader_election.cc:290] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 05e04be5fbf84feca8531c53facf72c7 (127.19.104.254:43189), 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967)
I20260812 06:36:28.212182 19929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92de3a4448694c9b945dc0beeb24c1d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "05e04be5fbf84feca8531c53facf72c7" is_pre_election: true
I20260812 06:36:28.212189 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "92de3a4448694c9b945dc0beeb24c1d5" is_pre_election: true
I20260812 06:36:28.212327 20015 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05e04be5fbf84feca8531c53facf72c7 in term 2.
I20260812 06:36:28.212342 19929 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92de3a4448694c9b945dc0beeb24c1d5 in term 2.
I20260812 06:36:28.212508 20695 leader_election.cc:290] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 92de3a4448694c9b945dc0beeb24c1d5 (127.19.104.253:35395), 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967)
I20260812 06:36:28.212797 19988 leader_election.cc:304] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05e04be5fbf84feca8531c53facf72c7, 92de3a4448694c9b945dc0beeb24c1d5; no voters: 
I20260812 06:36:28.212807 19904 leader_election.cc:304] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05e04be5fbf84feca8531c53facf72c7, 92de3a4448694c9b945dc0beeb24c1d5; no voters: 
I20260812 06:36:28.212949 20696 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 06:36:28.212986 20695 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 06:36:28.213011 20696 raft_consensus.cc:493] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 751096c394f2450a8af13ab6b6586523)
I20260812 06:36:28.213053 20695 raft_consensus.cc:493] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 751096c394f2450a8af13ab6b6586523)
I20260812 06:36:28.213076 20696 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:36:28.213112 20695 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:36:28.213958 20695 raft_consensus.cc:515] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:28.213969 20696 raft_consensus.cc:515] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:28.214414 20695 leader_election.cc:290] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 3 election: Requested vote from peers 92de3a4448694c9b945dc0beeb24c1d5 (127.19.104.253:35395), 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967)
I20260812 06:36:28.214524 20696 leader_election.cc:290] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [CANDIDATE]: Term 3 election: Requested vote from peers 05e04be5fbf84feca8531c53facf72c7 (127.19.104.254:43189), 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967)
I20260812 06:36:28.214625 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "92de3a4448694c9b945dc0beeb24c1d5"
I20260812 06:36:28.214705 19929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92de3a4448694c9b945dc0beeb24c1d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "05e04be5fbf84feca8531c53facf72c7"
I20260812 06:36:28.214805 20015 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 05e04be5fbf84feca8531c53facf72c7 in current term 3: Already voted for candidate 92de3a4448694c9b945dc0beeb24c1d5 in this term.
I20260812 06:36:28.214915 19929 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 92de3a4448694c9b945dc0beeb24c1d5 in current term 3: Already voted for candidate 05e04be5fbf84feca8531c53facf72c7 in this term.
W20260812 06:36:28.764284 20351 heartbeater.cc:646] Failed to heartbeat to 127.19.104.252:37967 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.19.104.252:37967 timed out after 1.000s (SENT)
W20260812 06:36:28.961515 20505 heartbeater.cc:646] Failed to heartbeat to 127.19.104.252:37967 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.19.104.252:37967 timed out after 1.000s (SENT)
W20260812 06:36:29.173827 20669 heartbeater.cc:646] Failed to heartbeat to 127.19.104.252:37967 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.19.104.252:37967 timed out after 1.000s (SENT)
W20260812 06:36:29.312830 19989 leader_election.cc:336] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967): Timed out: RequestConsensusVote RPC to 127.19.104.252:37967 timed out after 0.549s (SENT)
W20260812 06:36:29.476775 19989 leader_election.cc:336] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967): Timed out: RequestConsensusVote RPC to 127.19.104.252:37967 timed out after 0.630s (SENT)
I20260812 06:36:29.476904 19989 leader_election.cc:304] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92de3a4448694c9b945dc0beeb24c1d5; no voters: 05e04be5fbf84feca8531c53facf72c7, 751096c394f2450a8af13ab6b6586523
I20260812 06:36:29.477399 20699 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260812 06:36:29.515878 19904 leader_election.cc:336] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967): Timed out: RequestConsensusVote RPC to 127.19.104.252:37967 timed out after 0.649s (SENT)
I20260812 06:36:29.516002 19904 leader_election.cc:304] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05e04be5fbf84feca8531c53facf72c7; no voters: 751096c394f2450a8af13ab6b6586523, 92de3a4448694c9b945dc0beeb24c1d5
I20260812 06:36:29.516405 20700 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260812 06:36:29.535599 19904 leader_election.cc:336] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967): Timed out: RequestConsensusVote RPC to 127.19.104.252:37967 timed out after 0.660s (SENT)
I20260812 06:36:29.674253 20669 heartbeater.cc:499] Master 127.19.104.252:37967 was elected leader, sending a full tablet report...
I20260812 06:36:30.009274 20701 raft_consensus.cc:493] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:30.009419 20701 raft_consensus.cc:515] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:30.009974 20701 leader_election.cc:290] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 05e04be5fbf84feca8531c53facf72c7 (127.19.104.254:43189), 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967)
I20260812 06:36:30.010512 19929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92de3a4448694c9b945dc0beeb24c1d5" candidate_term: 4 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "05e04be5fbf84feca8531c53facf72c7" is_pre_election: true
I20260812 06:36:30.010699 19929 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92de3a4448694c9b945dc0beeb24c1d5 in term 3.
I20260812 06:36:30.011076 19988 leader_election.cc:304] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05e04be5fbf84feca8531c53facf72c7, 92de3a4448694c9b945dc0beeb24c1d5; no voters: 
I20260812 06:36:30.011257 20701 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 06:36:30.011325 20701 raft_consensus.cc:493] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:36:30.011366 20701 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 3 FOLLOWER]: Advancing to term 4
I20260812 06:36:30.012290 20701 raft_consensus.cc:515] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:30.012739 20701 leader_election.cc:290] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [CANDIDATE]: Term 4 election: Requested vote from peers 05e04be5fbf84feca8531c53facf72c7 (127.19.104.254:43189), 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967)
I20260812 06:36:30.013020 19929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92de3a4448694c9b945dc0beeb24c1d5" candidate_term: 4 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "05e04be5fbf84feca8531c53facf72c7"
I20260812 06:36:30.013140 19929 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 3 FOLLOWER]: Advancing to term 4
I20260812 06:36:30.013968 19929 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92de3a4448694c9b945dc0beeb24c1d5 in term 4.
I20260812 06:36:30.014329 19988 leader_election.cc:304] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05e04be5fbf84feca8531c53facf72c7, 92de3a4448694c9b945dc0beeb24c1d5; no voters: 
I20260812 06:36:30.014510 20701 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 06:36:30.014765 20701 raft_consensus.cc:697] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [term 4 LEADER]: Becoming Leader. State: Replica: 92de3a4448694c9b945dc0beeb24c1d5, State: Running, Role: LEADER
I20260812 06:36:30.015085 20701 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 7, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:30.016168 20702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92de3a4448694c9b945dc0beeb24c1d5. Latest consensus state: current_term: 4 leader_uuid: "92de3a4448694c9b945dc0beeb24c1d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } } }
I20260812 06:36:30.016281 20702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [sys.catalog]: This master's current role is: LEADER
I20260812 06:36:30.016541 20705 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:36:30.018049 20705 catalog_manager.cc:679] Loaded metadata for table default.test_pause_after_create_table_issued [id=2001c2e8ab034601b45f6c3e7f333645]
I20260812 06:36:30.019052 20705 tablet_loader.cc:96] loaded metadata for tablet 73f545255ec1443199608669503f066f (table default.test_pause_after_create_table_issued [id=2001c2e8ab034601b45f6c3e7f333645])
I20260812 06:36:30.019219 20705 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:36:30.019601 20705 catalog_manager.cc:1295] Loaded cluster ID: c2f38e66602a4a4385d5be08edc0add3
I20260812 06:36:30.019670 20705 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:36:30.020197 20705 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:36:30.020641 20705 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5: Loaded TSK: 0
I20260812 06:36:30.021215 20705 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:36:30.494997 19929 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Refusing update from remote peer 92de3a4448694c9b945dc0beeb24c1d5: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 4 index: 8. (index mismatch)
I20260812 06:36:30.495647 20702 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 06:36:30.497136 20707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92de3a4448694c9b945dc0beeb24c1d5. Latest consensus state: current_term: 4 leader_uuid: "92de3a4448694c9b945dc0beeb24c1d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } } }
I20260812 06:36:30.497288 20707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:36:30.498581 20701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "92de3a4448694c9b945dc0beeb24c1d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } } }
I20260812 06:36:30.498709 20701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [sys.catalog]: This master's current role is: LEADER
I20260812 06:36:30.498857 20707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "92de3a4448694c9b945dc0beeb24c1d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } } }
I20260812 06:36:30.498999 20707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:36:30.679067 20671 heartbeater.cc:499] Master 127.19.104.253:35395 was elected leader, sending a full tablet report...
I20260812 06:36:30.766616 20353 heartbeater.cc:499] Master 127.19.104.253:35395 was elected leader, sending a full tablet report...
I20260812 06:36:30.970372 20507 heartbeater.cc:499] Master 127.19.104.253:35395 was elected leader, sending a full tablet report...
I20260812 06:36:30.999711 20714 raft_consensus.cc:493] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 92de3a4448694c9b945dc0beeb24c1d5)
I20260812 06:36:30.999840 20714 raft_consensus.cc:515] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:31.000263 20714 leader_election.cc:290] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 92de3a4448694c9b945dc0beeb24c1d5 (127.19.104.253:35395), 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967)
I20260812 06:36:31.000835 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "92de3a4448694c9b945dc0beeb24c1d5" is_pre_election: true
W20260812 06:36:31.160707 19989 leader_election.cc:336] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967): Timed out: RequestConsensusVote RPC to 127.19.104.252:37967 timed out after 0.572s (SENT)
I20260812 06:36:31.176424 20669 heartbeater.cc:499] Master 127.19.104.252:37967 was elected leader, sending a full tablet report...
W20260812 06:36:31.635336 19904 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.104.252:37967 timed out after 0.634s (SENT) [suppressed 7 similar messages]
W20260812 06:36:31.941270 19989 leader_election.cc:336] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967): Timed out: RequestConsensusVote RPC to 127.19.104.252:37967 timed out after 0.964s (SENT)
W20260812 06:36:32.271993 19904 leader_election.cc:336] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967): Timed out: RequestConsensusVote RPC to 127.19.104.252:37967 timed out after 0.634s (SENT)
I20260812 06:36:32.272163 19904 leader_election.cc:304] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05e04be5fbf84feca8531c53facf72c7; no voters: 751096c394f2450a8af13ab6b6586523, 92de3a4448694c9b945dc0beeb24c1d5
I20260812 06:36:32.272684 20718 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 06:36:32.678822 20669 heartbeater.cc:499] Master 127.19.104.252:37967 was elected leader, sending a full tablet report...
I20260812 06:36:33.394965 20720 raft_consensus.cc:493] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 92de3a4448694c9b945dc0beeb24c1d5)
I20260812 06:36:33.395112 20720 raft_consensus.cc:515] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:33.395658 20720 leader_election.cc:290] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 92de3a4448694c9b945dc0beeb24c1d5 (127.19.104.253:35395), 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967)
I20260812 06:36:33.396209 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "92de3a4448694c9b945dc0beeb24c1d5" is_pre_election: true
I20260812 06:36:34.181100 20669 heartbeater.cc:499] Master 127.19.104.252:37967 was elected leader, sending a full tablet report...
W20260812 06:36:34.615793 19904 leader_election.cc:336] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967): Timed out: RequestConsensusVote RPC to 127.19.104.252:37967 timed out after 0.608s (SENT)
I20260812 06:36:34.615912 19904 leader_election.cc:304] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05e04be5fbf84feca8531c53facf72c7; no voters: 751096c394f2450a8af13ab6b6586523, 92de3a4448694c9b945dc0beeb24c1d5
I20260812 06:36:34.616312 20724 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 06:36:35.566138 20727 raft_consensus.cc:493] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 92de3a4448694c9b945dc0beeb24c1d5)
I20260812 06:36:35.566294 20727 raft_consensus.cc:515] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:35.566823 20727 leader_election.cc:290] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 92de3a4448694c9b945dc0beeb24c1d5 (127.19.104.253:35395), 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967)
I20260812 06:36:35.567303 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "92de3a4448694c9b945dc0beeb24c1d5" is_pre_election: true
I20260812 06:36:35.683037 20669 heartbeater.cc:499] Master 127.19.104.252:37967 was elected leader, sending a full tablet report...
W20260812 06:36:36.956840 19904 leader_election.cc:336] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967): Timed out: RequestConsensusVote RPC to 127.19.104.252:37967 timed out after 0.693s (SENT)
I20260812 06:36:36.957029 19904 leader_election.cc:304] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05e04be5fbf84feca8531c53facf72c7; no voters: 751096c394f2450a8af13ab6b6586523, 92de3a4448694c9b945dc0beeb24c1d5
I20260812 06:36:36.957455 20732 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 06:36:37.185364 20669 heartbeater.cc:499] Master 127.19.104.252:37967 was elected leader, sending a full tablet report...
I20260812 06:36:37.489728 20733 raft_consensus.cc:493] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 92de3a4448694c9b945dc0beeb24c1d5)
I20260812 06:36:37.489971 20733 raft_consensus.cc:515] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:37.490864 20733 leader_election.cc:290] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 92de3a4448694c9b945dc0beeb24c1d5 (127.19.104.253:35395), 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967)
I20260812 06:36:37.491730 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "92de3a4448694c9b945dc0beeb24c1d5" is_pre_election: true
W20260812 06:36:37.724431 20682 master_proxy_rpc.cc:203] Re-attempting IsCreateTableDone request to leader Master (127.19.104.252:37967)
W20260812 06:36:38.049412 19904 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.104.252:37967 timed out after 0.557s (SENT) [suppressed 2 similar messages]
W20260812 06:36:38.608376 19904 leader_election.cc:336] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967): Timed out: RequestConsensusVote RPC to 127.19.104.252:37967 timed out after 0.557s (SENT)
I20260812 06:36:38.608744 19904 leader_election.cc:304] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05e04be5fbf84feca8531c53facf72c7; no voters: 751096c394f2450a8af13ab6b6586523, 92de3a4448694c9b945dc0beeb24c1d5
I20260812 06:36:38.609750 20739 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 06:36:38.687939 20669 heartbeater.cc:499] Master 127.19.104.252:37967 was elected leader, sending a full tablet report...
I20260812 06:36:40.189419 20669 heartbeater.cc:499] Master 127.19.104.252:37967 was elected leader, sending a full tablet report...
I20260812 06:36:40.432204 20740 raft_consensus.cc:493] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 92de3a4448694c9b945dc0beeb24c1d5)
I20260812 06:36:40.432399 20740 raft_consensus.cc:515] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:40.432917 20740 leader_election.cc:290] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 92de3a4448694c9b945dc0beeb24c1d5 (127.19.104.253:35395), 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967)
I20260812 06:36:40.433568 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "92de3a4448694c9b945dc0beeb24c1d5" is_pre_election: true
W20260812 06:36:41.538188 19904 leader_election.cc:336] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967): Timed out: RequestConsensusVote RPC to 127.19.104.252:37967 timed out after 0.551s (SENT)
I20260812 06:36:41.538368 19904 leader_election.cc:304] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05e04be5fbf84feca8531c53facf72c7; no voters: 751096c394f2450a8af13ab6b6586523, 92de3a4448694c9b945dc0beeb24c1d5
I20260812 06:36:41.538772 20745 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 06:36:41.691421 20669 heartbeater.cc:499] Master 127.19.104.252:37967 was elected leader, sending a full tablet report...
I20260812 06:36:42.199746 20747 raft_consensus.cc:493] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 92de3a4448694c9b945dc0beeb24c1d5)
I20260812 06:36:42.199890 20747 raft_consensus.cc:515] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:42.200489 20747 leader_election.cc:290] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 92de3a4448694c9b945dc0beeb24c1d5 (127.19.104.253:35395), 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967)
I20260812 06:36:42.201104 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "92de3a4448694c9b945dc0beeb24c1d5" is_pre_election: true
W20260812 06:36:43.383489 19904 leader_election.cc:336] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967): Timed out: RequestConsensusVote RPC to 127.19.104.252:37967 timed out after 0.590s (SENT)
I20260812 06:36:43.383607 19904 leader_election.cc:304] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05e04be5fbf84feca8531c53facf72c7; no voters: 751096c394f2450a8af13ab6b6586523, 92de3a4448694c9b945dc0beeb24c1d5
I20260812 06:36:43.383996 20753 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 06:36:45.208946 20755 raft_consensus.cc:493] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 92de3a4448694c9b945dc0beeb24c1d5)
I20260812 06:36:45.209101 20755 raft_consensus.cc:515] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:45.209560 20755 leader_election.cc:290] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 92de3a4448694c9b945dc0beeb24c1d5 (127.19.104.253:35395), 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967)
I20260812 06:36:45.210145 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "92de3a4448694c9b945dc0beeb24c1d5" is_pre_election: true
W20260812 06:36:45.758972 19904 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.104.252:37967 timed out after 0.548s (SENT) [suppressed 2 similar messages]
W20260812 06:36:46.308665 19904 leader_election.cc:336] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967): Timed out: RequestConsensusVote RPC to 127.19.104.252:37967 timed out after 0.548s (SENT)
I20260812 06:36:46.308945 19904 leader_election.cc:304] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05e04be5fbf84feca8531c53facf72c7; no voters: 751096c394f2450a8af13ab6b6586523, 92de3a4448694c9b945dc0beeb24c1d5
I20260812 06:36:46.309404 20759 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 06:36:48.159971 20761 raft_consensus.cc:493] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 92de3a4448694c9b945dc0beeb24c1d5)
I20260812 06:36:48.160132 20761 raft_consensus.cc:515] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:48.160698 20761 leader_election.cc:290] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 92de3a4448694c9b945dc0beeb24c1d5 (127.19.104.253:35395), 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967)
I20260812 06:36:48.161190 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "92de3a4448694c9b945dc0beeb24c1d5" is_pre_election: true
W20260812 06:36:49.499614 19904 leader_election.cc:336] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967): Timed out: RequestConsensusVote RPC to 127.19.104.252:37967 timed out after 0.668s (SENT)
I20260812 06:36:49.499750 19904 leader_election.cc:304] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05e04be5fbf84feca8531c53facf72c7; no voters: 751096c394f2450a8af13ab6b6586523, 92de3a4448694c9b945dc0beeb24c1d5
I20260812 06:36:49.500209 20765 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 06:36:50.179392 20766 raft_consensus.cc:493] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 92de3a4448694c9b945dc0beeb24c1d5)
I20260812 06:36:50.179584 20766 raft_consensus.cc:515] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:50.180086 20766 leader_election.cc:290] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 92de3a4448694c9b945dc0beeb24c1d5 (127.19.104.253:35395), 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967)
I20260812 06:36:50.180657 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "92de3a4448694c9b945dc0beeb24c1d5" is_pre_election: true
W20260812 06:36:51.273698 19904 leader_election.cc:336] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967): Timed out: RequestConsensusVote RPC to 127.19.104.252:37967 timed out after 0.545s (SENT)
I20260812 06:36:51.273829 19904 leader_election.cc:304] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05e04be5fbf84feca8531c53facf72c7; no voters: 751096c394f2450a8af13ab6b6586523, 92de3a4448694c9b945dc0beeb24c1d5
I20260812 06:36:51.274251 20771 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 06:36:53.845875 20773 raft_consensus.cc:493] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 92de3a4448694c9b945dc0beeb24c1d5)
I20260812 06:36:53.846071 20773 raft_consensus.cc:515] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:53.846642 20773 leader_election.cc:290] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 92de3a4448694c9b945dc0beeb24c1d5 (127.19.104.253:35395), 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967)
I20260812 06:36:53.847023 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "92de3a4448694c9b945dc0beeb24c1d5" is_pre_election: true
W20260812 06:36:54.441586 19904 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.104.252:37967 timed out after 0.593s (SENT) [suppressed 2 similar messages]
W20260812 06:36:55.036938 19904 leader_election.cc:336] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967): Timed out: RequestConsensusVote RPC to 127.19.104.252:37967 timed out after 0.593s (SENT)
I20260812 06:36:55.037047 19904 leader_election.cc:304] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05e04be5fbf84feca8531c53facf72c7; no voters: 751096c394f2450a8af13ab6b6586523, 92de3a4448694c9b945dc0beeb24c1d5
I20260812 06:36:55.037376 20778 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 06:36:56.703065 20780 raft_consensus.cc:493] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 92de3a4448694c9b945dc0beeb24c1d5)
I20260812 06:36:56.703217 20780 raft_consensus.cc:515] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:56.703647 20780 leader_election.cc:290] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 92de3a4448694c9b945dc0beeb24c1d5 (127.19.104.253:35395), 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967)
I20260812 06:36:56.704228 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "92de3a4448694c9b945dc0beeb24c1d5" is_pre_election: true
W20260812 06:36:58.155320 19904 leader_election.cc:336] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967): Timed out: RequestConsensusVote RPC to 127.19.104.252:37967 timed out after 0.724s (SENT)
I20260812 06:36:58.155447 19904 leader_election.cc:304] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05e04be5fbf84feca8531c53facf72c7; no voters: 751096c394f2450a8af13ab6b6586523, 92de3a4448694c9b945dc0beeb24c1d5
I20260812 06:36:58.155896 20783 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 06:36:59.543614 20787 raft_consensus.cc:493] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 92de3a4448694c9b945dc0beeb24c1d5)
I20260812 06:36:59.543768 20787 raft_consensus.cc:515] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:36:59.544260 20787 leader_election.cc:290] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 92de3a4448694c9b945dc0beeb24c1d5 (127.19.104.253:35395), 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967)
I20260812 06:36:59.544894 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "92de3a4448694c9b945dc0beeb24c1d5" is_pre_election: true
W20260812 06:37:00.246110 19904 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.104.252:37967 timed out after 0.701s (SENT) [suppressed 1 similar messages]
W20260812 06:37:00.499913 19989 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: UpdateConsensus RPC to 127.19.104.252:37967 timed out after 30.000s (SENT) [suppressed 6 similar messages]
W20260812 06:37:00.828970 20167 catalog_manager.cc:6182] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5: Tablet 73f545255ec1443199608669503f066f (table default.test_pause_after_create_table_issued [id=2001c2e8ab034601b45f6c3e7f333645]) was not created within the allowed timeout. Replacing with a new tablet f4feaa265d594b1d8bafd20fdc7fe987
I20260812 06:37:00.835467 20167 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5: Sending DeleteTablet for 3 replicas of tablet 73f545255ec1443199608669503f066f
I20260812 06:37:00.842608 20276 tablet_service.cc:1558] Processing DeleteTablet for tablet 73f545255ec1443199608669503f066f with delete_type TABLET_DATA_DELETED (Replaced by f4feaa265d594b1d8bafd20fdc7fe987 at 2026-08-12 06:37:00 UTC) from {username='slave'} at 127.0.0.1:39478
I20260812 06:37:00.843405 20590 tablet_service.cc:1558] Processing DeleteTablet for tablet 73f545255ec1443199608669503f066f with delete_type TABLET_DATA_DELETED (Replaced by f4feaa265d594b1d8bafd20fdc7fe987 at 2026-08-12 06:37:00 UTC) from {username='slave'} at 127.0.0.1:39742
I20260812 06:37:00.844244 20431 tablet_service.cc:1558] Processing DeleteTablet for tablet 73f545255ec1443199608669503f066f with delete_type TABLET_DATA_DELETED (Replaced by f4feaa265d594b1d8bafd20fdc7fe987 at 2026-08-12 06:37:00 UTC) from {username='slave'} at 127.0.0.1:51802
I20260812 06:37:00.844702 20275 tablet_service.cc:1511] Processing CreateTablet for tablet f4feaa265d594b1d8bafd20fdc7fe987 (DEFAULT_TABLE table=default.test_pause_after_create_table_issued [id=2001c2e8ab034601b45f6c3e7f333645]), partition=RANGE (key) PARTITION UNBOUNDED
W20260812 06:37:00.845225 19989 catalog_manager.cc:5017] TS 69b835823c91478787fbf47bbf635642 (127.19.104.195:46055): delete failed for tablet 73f545255ec1443199608669503f066f because the tablet was not found. No further retry: Not found: Tablet not found: 73f545255ec1443199608669503f066f
I20260812 06:37:00.845268 20429 tablet_service.cc:1511] Processing CreateTablet for tablet f4feaa265d594b1d8bafd20fdc7fe987 (DEFAULT_TABLE table=default.test_pause_after_create_table_issued [id=2001c2e8ab034601b45f6c3e7f333645]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:37:00.845679 20275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4feaa265d594b1d8bafd20fdc7fe987. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:00.846155 20429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4feaa265d594b1d8bafd20fdc7fe987. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:00.846700 20589 tablet_service.cc:1511] Processing CreateTablet for tablet f4feaa265d594b1d8bafd20fdc7fe987 (DEFAULT_TABLE table=default.test_pause_after_create_table_issued [id=2001c2e8ab034601b45f6c3e7f333645]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:37:00.847488 20589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4feaa265d594b1d8bafd20fdc7fe987. 1 dirs total, 0 dirs full, 0 dirs failed
W20260812 06:37:00.849715 19987 catalog_manager.cc:5017] TS 40f95405712443ac98dc145f98dd3293 (127.19.104.194:39609): delete failed for tablet 73f545255ec1443199608669503f066f because the tablet was not found. No further retry: Not found: Tablet not found: 73f545255ec1443199608669503f066f
I20260812 06:37:00.850378 20811 tablet_bootstrap.cc:492] T f4feaa265d594b1d8bafd20fdc7fe987 P 52ce252555214626853d4de2f72c7b7c: Bootstrap starting.
W20260812 06:37:00.850509 19989 catalog_manager.cc:5017] TS 52ce252555214626853d4de2f72c7b7c (127.19.104.193:37403): delete failed for tablet 73f545255ec1443199608669503f066f because the tablet was not found. No further retry: Not found: Tablet not found: 73f545255ec1443199608669503f066f
I20260812 06:37:00.851263 20812 tablet_bootstrap.cc:492] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293: Bootstrap starting.
I20260812 06:37:00.852699 20811 tablet_bootstrap.cc:654] T f4feaa265d594b1d8bafd20fdc7fe987 P 52ce252555214626853d4de2f72c7b7c: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:00.853216 20815 tablet_bootstrap.cc:492] T f4feaa265d594b1d8bafd20fdc7fe987 P 69b835823c91478787fbf47bbf635642: Bootstrap starting.
I20260812 06:37:00.853343 20812 tablet_bootstrap.cc:654] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:00.853513 20811 log.cc:826] T f4feaa265d594b1d8bafd20fdc7fe987 P 52ce252555214626853d4de2f72c7b7c: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:00.854090 20812 log.cc:826] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:00.855243 20815 tablet_bootstrap.cc:654] T f4feaa265d594b1d8bafd20fdc7fe987 P 69b835823c91478787fbf47bbf635642: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:00.855260 20811 tablet_bootstrap.cc:492] T f4feaa265d594b1d8bafd20fdc7fe987 P 52ce252555214626853d4de2f72c7b7c: No bootstrap required, opened a new log
I20260812 06:37:00.855443 20812 tablet_bootstrap.cc:492] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293: No bootstrap required, opened a new log
I20260812 06:37:00.855458 20811 ts_tablet_manager.cc:1403] T f4feaa265d594b1d8bafd20fdc7fe987 P 52ce252555214626853d4de2f72c7b7c: Time spent bootstrapping tablet: real 0.005s	user 0.000s	sys 0.003s
I20260812 06:37:00.855633 20812 ts_tablet_manager.cc:1403] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.000s
I20260812 06:37:00.855989 20815 log.cc:826] T f4feaa265d594b1d8bafd20fdc7fe987 P 69b835823c91478787fbf47bbf635642: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:00.857625 20815 tablet_bootstrap.cc:492] T f4feaa265d594b1d8bafd20fdc7fe987 P 69b835823c91478787fbf47bbf635642: No bootstrap required, opened a new log
I20260812 06:37:00.857796 20815 ts_tablet_manager.cc:1403] T f4feaa265d594b1d8bafd20fdc7fe987 P 69b835823c91478787fbf47bbf635642: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.000s
I20260812 06:37:00.858642 20812 raft_consensus.cc:359] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69b835823c91478787fbf47bbf635642" member_type: VOTER last_known_addr { host: "127.19.104.195" port: 46055 } } peers { permanent_uuid: "52ce252555214626853d4de2f72c7b7c" member_type: VOTER last_known_addr { host: "127.19.104.193" port: 37403 } } peers { permanent_uuid: "40f95405712443ac98dc145f98dd3293" member_type: VOTER last_known_addr { host: "127.19.104.194" port: 39609 } }
I20260812 06:37:00.858834 20812 raft_consensus.cc:385] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:00.858901 20812 raft_consensus.cc:740] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40f95405712443ac98dc145f98dd3293, State: Initialized, Role: FOLLOWER
I20260812 06:37:00.859347 20812 consensus_queue.cc:260] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69b835823c91478787fbf47bbf635642" member_type: VOTER last_known_addr { host: "127.19.104.195" port: 46055 } } peers { permanent_uuid: "52ce252555214626853d4de2f72c7b7c" member_type: VOTER last_known_addr { host: "127.19.104.193" port: 37403 } } peers { permanent_uuid: "40f95405712443ac98dc145f98dd3293" member_type: VOTER last_known_addr { host: "127.19.104.194" port: 39609 } }
I20260812 06:37:00.859337 20811 raft_consensus.cc:359] T f4feaa265d594b1d8bafd20fdc7fe987 P 52ce252555214626853d4de2f72c7b7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69b835823c91478787fbf47bbf635642" member_type: VOTER last_known_addr { host: "127.19.104.195" port: 46055 } } peers { permanent_uuid: "52ce252555214626853d4de2f72c7b7c" member_type: VOTER last_known_addr { host: "127.19.104.193" port: 37403 } } peers { permanent_uuid: "40f95405712443ac98dc145f98dd3293" member_type: VOTER last_known_addr { host: "127.19.104.194" port: 39609 } }
I20260812 06:37:00.859579 20811 raft_consensus.cc:385] T f4feaa265d594b1d8bafd20fdc7fe987 P 52ce252555214626853d4de2f72c7b7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:00.859635 20811 raft_consensus.cc:740] T f4feaa265d594b1d8bafd20fdc7fe987 P 52ce252555214626853d4de2f72c7b7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ce252555214626853d4de2f72c7b7c, State: Initialized, Role: FOLLOWER
I20260812 06:37:00.860172 20812 ts_tablet_manager.cc:1434] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 06:37:00.860077 20811 consensus_queue.cc:260] T f4feaa265d594b1d8bafd20fdc7fe987 P 52ce252555214626853d4de2f72c7b7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69b835823c91478787fbf47bbf635642" member_type: VOTER last_known_addr { host: "127.19.104.195" port: 46055 } } peers { permanent_uuid: "52ce252555214626853d4de2f72c7b7c" member_type: VOTER last_known_addr { host: "127.19.104.193" port: 37403 } } peers { permanent_uuid: "40f95405712443ac98dc145f98dd3293" member_type: VOTER last_known_addr { host: "127.19.104.194" port: 39609 } }
I20260812 06:37:00.860950 20811 ts_tablet_manager.cc:1434] T f4feaa265d594b1d8bafd20fdc7fe987 P 52ce252555214626853d4de2f72c7b7c: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.002s
I20260812 06:37:00.861362 20815 raft_consensus.cc:359] T f4feaa265d594b1d8bafd20fdc7fe987 P 69b835823c91478787fbf47bbf635642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69b835823c91478787fbf47bbf635642" member_type: VOTER last_known_addr { host: "127.19.104.195" port: 46055 } } peers { permanent_uuid: "52ce252555214626853d4de2f72c7b7c" member_type: VOTER last_known_addr { host: "127.19.104.193" port: 37403 } } peers { permanent_uuid: "40f95405712443ac98dc145f98dd3293" member_type: VOTER last_known_addr { host: "127.19.104.194" port: 39609 } }
I20260812 06:37:00.861573 20815 raft_consensus.cc:385] T f4feaa265d594b1d8bafd20fdc7fe987 P 69b835823c91478787fbf47bbf635642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:00.861656 20815 raft_consensus.cc:740] T f4feaa265d594b1d8bafd20fdc7fe987 P 69b835823c91478787fbf47bbf635642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69b835823c91478787fbf47bbf635642, State: Initialized, Role: FOLLOWER
I20260812 06:37:00.862067 20815 consensus_queue.cc:260] T f4feaa265d594b1d8bafd20fdc7fe987 P 69b835823c91478787fbf47bbf635642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69b835823c91478787fbf47bbf635642" member_type: VOTER last_known_addr { host: "127.19.104.195" port: 46055 } } peers { permanent_uuid: "52ce252555214626853d4de2f72c7b7c" member_type: VOTER last_known_addr { host: "127.19.104.193" port: 37403 } } peers { permanent_uuid: "40f95405712443ac98dc145f98dd3293" member_type: VOTER last_known_addr { host: "127.19.104.194" port: 39609 } }
I20260812 06:37:00.862861 20815 ts_tablet_manager.cc:1434] T f4feaa265d594b1d8bafd20fdc7fe987 P 69b835823c91478787fbf47bbf635642: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
W20260812 06:37:00.936756 20672 tablet.cc:2406] T f4feaa265d594b1d8bafd20fdc7fe987 P 69b835823c91478787fbf47bbf635642: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:37:00.948832 19904 leader_election.cc:336] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967): Timed out: RequestConsensusVote RPC to 127.19.104.252:37967 timed out after 0.701s (SENT)
I20260812 06:37:00.948951 19904 leader_election.cc:304] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05e04be5fbf84feca8531c53facf72c7; no voters: 751096c394f2450a8af13ab6b6586523, 92de3a4448694c9b945dc0beeb24c1d5
I20260812 06:37:00.949405 20822 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260812 06:37:00.975577 20508 tablet.cc:2406] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:37:01.020398 20354 tablet.cc:2406] T f4feaa265d594b1d8bafd20fdc7fe987 P 52ce252555214626853d4de2f72c7b7c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:37:01.103092 20819 raft_consensus.cc:493] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:37:01.103351 20819 raft_consensus.cc:515] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69b835823c91478787fbf47bbf635642" member_type: VOTER last_known_addr { host: "127.19.104.195" port: 46055 } } peers { permanent_uuid: "52ce252555214626853d4de2f72c7b7c" member_type: VOTER last_known_addr { host: "127.19.104.193" port: 37403 } } peers { permanent_uuid: "40f95405712443ac98dc145f98dd3293" member_type: VOTER last_known_addr { host: "127.19.104.194" port: 39609 } }
I20260812 06:37:01.104601 20819 leader_election.cc:290] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69b835823c91478787fbf47bbf635642 (127.19.104.195:46055), 52ce252555214626853d4de2f72c7b7c (127.19.104.193:37403)
I20260812 06:37:01.109091 20617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4feaa265d594b1d8bafd20fdc7fe987" candidate_uuid: "40f95405712443ac98dc145f98dd3293" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69b835823c91478787fbf47bbf635642" is_pre_election: true
I20260812 06:37:01.109385 20617 raft_consensus.cc:2468] T f4feaa265d594b1d8bafd20fdc7fe987 P 69b835823c91478787fbf47bbf635642 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40f95405712443ac98dc145f98dd3293 in term 0.
I20260812 06:37:01.109368 20300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4feaa265d594b1d8bafd20fdc7fe987" candidate_uuid: "40f95405712443ac98dc145f98dd3293" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ce252555214626853d4de2f72c7b7c" is_pre_election: true
I20260812 06:37:01.109620 20300 raft_consensus.cc:2468] T f4feaa265d594b1d8bafd20fdc7fe987 P 52ce252555214626853d4de2f72c7b7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40f95405712443ac98dc145f98dd3293 in term 0.
I20260812 06:37:01.109946 20380 leader_election.cc:304] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40f95405712443ac98dc145f98dd3293, 69b835823c91478787fbf47bbf635642; no voters: 
I20260812 06:37:01.110241 20819 raft_consensus.cc:2804] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:37:01.110342 20819 raft_consensus.cc:493] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:37:01.110406 20819 raft_consensus.cc:3060] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:01.111768 20819 raft_consensus.cc:515] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69b835823c91478787fbf47bbf635642" member_type: VOTER last_known_addr { host: "127.19.104.195" port: 46055 } } peers { permanent_uuid: "52ce252555214626853d4de2f72c7b7c" member_type: VOTER last_known_addr { host: "127.19.104.193" port: 37403 } } peers { permanent_uuid: "40f95405712443ac98dc145f98dd3293" member_type: VOTER last_known_addr { host: "127.19.104.194" port: 39609 } }
I20260812 06:37:01.112188 20819 leader_election.cc:290] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293 [CANDIDATE]: Term 1 election: Requested vote from peers 69b835823c91478787fbf47bbf635642 (127.19.104.195:46055), 52ce252555214626853d4de2f72c7b7c (127.19.104.193:37403)
I20260812 06:37:01.112736 20617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4feaa265d594b1d8bafd20fdc7fe987" candidate_uuid: "40f95405712443ac98dc145f98dd3293" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69b835823c91478787fbf47bbf635642"
I20260812 06:37:01.112881 20617 raft_consensus.cc:3060] T f4feaa265d594b1d8bafd20fdc7fe987 P 69b835823c91478787fbf47bbf635642 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:01.113574 20300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4feaa265d594b1d8bafd20fdc7fe987" candidate_uuid: "40f95405712443ac98dc145f98dd3293" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ce252555214626853d4de2f72c7b7c"
I20260812 06:37:01.113694 20300 raft_consensus.cc:3060] T f4feaa265d594b1d8bafd20fdc7fe987 P 52ce252555214626853d4de2f72c7b7c [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:01.113982 20617 raft_consensus.cc:2468] T f4feaa265d594b1d8bafd20fdc7fe987 P 69b835823c91478787fbf47bbf635642 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40f95405712443ac98dc145f98dd3293 in term 1.
I20260812 06:37:01.114293 20380 leader_election.cc:304] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40f95405712443ac98dc145f98dd3293, 69b835823c91478787fbf47bbf635642; no voters: 
I20260812 06:37:01.114454 20819 raft_consensus.cc:2804] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:37:01.114734 20819 raft_consensus.cc:697] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293 [term 1 LEADER]: Becoming Leader. State: Replica: 40f95405712443ac98dc145f98dd3293, State: Running, Role: LEADER
I20260812 06:37:01.114758 20300 raft_consensus.cc:2468] T f4feaa265d594b1d8bafd20fdc7fe987 P 52ce252555214626853d4de2f72c7b7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40f95405712443ac98dc145f98dd3293 in term 1.
I20260812 06:37:01.115226 20819 consensus_queue.cc:237] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69b835823c91478787fbf47bbf635642" member_type: VOTER last_known_addr { host: "127.19.104.195" port: 46055 } } peers { permanent_uuid: "52ce252555214626853d4de2f72c7b7c" member_type: VOTER last_known_addr { host: "127.19.104.193" port: 37403 } } peers { permanent_uuid: "40f95405712443ac98dc145f98dd3293" member_type: VOTER last_known_addr { host: "127.19.104.194" port: 39609 } }
I20260812 06:37:01.118690 20004 catalog_manager.cc:5719] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293 reported cstate change: term changed from 0 to 1, leader changed from <none> to 40f95405712443ac98dc145f98dd3293 (127.19.104.194). New cstate: current_term: 1 leader_uuid: "40f95405712443ac98dc145f98dd3293" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69b835823c91478787fbf47bbf635642" member_type: VOTER last_known_addr { host: "127.19.104.195" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ce252555214626853d4de2f72c7b7c" member_type: VOTER last_known_addr { host: "127.19.104.193" port: 37403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40f95405712443ac98dc145f98dd3293" member_type: VOTER last_known_addr { host: "127.19.104.194" port: 39609 } health_report { overall_health: HEALTHY } } }
I20260812 06:37:01.516064 20819 consensus_queue.cc:1048] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ce252555214626853d4de2f72c7b7c" member_type: VOTER last_known_addr { host: "127.19.104.193" port: 37403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:01.521956 20833 consensus_queue.cc:1048] T f4feaa265d594b1d8bafd20fdc7fe987 P 40f95405712443ac98dc145f98dd3293 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69b835823c91478787fbf47bbf635642" member_type: VOTER last_known_addr { host: "127.19.104.195" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:01.623399 20839 raft_consensus.cc:493] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 92de3a4448694c9b945dc0beeb24c1d5)
I20260812 06:37:01.623562 20839 raft_consensus.cc:515] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:37:01.624138 20839 leader_election.cc:290] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 92de3a4448694c9b945dc0beeb24c1d5 (127.19.104.253:35395), 751096c394f2450a8af13ab6b6586523 (127.19.104.252:37967)
I20260812 06:37:01.624778 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: false dest_uuid: "92de3a4448694c9b945dc0beeb24c1d5" is_pre_election: true
I20260812 06:37:01.829634 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 20198
I20260812 06:37:01.831413 20097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.831820 20097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.832016 20097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.832206 20097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.832366 20097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.832644 20095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523"
I20260812 06:37:01.832820 20095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.832971 20095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.833139 20095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.833294 20095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.833441 20095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.833601 20095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.833776 20095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.833918 20095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.834062 20095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.834205 20095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.834340 20095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.834465 20095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.834623 20095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.834769 20095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.835491 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.840847 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
W20260812 06:37:01.840903 20069 connection.cc:256] RPC call timeout handler was delayed by -7.1307137509575114s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 3s for a response
I20260812 06:37:01.841172 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.844919 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92de3a4448694c9b945dc0beeb24c1d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.845196 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92de3a4448694c9b945dc0beeb24c1d5" candidate_term: 4 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523"
I20260812 06:37:01.845389 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92de3a4448694c9b945dc0beeb24c1d5" candidate_term: 4 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523"
I20260812 06:37:01.845572 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92de3a4448694c9b945dc0beeb24c1d5" candidate_term: 4 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.845774 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92de3a4448694c9b945dc0beeb24c1d5" candidate_term: 4 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.846051 20088 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 06:37:01.846135 20088 raft_consensus.cc:740] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 751096c394f2450a8af13ab6b6586523, State: Running, Role: LEADER
I20260812 06:37:01.846333 20088 consensus_queue.cc:260] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 7, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } }
I20260812 06:37:01.846536 20094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523"
I20260812 06:37:01.846642 20088 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [term 2 FOLLOWER]: Advancing to term 4
I20260812 06:37:01.846860 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.847028 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.847182 20092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.847323 20093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.848233 20088 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [term 4 FOLLOWER]: Refusing update from remote peer 92de3a4448694c9b945dc0beeb24c1d5: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 4 index: 8. (index mismatch)
I20260812 06:37:01.848820 19904 leader_election.cc:304] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05e04be5fbf84feca8531c53facf72c7; no voters: 751096c394f2450a8af13ab6b6586523, 92de3a4448694c9b945dc0beeb24c1d5
I20260812 06:37:01.848886 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92de3a4448694c9b945dc0beeb24c1d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523"
I20260812 06:37:01.849012 20089 raft_consensus.cc:2368] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 92de3a4448694c9b945dc0beeb24c1d5 for earlier term 3. Current term is 4.
I20260812 06:37:01.849597 20839 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 05e04be5fbf84feca8531c53facf72c7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 06:37:01.851321 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92de3a4448694c9b945dc0beeb24c1d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.851442 20090 raft_consensus.cc:2368] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92de3a4448694c9b945dc0beeb24c1d5 for earlier term 3. Current term is 4.
I20260812 06:37:01.853318 20094 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [term 4 FOLLOWER]: Refusing update from remote peer 92de3a4448694c9b945dc0beeb24c1d5: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 4 index: 8. (index mismatch)
I20260812 06:37:01.854214 20097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05e04be5fbf84feca8531c53facf72c7" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523" is_pre_election: true
I20260812 06:37:01.854312 20097 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05e04be5fbf84feca8531c53facf72c7 in term 4.
I20260812 06:37:01.854389 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 20359
W20260812 06:37:01.854609 20067 connection.cc:769] server connection from 127.19.104.193:58575 send error: Network error: sendmsg error: Broken pipe (error 32)
W20260812 06:37:01.854691 20067 connection.cc:466] server connection from 127.19.104.193:58575 torn down before Call kudu.master.MasterService.TSHeartbeat from 127.19.104.193:58575 (request call id 11) could send its response
W20260812 06:37:01.854820 20067 connection.cc:466] server connection from 127.19.104.193:58575 torn down before Call kudu.master.MasterService.Ping from 127.19.104.193:58575 (request call id 12) could send its response
W20260812 06:37:01.854894 20067 connection.cc:466] server connection from 127.19.104.193:58575 torn down before Call kudu.master.MasterService.Ping from 127.19.104.193:58575 (request call id 13) could send its response
W20260812 06:37:01.854955 20067 connection.cc:466] server connection from 127.19.104.193:58575 torn down before Call kudu.master.MasterService.Ping from 127.19.104.193:58575 (request call id 14) could send its response
W20260812 06:37:01.855008 20067 connection.cc:466] server connection from 127.19.104.193:58575 torn down before Call kudu.master.MasterService.Ping from 127.19.104.193:58575 (request call id 15) could send its response
W20260812 06:37:01.855069 20067 connection.cc:466] server connection from 127.19.104.193:58575 torn down before Call kudu.master.MasterService.Ping from 127.19.104.193:58575 (request call id 16) could send its response
W20260812 06:37:01.855123 20067 connection.cc:466] server connection from 127.19.104.193:58575 torn down before Call kudu.master.MasterService.Ping from 127.19.104.193:58575 (request call id 17) could send its response
W20260812 06:37:01.855178 20067 connection.cc:466] server connection from 127.19.104.193:58575 torn down before Call kudu.master.MasterService.Ping from 127.19.104.193:58575 (request call id 18) could send its response
W20260812 06:37:01.855234 20067 connection.cc:466] server connection from 127.19.104.193:58575 torn down before Call kudu.master.MasterService.Ping from 127.19.104.193:58575 (request call id 19) could send its response
W20260812 06:37:01.855289 20067 connection.cc:466] server connection from 127.19.104.193:58575 torn down before Call kudu.master.MasterService.Ping from 127.19.104.193:58575 (request call id 20) could send its response
W20260812 06:37:01.855345 20067 connection.cc:466] server connection from 127.19.104.193:58575 torn down before Call kudu.master.MasterService.Ping from 127.19.104.193:58575 (request call id 21) could send its response
W20260812 06:37:01.855398 20067 connection.cc:466] server connection from 127.19.104.193:58575 torn down before Call kudu.master.MasterService.Ping from 127.19.104.193:58575 (request call id 22) could send its response
W20260812 06:37:01.855451 20067 connection.cc:466] server connection from 127.19.104.193:58575 torn down before Call kudu.master.MasterService.Ping from 127.19.104.193:58575 (request call id 23) could send its response
W20260812 06:37:01.855520 20067 connection.cc:466] server connection from 127.19.104.193:58575 torn down before Call kudu.master.MasterService.Ping from 127.19.104.193:58575 (request call id 24) could send its response
W20260812 06:37:01.855576 20067 connection.cc:466] server connection from 127.19.104.193:58575 torn down before Call kudu.master.MasterService.Ping from 127.19.104.193:58575 (request call id 25) could send its response
W20260812 06:37:01.855638 20067 connection.cc:466] server connection from 127.19.104.193:58575 torn down before Call kudu.master.MasterService.TSHeartbeat from 127.19.104.193:58575 (request call id 3) could send its response
W20260812 06:37:01.855723 20067 connection.cc:466] server connection from 127.19.104.193:58575 torn down before Call kudu.master.MasterService.TSHeartbeat from 127.19.104.193:58575 (request call id 4) could send its response
W20260812 06:37:01.855814 20067 connection.cc:466] server connection from 127.19.104.193:58575 torn down before Call kudu.master.MasterService.TSHeartbeat from 127.19.104.193:58575 (request call id 5) could send its response
W20260812 06:37:01.855907 20067 connection.cc:466] server connection from 127.19.104.193:58575 torn down before Call kudu.master.MasterService.TSHeartbeat from 127.19.104.193:58575 (request call id 6) could send its response
W20260812 06:37:01.855996 20067 connection.cc:466] server connection from 127.19.104.193:58575 torn down before Call kudu.master.MasterService.TSHeartbeat from 127.19.104.193:58575 (request call id 7) could send its response
W20260812 06:37:01.856089 20067 connection.cc:466] server connection from 127.19.104.193:58575 torn down before Call kudu.master.MasterService.TSHeartbeat from 127.19.104.193:58575 (request call id 8) could send its response
W20260812 06:37:01.856168 20067 connection.cc:466] server connection from 127.19.104.193:58575 torn down before Call kudu.master.MasterService.TSHeartbeat from 127.19.104.193:58575 (request call id 9) could send its response
W20260812 06:37:01.856240 20067 connection.cc:466] server connection from 127.19.104.193:58575 torn down before Call kudu.master.MasterService.TSHeartbeat from 127.19.104.193:58575 (request call id 10) could send its response
W20260812 06:37:01.858760 20851 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:01.844070 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.19.104.193:37403 (local address 127.0.0.1:39484)
0812 06:37:01.858113 (+ 14043us) negotiation.cc:107] Waiting for socket to connect
0812 06:37:01.858178 (+    65us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.104.193:37403: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":13961,"thread_start_us":65,"threads_started":1}
I20260812 06:37:01.859220 20095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92de3a4448694c9b945dc0beeb24c1d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "751096c394f2450a8af13ab6b6586523"
I20260812 06:37:01.859349 20095 raft_consensus.cc:2368] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 92de3a4448694c9b945dc0beeb24c1d5 for earlier term 3. Current term is 4.
W20260812 06:37:01.861050 20070 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.104.193:37403: connect: Connection reset by peer (error 104)
I20260812 06:37:01.861217 20840 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 92de3a4448694c9b945dc0beeb24c1d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 06:37:01.864249 20589 tablet_service.cc:1511] Processing CreateTablet for tablet 73f545255ec1443199608669503f066f (DEFAULT_TABLE table=default.test_pause_after_create_table_issued [id=2001c2e8ab034601b45f6c3e7f333645]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:37:01.864610 20589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73f545255ec1443199608669503f066f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260812 06:37:01.864876 20070 catalog_manager.cc:4777] TS 52ce252555214626853d4de2f72c7b7c (127.19.104.193:37403): CreateTablet RPC failed for tablet 73f545255ec1443199608669503f066f: Network error: Client connection negotiation failed: client connection to 127.19.104.193:37403: connect: Connection refused (error 111)
I20260812 06:37:01.865724 20669 heartbeater.cc:344] Connected to a master server at 127.19.104.252:37967
I20260812 06:37:01.865844 20669 heartbeater.cc:499] Master 127.19.104.252:37967 was elected leader, sending a full tablet report...
I20260812 06:37:01.867283 20848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92de3a4448694c9b945dc0beeb24c1d5. Latest consensus state: current_term: 4 leader_uuid: "92de3a4448694c9b945dc0beeb24c1d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } } }
I20260812 06:37:01.867391 20848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:01.867498 20848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "92de3a4448694c9b945dc0beeb24c1d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05e04be5fbf84feca8531c53facf72c7" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43189 } } peers { permanent_uuid: "92de3a4448694c9b945dc0beeb24c1d5" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 35395 } } peers { permanent_uuid: "751096c394f2450a8af13ab6b6586523" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 37967 } } }
I20260812 06:37:01.867570 20848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 751096c394f2450a8af13ab6b6586523 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:01.868363 20856 tablet_bootstrap.cc:492] T 73f545255ec1443199608669503f066f P 69b835823c91478787fbf47bbf635642: Bootstrap starting.
I20260812 06:37:01.869338 20856 tablet_bootstrap.cc:654] T 73f545255ec1443199608669503f066f P 69b835823c91478787fbf47bbf635642: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:01.870494 20856 tablet_bootstrap.cc:492] T 73f545255ec1443199608669503f066f P 69b835823c91478787fbf47bbf635642: No bootstrap required, opened a new log
I20260812 06:37:01.870608 20856 ts_tablet_manager.cc:1403] T 73f545255ec1443199608669503f066f P 69b835823c91478787fbf47bbf635642: Time spent bootstrapping tablet: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:01.871119 20856 raft_consensus.cc:359] T 73f545255ec1443199608669503f066f P 69b835823c91478787fbf47bbf635642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ce252555214626853d4de2f72c7b7c" member_type: VOTER last_known_addr { host: "127.19.104.193" port: 37403 } } peers { permanent_uuid: "40f95405712443ac98dc145f98dd3293" member_type: VOTER last_known_addr { host: "127.19.104.194" port: 39609 } } peers { permanent_uuid: "69b835823c91478787fbf47bbf635642" member_type: VOTER last_known_addr { host: "127.19.104.195" port: 46055 } }
I20260812 06:37:01.871230 20856 raft_consensus.cc:385] T 73f545255ec1443199608669503f066f P 69b835823c91478787fbf47bbf635642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:01.871335 20856 raft_consensus.cc:740] T 73f545255ec1443199608669503f066f P 69b835823c91478787fbf47bbf635642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69b835823c91478787fbf47bbf635642, State: Initialized, Role: FOLLOWER
I20260812 06:37:01.871515 20856 consensus_queue.cc:260] T 73f545255ec1443199608669503f066f P 69b835823c91478787fbf47bbf635642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ce252555214626853d4de2f72c7b7c" member_type: VOTER last_known_addr { host: "127.19.104.193" port: 37403 } } peers { permanent_uuid: "40f95405712443ac98dc145f98dd3293" member_type: VOTER last_known_addr { host: "127.19.104.194" port: 39609 } } peers { permanent_uuid: "69b835823c91478787fbf47bbf635642" member_type: VOTER last_known_addr { host: "127.19.104.195" port: 46055 } }
I20260812 06:37:01.871835 20856 ts_tablet_manager.cc:1434] T 73f545255ec1443199608669503f066f P 69b835823c91478787fbf47bbf635642: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
W20260812 06:37:01.873771 20850 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:01.843017 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.19.104.194:39609 (local address 127.0.0.1:51816)
0812 06:37:01.856644 (+ 13627us) negotiation.cc:107] Waiting for socket to connect
0812 06:37:01.856662 (+    18us) client_negotiation.cc:175] Beginning negotiation
0812 06:37:01.856802 (+   140us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:37:01.873665 (+ 16863us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.104.194:39609: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":13515,"thread_start_us":288,"threads_started":1}
W20260812 06:37:01.874344 20069 connection.cc:573] server connection from 127.19.104.194:57403 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 06:37:01.874585 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 20515
W20260812 06:37:01.887284 19989 connection.cc:573] server connection from 127.19.104.195:36449 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 06:37:01.887284 20068 connection.cc:573] server connection from 127.19.104.195:44685 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 06:37:01.887537 19902 connection.cc:573] server connection from 127.19.104.195:44731 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 06:37:01.887544 19989 connection.cc:573] client connection to 127.19.104.195:46055 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 06:37:01.887918 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 19884
W20260812 06:37:01.893112 19989 catalog_manager.cc:4777] TS 69b835823c91478787fbf47bbf635642 (127.19.104.195:46055): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 73f545255ec1443199608669503f066f: Network error: Client connection negotiation failed: client connection to 127.19.104.195:46055: connect: Connection refused (error 111)
I20260812 06:37:01.905112 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 19963
I20260812 06:37:01.918061 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 20046
2026-08-12T06:37:01Z chronyd exiting
[       OK ] HmsConfigurations/MasterFailoverTest.TestPauseAfterCreateTableIssued/0 (35772 ms)
[ RUN      ] HmsConfigurations/MasterFailoverTest.TestRenameTableSync/0
2026-08-12T06:37:01Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:37:01Z Disabled control of system clock
I20260812 06:37:01.954200 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.254:43859
--webserver_interface=127.19.104.254
--webserver_port=0
--builtin_ntp_servers=127.19.104.212:43903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.104.254:43859,127.19.104.253:44447,127.19.104.252:43887
--leader_failure_max_missed_heartbeat_periods=1.0 with env {}
W20260812 06:37:02.058288 20864 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:02.058516 20864 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:02.058553 20864 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:02.061975 20864 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:37:02.062041 20864 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:02.062062 20864 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:37:02.062080 20864 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:37:02.065940 20864 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:43903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.104.254:43859,127.19.104.253:44447,127.19.104.252:43887
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.254:43859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.104.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.20864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:02.067016 20864 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:02.068076 20864 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:02.073995 20871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:02.074000 20872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:02.074113 20874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:02.074421 20864 server_base.cc:1061] running on GCE node
I20260812 06:37:02.074976 20864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:02.075914 20864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:02.077070 20864 hybrid_clock.cc:648] HybridClock initialized: now 1786516622077054 us; error 32 us; skew 500 ppm
I20260812 06:37:02.079154 20864 webserver.cc:533] Webserver started at http://127.19.104.254:39931/ using document root <none> and password file <none>
I20260812 06:37:02.079777 20864 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:02.079836 20864 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:02.080055 20864 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:02.081686 20864 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-0/data/instance:
uuid: "3760db14241b4cbd9002a1018eccd803"
format_stamp: "Formatted at 2026-08-12 06:37:02 on dist-test-slave-8n49"
I20260812 06:37:02.082186 20864 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-0/wal/instance:
uuid: "3760db14241b4cbd9002a1018eccd803"
format_stamp: "Formatted at 2026-08-12 06:37:02 on dist-test-slave-8n49"
I20260812 06:37:02.085629 20864 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.000s	sys 0.004s
I20260812 06:37:02.087873 20880 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:02.089000 20864 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:02.089143 20864 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-0/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-0/wal
uuid: "3760db14241b4cbd9002a1018eccd803"
format_stamp: "Formatted at 2026-08-12 06:37:02 on dist-test-slave-8n49"
I20260812 06:37:02.089253 20864 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:02.119916 20864 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:02.120738 20864 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:02.120934 20864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:02.128180 20944 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.254:43859 every 8 connection(s)
I20260812 06:37:02.128186 20864 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.254:43859
I20260812 06:37:02.129347 20864 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-0/data/info.pb
I20260812 06:37:02.130143 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 20864
I20260812 06:37:02.130254 19875 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-0/wal/instance
I20260812 06:37:02.131024 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.253:44447
--webserver_interface=127.19.104.253
--webserver_port=0
--builtin_ntp_servers=127.19.104.212:43903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.104.254:43859,127.19.104.253:44447,127.19.104.252:43887
--leader_failure_max_missed_heartbeat_periods=1.0 with env {}
I20260812 06:37:02.133006 20945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:02.136420 20945 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:02.150036 20945 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } has no permanent_uuid. Determining permanent_uuid...
W20260812 06:37:02.150938 20885 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.104.253:44447: connect: Connection refused (error 111)
W20260812 06:37:02.151960 20945 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.253:44447: Network error: Client connection negotiation failed: client connection to 127.19.104.253:44447: connect: Connection refused (error 111)
I20260812 06:37:02.201241 20945 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } attempt: 1
W20260812 06:37:02.202898 20945 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.253:44447: Network error: Client connection negotiation failed: client connection to 127.19.104.253:44447: connect: Connection refused (error 111)
W20260812 06:37:02.238829 20948 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:02.239060 20948 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:02.239099 20948 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:02.242434 20948 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:37:02.242503 20948 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:02.242523 20948 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:37:02.242542 20948 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:37:02.246408 20948 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:43903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.104.254:43859,127.19.104.253:44447,127.19.104.252:43887
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.253:44447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.104.253
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.20948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:02.247573 20948 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:02.248762 20948 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:02.254175 20957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:02.254132 20960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:02.254134 20956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:02.254326 20948 server_base.cc:1061] running on GCE node
I20260812 06:37:02.255678 20948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:02.256494 20948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:02.257656 20948 hybrid_clock.cc:648] HybridClock initialized: now 1786516622257637 us; error 30 us; skew 500 ppm
I20260812 06:37:02.259338 20948 webserver.cc:533] Webserver started at http://127.19.104.253:36269/ using document root <none> and password file <none>
I20260812 06:37:02.259920 20948 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:02.259976 20948 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:02.260133 20948 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:02.261646 20948 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-1/data/instance:
uuid: "d670d64a80c24cacb1d5a6cbbff302e6"
format_stamp: "Formatted at 2026-08-12 06:37:02 on dist-test-slave-8n49"
I20260812 06:37:02.262066 20948 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-1/wal/instance:
uuid: "d670d64a80c24cacb1d5a6cbbff302e6"
format_stamp: "Formatted at 2026-08-12 06:37:02 on dist-test-slave-8n49"
I20260812 06:37:02.265225 20948 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.001s	sys 0.002s
I20260812 06:37:02.267264 20967 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:02.268220 20948 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 06:37:02.268343 20948 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-1/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-1/wal
uuid: "d670d64a80c24cacb1d5a6cbbff302e6"
format_stamp: "Formatted at 2026-08-12 06:37:02 on dist-test-slave-8n49"
I20260812 06:37:02.268442 20948 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:02.271114 20945 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } attempt: 2
W20260812 06:37:02.272615 20945 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.253:44447: Network error: Client connection negotiation failed: client connection to 127.19.104.253:44447: connect: Connection refused (error 111)
I20260812 06:37:02.283247 20948 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:02.283861 20948 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:02.284008 20948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:02.291168 21026 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.253:44447 every 8 connection(s)
I20260812 06:37:02.291175 20948 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.253:44447
I20260812 06:37:02.292218 20948 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-1/data/info.pb
I20260812 06:37:02.295146 21027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:02.297513 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 20948
I20260812 06:37:02.297648 19875 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-1/wal/instance
I20260812 06:37:02.298277 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.252:43887
--webserver_interface=127.19.104.252
--webserver_port=0
--builtin_ntp_servers=127.19.104.212:43903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.104.254:43859,127.19.104.253:44447,127.19.104.252:43887
--leader_failure_max_missed_heartbeat_periods=1.0 with env {}
I20260812 06:37:02.298550 21027 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:02.307278 21027 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:02.314004 21027 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } has no permanent_uuid. Determining permanent_uuid...
W20260812 06:37:02.314798 20970 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.104.252:43887: connect: Connection refused (error 111)
W20260812 06:37:02.315682 21027 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.252:43887: Network error: Client connection negotiation failed: client connection to 127.19.104.252:43887: connect: Connection refused (error 111)
I20260812 06:37:02.363807 20945 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } attempt: 3
I20260812 06:37:02.364917 21027 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } attempt: 1
W20260812 06:37:02.366963 21027 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.252:43887: Network error: Client connection negotiation failed: client connection to 127.19.104.252:43887: connect: Connection refused (error 111)
I20260812 06:37:02.368677 20945 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } has no permanent_uuid. Determining permanent_uuid...
W20260812 06:37:02.370100 20945 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.252:43887: Network error: Client connection negotiation failed: client connection to 127.19.104.252:43887: connect: Connection refused (error 111)
I20260812 06:37:02.401306 20945 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } attempt: 1
W20260812 06:37:02.402889 20945 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.252:43887: Network error: Client connection negotiation failed: client connection to 127.19.104.252:43887: connect: Connection refused (error 111)
W20260812 06:37:02.407752 21029 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:02.407953 21029 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:02.407990 21029 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:02.411221 21029 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:37:02.411273 21029 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:02.411293 21029 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:37:02.411310 21029 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:37:02.415195 21029 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:43903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.104.254:43859,127.19.104.253:44447,127.19.104.252:43887
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.252:43887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.104.252
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.21029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:02.416231 21029 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:02.417313 21029 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:02.422659 21043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:02.422675 21041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:02.422854 21029 server_base.cc:1061] running on GCE node
W20260812 06:37:02.422688 21040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:02.423416 21029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:02.424305 21029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:02.425475 21029 hybrid_clock.cc:648] HybridClock initialized: now 1786516622425469 us; error 41 us; skew 500 ppm
I20260812 06:37:02.427145 21029 webserver.cc:533] Webserver started at http://127.19.104.252:45169/ using document root <none> and password file <none>
I20260812 06:37:02.427709 21029 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:02.427793 21029 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:02.428015 21029 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:02.429597 21029 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-2/data/instance:
uuid: "790729e8843a44858d3eba2cfe10a572"
format_stamp: "Formatted at 2026-08-12 06:37:02 on dist-test-slave-8n49"
I20260812 06:37:02.430091 21029 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-2/wal/instance:
uuid: "790729e8843a44858d3eba2cfe10a572"
format_stamp: "Formatted at 2026-08-12 06:37:02 on dist-test-slave-8n49"
I20260812 06:37:02.433444 21029 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 06:37:02.435128 21027 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } attempt: 2
I20260812 06:37:02.435647 21049 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
W20260812 06:37:02.436710 21027 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.252:43887: Network error: Client connection negotiation failed: client connection to 127.19.104.252:43887: connect: Connection refused (error 111)
I20260812 06:37:02.436899 21029 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 06:37:02.437017 21029 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-2/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-2/wal
uuid: "790729e8843a44858d3eba2cfe10a572"
format_stamp: "Formatted at 2026-08-12 06:37:02 on dist-test-slave-8n49"
I20260812 06:37:02.437132 21029 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:02.453706 21029 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:02.454353 21029 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:02.454540 21029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:02.462085 21115 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.252:43887 every 8 connection(s)
I20260812 06:37:02.462088 21029 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.252:43887
I20260812 06:37:02.463092 21029 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-2/data/info.pb
I20260812 06:37:02.465085 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 21029
I20260812 06:37:02.465219 19875 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/master-2/wal/instance
I20260812 06:37:02.466064 21116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:02.470255 21116 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:02.478082 20945 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } attempt: 2
I20260812 06:37:02.478331 21116 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:02.482393 21116 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:02.484333 20945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803: Bootstrap starting.
I20260812 06:37:02.486882 20945 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:02.487790 20945 log.cc:826] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:02.488786 21116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572: Bootstrap starting.
I20260812 06:37:02.489531 20945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803: No bootstrap required, opened a new log
I20260812 06:37:02.490696 21116 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:02.491343 21116 log.cc:826] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:02.492909 20945 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } }
I20260812 06:37:02.493023 21116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572: No bootstrap required, opened a new log
I20260812 06:37:02.493095 20945 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:02.493170 20945 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3760db14241b4cbd9002a1018eccd803, State: Initialized, Role: FOLLOWER
I20260812 06:37:02.493675 20945 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } }
I20260812 06:37:02.494195 21124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } } }
I20260812 06:37:02.494316 21124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:02.494767 20945 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:37:02.496116 21116 raft_consensus.cc:359] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } }
I20260812 06:37:02.496265 21116 raft_consensus.cc:385] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:02.496325 21116 raft_consensus.cc:740] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 790729e8843a44858d3eba2cfe10a572, State: Initialized, Role: FOLLOWER
I20260812 06:37:02.496851 21116 consensus_queue.cc:260] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } }
I20260812 06:37:02.498167 21116 sys_catalog.cc:565] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 06:37:02.500283 21138 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:37:02.500363 21138 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:37:02.499529 20864 master_runner.cc:187] Error getting master registration for 127.19.104.253:44447: OK, instance_id { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" instance_seqno: 1786516622289304 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 06:37:02.501842 20864 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260812 06:37:02.503000 21152 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:37:02.503084 21152 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:37:02.503422 21139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } } }
I20260812 06:37:02.503528 21139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:02.503667 21124 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:37:02.503847 21124 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } }
I20260812 06:37:02.504838 21124 leader_election.cc:290] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d670d64a80c24cacb1d5a6cbbff302e6 (127.19.104.253:44447), 790729e8843a44858d3eba2cfe10a572 (127.19.104.252:43887)
I20260812 06:37:02.505582 21086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3760db14241b4cbd9002a1018eccd803" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "790729e8843a44858d3eba2cfe10a572" is_pre_election: true
I20260812 06:37:02.505853 21086 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3760db14241b4cbd9002a1018eccd803 in term 0.
I20260812 06:37:02.506350 20883 leader_election.cc:304] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3760db14241b4cbd9002a1018eccd803, 790729e8843a44858d3eba2cfe10a572; no voters: 
I20260812 06:37:02.506742 21124 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:37:02.506820 21124 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:37:02.506887 21124 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:02.508539 21124 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } }
I20260812 06:37:02.506700 21029 master_runner.cc:187] Error getting master registration for 127.19.104.253:44447: OK, instance_id { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" instance_seqno: 1786516622289304 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 06:37:02.508953 21029 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260812 06:37:02.509330 21124 leader_election.cc:290] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [CANDIDATE]: Term 1 election: Requested vote from peers d670d64a80c24cacb1d5a6cbbff302e6 (127.19.104.253:44447), 790729e8843a44858d3eba2cfe10a572 (127.19.104.252:43887)
I20260812 06:37:02.509649 21086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3760db14241b4cbd9002a1018eccd803" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "790729e8843a44858d3eba2cfe10a572"
I20260812 06:37:02.509770 21086 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:02.510890 21086 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3760db14241b4cbd9002a1018eccd803 in term 1.
I20260812 06:37:02.511299 20883 leader_election.cc:304] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3760db14241b4cbd9002a1018eccd803, 790729e8843a44858d3eba2cfe10a572; no voters: 
I20260812 06:37:02.511595 21124 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:37:02.511834 21124 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [term 1 LEADER]: Becoming Leader. State: Replica: 3760db14241b4cbd9002a1018eccd803, State: Running, Role: LEADER
I20260812 06:37:02.512146 21124 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } }
I20260812 06:37:02.513942 21124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3760db14241b4cbd9002a1018eccd803. Latest consensus state: current_term: 1 leader_uuid: "3760db14241b4cbd9002a1018eccd803" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } } }
I20260812 06:37:02.514050 21124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:02.514707 21156 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:37:02.515825 21156 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:37:02.522624 21086 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [term 1 FOLLOWER]: Refusing update from remote peer 3760db14241b4cbd9002a1018eccd803: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:37:02.523411 21154 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [LEADER]: Connected to new peer: Peer: permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:02.528420 21139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3760db14241b4cbd9002a1018eccd803. Latest consensus state: current_term: 1 leader_uuid: "3760db14241b4cbd9002a1018eccd803" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } } }
I20260812 06:37:02.528527 21139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:02.528808 21027 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } attempt: 3
I20260812 06:37:02.534703 21124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3760db14241b4cbd9002a1018eccd803" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } } }
I20260812 06:37:02.534832 21124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:02.535091 21139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3760db14241b4cbd9002a1018eccd803" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } } }
I20260812 06:37:02.535193 21139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:02.535400 21158 mvcc.cc:204] Tried to move back new op lower bound from 7317572085843681280 to 7317572085813047296. Current Snapshot: MvccSnapshot[applied={T|T < 7317572085843681280}]
I20260812 06:37:02.536708 21159 mvcc.cc:204] Tried to move back new op lower bound from 7317572085843681280 to 7317572085813047296. Current Snapshot: MvccSnapshot[applied={T|T < 7317572085843681280}]
I20260812 06:37:02.538089 21156 catalog_manager.cc:1383] Generated new cluster ID: 60992ffc946148aaa7609b0f49ffc93e
I20260812 06:37:02.538153 21156 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:37:02.539302 21027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6: Bootstrap starting.
I20260812 06:37:02.545081 21027 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:02.546887 21027 log.cc:826] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:02.552891 21156 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 06:37:02.554752 21156 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:37:02.568296 21027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6: No bootstrap required, opened a new log
I20260812 06:37:02.571830 21027 raft_consensus.cc:359] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } }
I20260812 06:37:02.572032 21027 raft_consensus.cc:385] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:02.572093 21027 raft_consensus.cc:740] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d670d64a80c24cacb1d5a6cbbff302e6, State: Initialized, Role: FOLLOWER
I20260812 06:37:02.572611 21027 consensus_queue.cc:260] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } }
I20260812 06:37:02.572711 21156 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803: Generated new TSK 0
I20260812 06:37:02.573963 21027 sys_catalog.cc:565] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:37:02.574234 20997 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:02.575495 20997 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [term 1 FOLLOWER]: Refusing update from remote peer 3760db14241b4cbd9002a1018eccd803: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:37:02.575945 20998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3760db14241b4cbd9002a1018eccd803" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d670d64a80c24cacb1d5a6cbbff302e6"
I20260812 06:37:02.576145 21154 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:02.576686 20999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3760db14241b4cbd9002a1018eccd803" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" is_pre_election: true
I20260812 06:37:02.577593 20998 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3760db14241b4cbd9002a1018eccd803 in term 1.
I20260812 06:37:02.578013 21156 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:37:02.582971 21177 sys_catalog.cc:455] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3760db14241b4cbd9002a1018eccd803" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } } }
I20260812 06:37:02.583086 21177 sys_catalog.cc:458] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:02.583165 21177 sys_catalog.cc:455] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3760db14241b4cbd9002a1018eccd803. Latest consensus state: current_term: 1 leader_uuid: "3760db14241b4cbd9002a1018eccd803" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } } }
I20260812 06:37:02.583222 21177 sys_catalog.cc:458] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:02.583280 21177 sys_catalog.cc:455] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3760db14241b4cbd9002a1018eccd803" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } } }
I20260812 06:37:02.583338 21177 sys_catalog.cc:458] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:02.584578 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.104.193:0
--local_ip_for_outbound_sockets=127.19.104.193
--webserver_interface=127.19.104.193
--webserver_port=0
--tserver_master_addrs=127.19.104.254:43859,127.19.104.253:44447,127.19.104.252:43887
--builtin_ntp_servers=127.19.104.212:43903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260812 06:37:02.589128 21174 mvcc.cc:204] Tried to move back new op lower bound from 7317572086026792960 to 7317572085813047296. Current Snapshot: MvccSnapshot[applied={T|T < 7317572085843681280}]
W20260812 06:37:02.591677 21182 catalog_manager.cc:1594] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:37:02.591751 21182 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260812 06:37:02.739487 21184 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:02.740149 21184 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:02.740283 21184 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:02.745412 21184 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:02.745690 21184 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.104.193
I20260812 06:37:02.751437 21184 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:43903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.104.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.104.193
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.19.104.254:43859,127.19.104.253:44447,127.19.104.252:43887
--never_fsync=true
--heap_profile_path=/tmp/kudu.21184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.104.193

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:02.753157 21184 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:02.754803 21184 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:02.762996 21200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:02.764288 21196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:02.764611 21184 server_base.cc:1061] running on GCE node
W20260812 06:37:02.765050 21197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:02.765488 21184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:02.766134 21184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:02.767310 21184 hybrid_clock.cc:648] HybridClock initialized: now 1786516622767259 us; error 66 us; skew 500 ppm
I20260812 06:37:02.769677 21184 webserver.cc:533] Webserver started at http://127.19.104.193:37199/ using document root <none> and password file <none>
I20260812 06:37:02.770344 21184 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:02.770418 21184 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:02.770615 21184 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:02.772706 21184 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-0/data/instance:
uuid: "50ac34da4237497e9ef12dcc51e1ec65"
format_stamp: "Formatted at 2026-08-12 06:37:02 on dist-test-slave-8n49"
I20260812 06:37:02.773433 21184 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-0/wal/instance:
uuid: "50ac34da4237497e9ef12dcc51e1ec65"
format_stamp: "Formatted at 2026-08-12 06:37:02 on dist-test-slave-8n49"
I20260812 06:37:02.777402 21184 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.001s
I20260812 06:37:02.779711 21208 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:02.780930 21184 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 06:37:02.781034 21184 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-0/wal
uuid: "50ac34da4237497e9ef12dcc51e1ec65"
format_stamp: "Formatted at 2026-08-12 06:37:02 on dist-test-slave-8n49"
I20260812 06:37:02.781181 21184 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:02.808507 21184 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:02.809289 21184 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:02.809514 21184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:02.810214 21184 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:02.811337 21184 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:02.811388 21184 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:02.811456 21184 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:02.811496 21184 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:02.823836 21184 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.193:42435
I20260812 06:37:02.823858 21343 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.193:42435 every 8 connection(s)
I20260812 06:37:02.824926 21184 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-0/data/info.pb
I20260812 06:37:02.833722 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 21184
I20260812 06:37:02.833858 19875 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-0/wal/instance
I20260812 06:37:02.835497 21347 heartbeater.cc:344] Connected to a master server at 127.19.104.253:44447
I20260812 06:37:02.835731 21347 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:02.836004 21345 heartbeater.cc:344] Connected to a master server at 127.19.104.252:43887
I20260812 06:37:02.836069 21346 heartbeater.cc:344] Connected to a master server at 127.19.104.254:43859
I20260812 06:37:02.836103 21345 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:02.836120 21346 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:02.836374 21347 heartbeater.cc:507] Master 127.19.104.253:44447 requested a full tablet report, sending...
I20260812 06:37:02.836373 21346 heartbeater.cc:507] Master 127.19.104.254:43859 requested a full tablet report, sending...
I20260812 06:37:02.837052 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.104.194:0
--local_ip_for_outbound_sockets=127.19.104.194
--webserver_interface=127.19.104.194
--webserver_port=0
--tserver_master_addrs=127.19.104.254:43859,127.19.104.253:44447,127.19.104.252:43887
--builtin_ntp_servers=127.19.104.212:43903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260812 06:37:02.837078 21345 heartbeater.cc:507] Master 127.19.104.252:43887 requested a full tablet report, sending...
I20260812 06:37:02.838143 21072 ts_manager.cc:194] Registered new tserver with Master: 50ac34da4237497e9ef12dcc51e1ec65 (127.19.104.193:42435)
I20260812 06:37:02.838131 20903 ts_manager.cc:194] Registered new tserver with Master: 50ac34da4237497e9ef12dcc51e1ec65 (127.19.104.193:42435)
I20260812 06:37:02.838135 20988 ts_manager.cc:194] Registered new tserver with Master: 50ac34da4237497e9ef12dcc51e1ec65 (127.19.104.193:42435)
I20260812 06:37:02.839704 20903 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.104.193:42747
W20260812 06:37:02.944525 21352 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:02.944868 21352 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:02.944967 21352 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:02.948415 21352 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:02.948628 21352 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.104.194
I20260812 06:37:02.952512 21352 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:43903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.104.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.104.194
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.19.104.254:43859,127.19.104.253:44447,127.19.104.252:43887
--never_fsync=true
--heap_profile_path=/tmp/kudu.21352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.104.194

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:02.953759 21352 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:02.954887 21352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:02.961673 21358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:02.961673 21362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:02.961673 21359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:02.962141 21352 server_base.cc:1061] running on GCE node
I20260812 06:37:02.962527 21352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:02.963052 21352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:02.964232 21352 hybrid_clock.cc:648] HybridClock initialized: now 1786516622964218 us; error 33 us; skew 500 ppm
I20260812 06:37:02.965978 21352 webserver.cc:533] Webserver started at http://127.19.104.194:35565/ using document root <none> and password file <none>
I20260812 06:37:02.966552 21352 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:02.966639 21352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:02.966866 21352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:02.968468 21352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-1/data/instance:
uuid: "84c89832a92343f0b279baf89d1086a5"
format_stamp: "Formatted at 2026-08-12 06:37:02 on dist-test-slave-8n49"
I20260812 06:37:02.969022 21352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-1/wal/instance:
uuid: "84c89832a92343f0b279baf89d1086a5"
format_stamp: "Formatted at 2026-08-12 06:37:02 on dist-test-slave-8n49"
I20260812 06:37:02.972252 21352 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 06:37:02.974462 21368 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:02.975445 21352 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260812 06:37:02.975598 21352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-1/wal
uuid: "84c89832a92343f0b279baf89d1086a5"
format_stamp: "Formatted at 2026-08-12 06:37:02 on dist-test-slave-8n49"
I20260812 06:37:02.975713 21352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:03.001075 21352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:03.001822 21352 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:03.002051 21352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:03.002671 21352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:03.003798 21352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:03.003880 21352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:03.003991 21352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:03.004048 21352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:03.014557 21352 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.194:35279
I20260812 06:37:03.014580 21497 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.194:35279 every 8 connection(s)
I20260812 06:37:03.015571 21352 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-1/data/info.pb
I20260812 06:37:03.024741 21499 heartbeater.cc:344] Connected to a master server at 127.19.104.252:43887
I20260812 06:37:03.024828 21501 heartbeater.cc:344] Connected to a master server at 127.19.104.253:44447
I20260812 06:37:03.024989 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 21352
I20260812 06:37:03.025074 21499 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:03.025075 21501 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:03.025115 19875 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-1/wal/instance
I20260812 06:37:03.025687 21501 heartbeater.cc:507] Master 127.19.104.253:44447 requested a full tablet report, sending...
I20260812 06:37:03.025687 21499 heartbeater.cc:507] Master 127.19.104.252:43887 requested a full tablet report, sending...
I20260812 06:37:03.024742 21500 heartbeater.cc:344] Connected to a master server at 127.19.104.254:43859
I20260812 06:37:03.025888 21500 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:03.026088 21500 heartbeater.cc:507] Master 127.19.104.254:43859 requested a full tablet report, sending...
I20260812 06:37:03.026885 21072 ts_manager.cc:194] Registered new tserver with Master: 84c89832a92343f0b279baf89d1086a5 (127.19.104.194:35279)
I20260812 06:37:03.027319 20988 ts_manager.cc:194] Registered new tserver with Master: 84c89832a92343f0b279baf89d1086a5 (127.19.104.194:35279)
I20260812 06:37:03.027371 20903 ts_manager.cc:194] Registered new tserver with Master: 84c89832a92343f0b279baf89d1086a5 (127.19.104.194:35279)
I20260812 06:37:03.027810 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.104.195:0
--local_ip_for_outbound_sockets=127.19.104.195
--webserver_interface=127.19.104.195
--webserver_port=0
--tserver_master_addrs=127.19.104.254:43859,127.19.104.253:44447,127.19.104.252:43887
--builtin_ntp_servers=127.19.104.212:43903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260812 06:37:03.027982 20903 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.104.194:37237
I20260812 06:37:03.070883 21507 raft_consensus.cc:493] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3760db14241b4cbd9002a1018eccd803)
I20260812 06:37:03.071077 21507 raft_consensus.cc:515] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } }
I20260812 06:37:03.071969 21507 leader_election.cc:290] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3760db14241b4cbd9002a1018eccd803 (127.19.104.254:43859), d670d64a80c24cacb1d5a6cbbff302e6 (127.19.104.253:44447)
I20260812 06:37:03.072593 20997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "790729e8843a44858d3eba2cfe10a572" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" is_pre_election: true
I20260812 06:37:03.072631 20917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "790729e8843a44858d3eba2cfe10a572" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3760db14241b4cbd9002a1018eccd803" is_pre_election: true
I20260812 06:37:03.073236 21057 leader_election.cc:304] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 790729e8843a44858d3eba2cfe10a572; no voters: 3760db14241b4cbd9002a1018eccd803, d670d64a80c24cacb1d5a6cbbff302e6
I20260812 06:37:03.073503 21507 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 06:37:03.079187 21164 raft_consensus.cc:493] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3760db14241b4cbd9002a1018eccd803)
I20260812 06:37:03.079344 21164 raft_consensus.cc:515] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } }
I20260812 06:37:03.080340 21164 leader_election.cc:290] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3760db14241b4cbd9002a1018eccd803 (127.19.104.254:43859), 790729e8843a44858d3eba2cfe10a572 (127.19.104.252:43887)
I20260812 06:37:03.080834 21086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "790729e8843a44858d3eba2cfe10a572" is_pre_election: true
I20260812 06:37:03.080942 20917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3760db14241b4cbd9002a1018eccd803" is_pre_election: true
I20260812 06:37:03.081502 20973 leader_election.cc:304] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d670d64a80c24cacb1d5a6cbbff302e6; no voters: 3760db14241b4cbd9002a1018eccd803, 790729e8843a44858d3eba2cfe10a572
I20260812 06:37:03.081746 21164 raft_consensus.cc:2749] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 06:37:03.137912 21506 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:03.138149 21506 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:03.138245 21506 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260812 06:37:03.139706 21154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3760db14241b4cbd9002a1018eccd803" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } } }
I20260812 06:37:03.139845 21154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [sys.catalog]: This master's current role is: LEADER
W20260812 06:37:03.142205 21506 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:03.142365 21506 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.104.195
I20260812 06:37:03.146198 21506 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:43903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.104.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.104.195
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.19.104.254:43859,127.19.104.253:44447,127.19.104.252:43887
--never_fsync=true
--heap_profile_path=/tmp/kudu.21506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.104.195

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:03.147226 21506 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:03.148319 21506 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:03.154773 21515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:03.154850 21506 server_base.cc:1061] running on GCE node
W20260812 06:37:03.154863 21517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:03.154773 21519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:03.155447 21506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:03.155970 21506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:03.157147 21506 hybrid_clock.cc:648] HybridClock initialized: now 1786516623157134 us; error 34 us; skew 500 ppm
I20260812 06:37:03.158756 21506 webserver.cc:533] Webserver started at http://127.19.104.195:39833/ using document root <none> and password file <none>
I20260812 06:37:03.159307 21506 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:03.159391 21506 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:03.159606 21506 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:03.161197 21506 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-2/data/instance:
uuid: "2da2eeaba58644fea7afdd531ff9fdfc"
format_stamp: "Formatted at 2026-08-12 06:37:03 on dist-test-slave-8n49"
I20260812 06:37:03.161674 21506 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-2/wal/instance:
uuid: "2da2eeaba58644fea7afdd531ff9fdfc"
format_stamp: "Formatted at 2026-08-12 06:37:03 on dist-test-slave-8n49"
I20260812 06:37:03.165077 21506 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.001s
I20260812 06:37:03.167241 21525 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:03.168238 21506 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:03.168380 21506 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-2/wal
uuid: "2da2eeaba58644fea7afdd531ff9fdfc"
format_stamp: "Formatted at 2026-08-12 06:37:03 on dist-test-slave-8n49"
I20260812 06:37:03.168484 21506 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:03.191385 21506 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:03.192013 21506 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:03.192210 21506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:03.192828 21506 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:03.193905 21506 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:03.193979 21506 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:03.194051 21506 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:03.194087 21506 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:03.203204 21506 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.195:36369
I20260812 06:37:03.203277 21661 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.195:36369 every 8 connection(s)
I20260812 06:37:03.204137 21506 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-2/data/info.pb
I20260812 06:37:03.204424 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 21506
I20260812 06:37:03.204591 19875 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestRenameTableSync_0.1786516586160649-19875-0/minicluster-data/ts-2/wal/instance
I20260812 06:37:03.214310 21663 heartbeater.cc:344] Connected to a master server at 127.19.104.254:43859
I20260812 06:37:03.214313 21662 heartbeater.cc:344] Connected to a master server at 127.19.104.252:43887
I20260812 06:37:03.214600 21663 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:03.214605 21662 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:03.214618 21664 heartbeater.cc:344] Connected to a master server at 127.19.104.253:44447
I20260812 06:37:03.214747 21664 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:03.215188 21663 heartbeater.cc:507] Master 127.19.104.254:43859 requested a full tablet report, sending...
I20260812 06:37:03.215193 21664 heartbeater.cc:507] Master 127.19.104.253:44447 requested a full tablet report, sending...
I20260812 06:37:03.215193 21662 heartbeater.cc:507] Master 127.19.104.252:43887 requested a full tablet report, sending...
I20260812 06:37:03.216393 21072 ts_manager.cc:194] Registered new tserver with Master: 2da2eeaba58644fea7afdd531ff9fdfc (127.19.104.195:36369)
I20260812 06:37:03.216408 20903 ts_manager.cc:194] Registered new tserver with Master: 2da2eeaba58644fea7afdd531ff9fdfc (127.19.104.195:36369)
I20260812 06:37:03.216542 20988 ts_manager.cc:194] Registered new tserver with Master: 2da2eeaba58644fea7afdd531ff9fdfc (127.19.104.195:36369)
I20260812 06:37:03.217177 20903 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.104.195:56179
I20260812 06:37:03.226707 19875 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 06:37:03.235939 20903 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49568:
name: "default.test_alter_table_sync"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260812 06:37:03.237171 20903 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.test_alter_table_sync in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:37:03.257100 21423 tablet_service.cc:1511] Processing CreateTablet for tablet 6d8451a4697b459c9aab7125621c33f5 (DEFAULT_TABLE table=default.test_alter_table_sync [id=148a3a5cfcda40c5ab122f38f5a5e333]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:37:03.257300 21267 tablet_service.cc:1511] Processing CreateTablet for tablet 6d8451a4697b459c9aab7125621c33f5 (DEFAULT_TABLE table=default.test_alter_table_sync [id=148a3a5cfcda40c5ab122f38f5a5e333]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:37:03.258095 21267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d8451a4697b459c9aab7125621c33f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:03.258111 21423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d8451a4697b459c9aab7125621c33f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:03.258451 21583 tablet_service.cc:1511] Processing CreateTablet for tablet 6d8451a4697b459c9aab7125621c33f5 (DEFAULT_TABLE table=default.test_alter_table_sync [id=148a3a5cfcda40c5ab122f38f5a5e333]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:37:03.259254 21583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d8451a4697b459c9aab7125621c33f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:03.262925 21691 tablet_bootstrap.cc:492] T 6d8451a4697b459c9aab7125621c33f5 P 50ac34da4237497e9ef12dcc51e1ec65: Bootstrap starting.
I20260812 06:37:03.264106 21692 tablet_bootstrap.cc:492] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5: Bootstrap starting.
I20260812 06:37:03.264897 21693 tablet_bootstrap.cc:492] T 6d8451a4697b459c9aab7125621c33f5 P 2da2eeaba58644fea7afdd531ff9fdfc: Bootstrap starting.
I20260812 06:37:03.265226 21691 tablet_bootstrap.cc:654] T 6d8451a4697b459c9aab7125621c33f5 P 50ac34da4237497e9ef12dcc51e1ec65: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:03.266119 21692 tablet_bootstrap.cc:654] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:03.266227 21691 log.cc:826] T 6d8451a4697b459c9aab7125621c33f5 P 50ac34da4237497e9ef12dcc51e1ec65: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:03.266877 21693 tablet_bootstrap.cc:654] T 6d8451a4697b459c9aab7125621c33f5 P 2da2eeaba58644fea7afdd531ff9fdfc: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:03.267226 21692 log.cc:826] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:03.267753 21693 log.cc:826] T 6d8451a4697b459c9aab7125621c33f5 P 2da2eeaba58644fea7afdd531ff9fdfc: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:03.268064 21691 tablet_bootstrap.cc:492] T 6d8451a4697b459c9aab7125621c33f5 P 50ac34da4237497e9ef12dcc51e1ec65: No bootstrap required, opened a new log
I20260812 06:37:03.268273 21691 ts_tablet_manager.cc:1403] T 6d8451a4697b459c9aab7125621c33f5 P 50ac34da4237497e9ef12dcc51e1ec65: Time spent bootstrapping tablet: real 0.006s	user 0.004s	sys 0.000s
I20260812 06:37:03.269245 21693 tablet_bootstrap.cc:492] T 6d8451a4697b459c9aab7125621c33f5 P 2da2eeaba58644fea7afdd531ff9fdfc: No bootstrap required, opened a new log
I20260812 06:37:03.269429 21693 ts_tablet_manager.cc:1403] T 6d8451a4697b459c9aab7125621c33f5 P 2da2eeaba58644fea7afdd531ff9fdfc: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 06:37:03.270877 21691 raft_consensus.cc:359] T 6d8451a4697b459c9aab7125621c33f5 P 50ac34da4237497e9ef12dcc51e1ec65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da2eeaba58644fea7afdd531ff9fdfc" member_type: VOTER last_known_addr { host: "127.19.104.195" port: 36369 } } peers { permanent_uuid: "84c89832a92343f0b279baf89d1086a5" member_type: VOTER last_known_addr { host: "127.19.104.194" port: 35279 } } peers { permanent_uuid: "50ac34da4237497e9ef12dcc51e1ec65" member_type: VOTER last_known_addr { host: "127.19.104.193" port: 42435 } }
I20260812 06:37:03.271075 21691 raft_consensus.cc:385] T 6d8451a4697b459c9aab7125621c33f5 P 50ac34da4237497e9ef12dcc51e1ec65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:03.271142 21691 raft_consensus.cc:740] T 6d8451a4697b459c9aab7125621c33f5 P 50ac34da4237497e9ef12dcc51e1ec65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50ac34da4237497e9ef12dcc51e1ec65, State: Initialized, Role: FOLLOWER
I20260812 06:37:03.271615 21691 consensus_queue.cc:260] T 6d8451a4697b459c9aab7125621c33f5 P 50ac34da4237497e9ef12dcc51e1ec65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da2eeaba58644fea7afdd531ff9fdfc" member_type: VOTER last_known_addr { host: "127.19.104.195" port: 36369 } } peers { permanent_uuid: "84c89832a92343f0b279baf89d1086a5" member_type: VOTER last_known_addr { host: "127.19.104.194" port: 35279 } } peers { permanent_uuid: "50ac34da4237497e9ef12dcc51e1ec65" member_type: VOTER last_known_addr { host: "127.19.104.193" port: 42435 } }
I20260812 06:37:03.271816 21693 raft_consensus.cc:359] T 6d8451a4697b459c9aab7125621c33f5 P 2da2eeaba58644fea7afdd531ff9fdfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da2eeaba58644fea7afdd531ff9fdfc" member_type: VOTER last_known_addr { host: "127.19.104.195" port: 36369 } } peers { permanent_uuid: "84c89832a92343f0b279baf89d1086a5" member_type: VOTER last_known_addr { host: "127.19.104.194" port: 35279 } } peers { permanent_uuid: "50ac34da4237497e9ef12dcc51e1ec65" member_type: VOTER last_known_addr { host: "127.19.104.193" port: 42435 } }
I20260812 06:37:03.272029 21693 raft_consensus.cc:385] T 6d8451a4697b459c9aab7125621c33f5 P 2da2eeaba58644fea7afdd531ff9fdfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:03.272083 21693 raft_consensus.cc:740] T 6d8451a4697b459c9aab7125621c33f5 P 2da2eeaba58644fea7afdd531ff9fdfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2da2eeaba58644fea7afdd531ff9fdfc, State: Initialized, Role: FOLLOWER
I20260812 06:37:03.272459 21693 consensus_queue.cc:260] T 6d8451a4697b459c9aab7125621c33f5 P 2da2eeaba58644fea7afdd531ff9fdfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da2eeaba58644fea7afdd531ff9fdfc" member_type: VOTER last_known_addr { host: "127.19.104.195" port: 36369 } } peers { permanent_uuid: "84c89832a92343f0b279baf89d1086a5" member_type: VOTER last_known_addr { host: "127.19.104.194" port: 35279 } } peers { permanent_uuid: "50ac34da4237497e9ef12dcc51e1ec65" member_type: VOTER last_known_addr { host: "127.19.104.193" port: 42435 } }
I20260812 06:37:03.272787 21691 ts_tablet_manager.cc:1434] T 6d8451a4697b459c9aab7125621c33f5 P 50ac34da4237497e9ef12dcc51e1ec65: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 06:37:03.273041 21346 heartbeater.cc:499] Master 127.19.104.254:43859 was elected leader, sending a full tablet report...
I20260812 06:37:03.273442 21693 ts_tablet_manager.cc:1434] T 6d8451a4697b459c9aab7125621c33f5 P 2da2eeaba58644fea7afdd531ff9fdfc: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 06:37:03.274474 21663 heartbeater.cc:499] Master 127.19.104.254:43859 was elected leader, sending a full tablet report...
I20260812 06:37:03.275214 21692 tablet_bootstrap.cc:492] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5: No bootstrap required, opened a new log
I20260812 06:37:03.275393 21692 ts_tablet_manager.cc:1403] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5: Time spent bootstrapping tablet: real 0.011s	user 0.005s	sys 0.000s
I20260812 06:37:03.278599 21692 raft_consensus.cc:359] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da2eeaba58644fea7afdd531ff9fdfc" member_type: VOTER last_known_addr { host: "127.19.104.195" port: 36369 } } peers { permanent_uuid: "84c89832a92343f0b279baf89d1086a5" member_type: VOTER last_known_addr { host: "127.19.104.194" port: 35279 } } peers { permanent_uuid: "50ac34da4237497e9ef12dcc51e1ec65" member_type: VOTER last_known_addr { host: "127.19.104.193" port: 42435 } }
I20260812 06:37:03.278793 21692 raft_consensus.cc:385] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:03.278872 21692 raft_consensus.cc:740] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c89832a92343f0b279baf89d1086a5, State: Initialized, Role: FOLLOWER
I20260812 06:37:03.279281 21692 consensus_queue.cc:260] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da2eeaba58644fea7afdd531ff9fdfc" member_type: VOTER last_known_addr { host: "127.19.104.195" port: 36369 } } peers { permanent_uuid: "84c89832a92343f0b279baf89d1086a5" member_type: VOTER last_known_addr { host: "127.19.104.194" port: 35279 } } peers { permanent_uuid: "50ac34da4237497e9ef12dcc51e1ec65" member_type: VOTER last_known_addr { host: "127.19.104.193" port: 42435 } }
I20260812 06:37:03.280156 21692 ts_tablet_manager.cc:1434] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.004s
I20260812 06:37:03.280646 21500 heartbeater.cc:499] Master 127.19.104.254:43859 was elected leader, sending a full tablet report...
W20260812 06:37:03.326419 21348 tablet.cc:2406] T 6d8451a4697b459c9aab7125621c33f5 P 50ac34da4237497e9ef12dcc51e1ec65: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:37:03.422742 21701 raft_consensus.cc:493] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:37:03.422951 21701 raft_consensus.cc:515] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da2eeaba58644fea7afdd531ff9fdfc" member_type: VOTER last_known_addr { host: "127.19.104.195" port: 36369 } } peers { permanent_uuid: "84c89832a92343f0b279baf89d1086a5" member_type: VOTER last_known_addr { host: "127.19.104.194" port: 35279 } } peers { permanent_uuid: "50ac34da4237497e9ef12dcc51e1ec65" member_type: VOTER last_known_addr { host: "127.19.104.193" port: 42435 } }
I20260812 06:37:03.424187 21701 leader_election.cc:290] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2da2eeaba58644fea7afdd531ff9fdfc (127.19.104.195:36369), 50ac34da4237497e9ef12dcc51e1ec65 (127.19.104.193:42435)
I20260812 06:37:03.428653 21289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d8451a4697b459c9aab7125621c33f5" candidate_uuid: "84c89832a92343f0b279baf89d1086a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ac34da4237497e9ef12dcc51e1ec65" is_pre_election: true
I20260812 06:37:03.428668 21607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d8451a4697b459c9aab7125621c33f5" candidate_uuid: "84c89832a92343f0b279baf89d1086a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2da2eeaba58644fea7afdd531ff9fdfc" is_pre_election: true
I20260812 06:37:03.428953 21289 raft_consensus.cc:2468] T 6d8451a4697b459c9aab7125621c33f5 P 50ac34da4237497e9ef12dcc51e1ec65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84c89832a92343f0b279baf89d1086a5 in term 0.
I20260812 06:37:03.428953 21607 raft_consensus.cc:2468] T 6d8451a4697b459c9aab7125621c33f5 P 2da2eeaba58644fea7afdd531ff9fdfc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84c89832a92343f0b279baf89d1086a5 in term 0.
I20260812 06:37:03.429466 21370 leader_election.cc:304] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50ac34da4237497e9ef12dcc51e1ec65, 84c89832a92343f0b279baf89d1086a5; no voters: 
I20260812 06:37:03.429721 21701 raft_consensus.cc:2804] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:37:03.429807 21701 raft_consensus.cc:493] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:37:03.429868 21701 raft_consensus.cc:3060] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:03.430783 21701 raft_consensus.cc:515] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da2eeaba58644fea7afdd531ff9fdfc" member_type: VOTER last_known_addr { host: "127.19.104.195" port: 36369 } } peers { permanent_uuid: "84c89832a92343f0b279baf89d1086a5" member_type: VOTER last_known_addr { host: "127.19.104.194" port: 35279 } } peers { permanent_uuid: "50ac34da4237497e9ef12dcc51e1ec65" member_type: VOTER last_known_addr { host: "127.19.104.193" port: 42435 } }
I20260812 06:37:03.431169 21701 leader_election.cc:290] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5 [CANDIDATE]: Term 1 election: Requested vote from peers 2da2eeaba58644fea7afdd531ff9fdfc (127.19.104.195:36369), 50ac34da4237497e9ef12dcc51e1ec65 (127.19.104.193:42435)
I20260812 06:37:03.431538 21289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d8451a4697b459c9aab7125621c33f5" candidate_uuid: "84c89832a92343f0b279baf89d1086a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ac34da4237497e9ef12dcc51e1ec65"
I20260812 06:37:03.431561 21607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d8451a4697b459c9aab7125621c33f5" candidate_uuid: "84c89832a92343f0b279baf89d1086a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2da2eeaba58644fea7afdd531ff9fdfc"
I20260812 06:37:03.431636 21289 raft_consensus.cc:3060] T 6d8451a4697b459c9aab7125621c33f5 P 50ac34da4237497e9ef12dcc51e1ec65 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:03.431690 21607 raft_consensus.cc:3060] T 6d8451a4697b459c9aab7125621c33f5 P 2da2eeaba58644fea7afdd531ff9fdfc [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:03.432492 21289 raft_consensus.cc:2468] T 6d8451a4697b459c9aab7125621c33f5 P 50ac34da4237497e9ef12dcc51e1ec65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c89832a92343f0b279baf89d1086a5 in term 1.
I20260812 06:37:03.432638 21607 raft_consensus.cc:2468] T 6d8451a4697b459c9aab7125621c33f5 P 2da2eeaba58644fea7afdd531ff9fdfc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c89832a92343f0b279baf89d1086a5 in term 1.
I20260812 06:37:03.432883 21370 leader_election.cc:304] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50ac34da4237497e9ef12dcc51e1ec65, 84c89832a92343f0b279baf89d1086a5; no voters: 
I20260812 06:37:03.433066 21701 raft_consensus.cc:2804] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:37:03.433264 21701 raft_consensus.cc:697] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5 [term 1 LEADER]: Becoming Leader. State: Replica: 84c89832a92343f0b279baf89d1086a5, State: Running, Role: LEADER
I20260812 06:37:03.433593 21701 consensus_queue.cc:237] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da2eeaba58644fea7afdd531ff9fdfc" member_type: VOTER last_known_addr { host: "127.19.104.195" port: 36369 } } peers { permanent_uuid: "84c89832a92343f0b279baf89d1086a5" member_type: VOTER last_known_addr { host: "127.19.104.194" port: 35279 } } peers { permanent_uuid: "50ac34da4237497e9ef12dcc51e1ec65" member_type: VOTER last_known_addr { host: "127.19.104.193" port: 42435 } }
I20260812 06:37:03.437577 20904 catalog_manager.cc:5719] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84c89832a92343f0b279baf89d1086a5 (127.19.104.194). New cstate: current_term: 1 leader_uuid: "84c89832a92343f0b279baf89d1086a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da2eeaba58644fea7afdd531ff9fdfc" member_type: VOTER last_known_addr { host: "127.19.104.195" port: 36369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84c89832a92343f0b279baf89d1086a5" member_type: VOTER last_known_addr { host: "127.19.104.194" port: 35279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50ac34da4237497e9ef12dcc51e1ec65" member_type: VOTER last_known_addr { host: "127.19.104.193" port: 42435 } health_report { overall_health: UNKNOWN } } }
W20260812 06:37:03.455408 21665 tablet.cc:2406] T 6d8451a4697b459c9aab7125621c33f5 P 2da2eeaba58644fea7afdd531ff9fdfc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:37:03.483609 19875 master_failover-itest.cc:269] Pausing leader master
I20260812 06:37:03.504412 21152 catalog_manager.cc:1295] Loaded cluster ID: 60992ffc946148aaa7609b0f49ffc93e
I20260812 06:37:03.504498 21152 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572: loading cluster ID for follower catalog manager: success
I20260812 06:37:03.506235 21152 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572: acquiring CA information for follower catalog manager: success
I20260812 06:37:03.507246 21152 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 06:37:03.517102 21502 tablet.cc:2406] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:37:03.593170 21182 catalog_manager.cc:1295] Loaded cluster ID: 60992ffc946148aaa7609b0f49ffc93e
I20260812 06:37:03.593257 21182 catalog_manager.cc:1588] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6: loading cluster ID for follower catalog manager: success
I20260812 06:37:03.594931 21182 catalog_manager.cc:1610] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6: acquiring CA information for follower catalog manager: success
I20260812 06:37:03.595994 21182 catalog_manager.cc:1638] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:37:03.941725 21704 raft_consensus.cc:493] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3760db14241b4cbd9002a1018eccd803)
I20260812 06:37:03.941900 21704 raft_consensus.cc:515] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } }
I20260812 06:37:03.942344 21704 leader_election.cc:290] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3760db14241b4cbd9002a1018eccd803 (127.19.104.254:43859), d670d64a80c24cacb1d5a6cbbff302e6 (127.19.104.253:44447)
I20260812 06:37:03.942919 20997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "790729e8843a44858d3eba2cfe10a572" candidate_term: 2 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: false dest_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" is_pre_election: true
I20260812 06:37:03.943048 20997 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 790729e8843a44858d3eba2cfe10a572 in term 1.
I20260812 06:37:03.943409 21057 leader_election.cc:304] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 790729e8843a44858d3eba2cfe10a572, d670d64a80c24cacb1d5a6cbbff302e6; no voters: 
I20260812 06:37:03.943635 21704 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 06:37:03.943694 21704 raft_consensus.cc:493] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3760db14241b4cbd9002a1018eccd803)
I20260812 06:37:03.943744 21704 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:37:03.944710 21704 raft_consensus.cc:515] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } }
I20260812 06:37:03.945070 21704 leader_election.cc:290] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [CANDIDATE]: Term 2 election: Requested vote from peers 3760db14241b4cbd9002a1018eccd803 (127.19.104.254:43859), d670d64a80c24cacb1d5a6cbbff302e6 (127.19.104.253:44447)
I20260812 06:37:03.945515 20997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "790729e8843a44858d3eba2cfe10a572" candidate_term: 2 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: false dest_uuid: "d670d64a80c24cacb1d5a6cbbff302e6"
I20260812 06:37:03.945622 20997 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:37:03.946452 20997 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 790729e8843a44858d3eba2cfe10a572 in term 2.
I20260812 06:37:03.946769 21057 leader_election.cc:304] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 790729e8843a44858d3eba2cfe10a572, d670d64a80c24cacb1d5a6cbbff302e6; no voters: 
I20260812 06:37:03.947005 21704 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 06:37:03.947206 21704 raft_consensus.cc:697] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [term 2 LEADER]: Becoming Leader. State: Replica: 790729e8843a44858d3eba2cfe10a572, State: Running, Role: LEADER
I20260812 06:37:03.947515 21704 consensus_queue.cc:237] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } }
I20260812 06:37:03.948716 21705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 790729e8843a44858d3eba2cfe10a572. Latest consensus state: current_term: 2 leader_uuid: "790729e8843a44858d3eba2cfe10a572" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } } }
I20260812 06:37:03.948838 21705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:03.949147 21706 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:37:03.950469 21706 catalog_manager.cc:679] Loaded metadata for table default.test_alter_table_sync [id=148a3a5cfcda40c5ab122f38f5a5e333]
I20260812 06:37:03.951365 21706 tablet_loader.cc:96] loaded metadata for tablet 6d8451a4697b459c9aab7125621c33f5 (table default.test_alter_table_sync [id=148a3a5cfcda40c5ab122f38f5a5e333])
I20260812 06:37:03.951483 21706 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:37:03.951815 21706 catalog_manager.cc:1295] Loaded cluster ID: 60992ffc946148aaa7609b0f49ffc93e
I20260812 06:37:03.951857 21706 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:37:03.952342 21706 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:37:03.952792 21706 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572: Loaded TSK: 0
I20260812 06:37:03.953342 21706 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:37:04.008002 21707 consensus_queue.cc:1048] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2da2eeaba58644fea7afdd531ff9fdfc" member_type: VOTER last_known_addr { host: "127.19.104.195" port: 36369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:04.013545 21662 heartbeater.cc:499] Master 127.19.104.252:43887 was elected leader, sending a full tablet report...
I20260812 06:37:04.014910 21713 consensus_queue.cc:1048] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50ac34da4237497e9ef12dcc51e1ec65" member_type: VOTER last_known_addr { host: "127.19.104.193" port: 42435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:04.019079 21345 heartbeater.cc:499] Master 127.19.104.252:43887 was elected leader, sending a full tablet report...
W20260812 06:37:04.449503 21057 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.104.254:43859 timed out after 0.506s (SENT)
I20260812 06:37:04.515054 21499 heartbeater.cc:499] Master 127.19.104.252:43887 was elected leader, sending a full tablet report...
I20260812 06:37:04.555116 20997 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [term 2 FOLLOWER]: Refusing update from remote peer 790729e8843a44858d3eba2cfe10a572: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20260812 06:37:04.555799 21705 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 06:37:04.557142 21718 sys_catalog.cc:455] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 790729e8843a44858d3eba2cfe10a572. Latest consensus state: current_term: 2 leader_uuid: "790729e8843a44858d3eba2cfe10a572" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } } }
I20260812 06:37:04.557250 21718 sys_catalog.cc:458] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:04.558537 21721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "790729e8843a44858d3eba2cfe10a572" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } } }
I20260812 06:37:04.558674 21721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:04.559033 21718 sys_catalog.cc:455] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "790729e8843a44858d3eba2cfe10a572" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } } }
I20260812 06:37:04.559136 21718 sys_catalog.cc:458] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 06:37:04.779857 21346 heartbeater.cc:646] Failed to heartbeat to 127.19.104.254:43859 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.19.104.254:43859 timed out after 1.000s (SENT)
W20260812 06:37:04.780743 21663 heartbeater.cc:646] Failed to heartbeat to 127.19.104.254:43859 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.19.104.254:43859 timed out after 1.000s (SENT)
W20260812 06:37:04.944115 21500 heartbeater.cc:646] Failed to heartbeat to 127.19.104.254:43859 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.19.104.254:43859 timed out after 1.000s (SENT)
W20260812 06:37:04.958173 21057 leader_election.cc:336] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3760db14241b4cbd9002a1018eccd803 (127.19.104.254:43859): Timed out: RequestConsensusVote RPC to 127.19.104.254:43859 timed out after 0.506s (SENT)
W20260812 06:37:05.369263 21057 leader_election.cc:336] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3760db14241b4cbd9002a1018eccd803 (127.19.104.254:43859): Timed out: RequestConsensusVote RPC to 127.19.104.254:43859 timed out after 0.711s (SENT)
I20260812 06:37:06.323506 21731 raft_consensus.cc:493] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 790729e8843a44858d3eba2cfe10a572)
I20260812 06:37:06.323686 21731 raft_consensus.cc:515] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } }
I20260812 06:37:06.324276 21731 leader_election.cc:290] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3760db14241b4cbd9002a1018eccd803 (127.19.104.254:43859), 790729e8843a44858d3eba2cfe10a572 (127.19.104.252:43887)
I20260812 06:37:06.324836 21086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: false dest_uuid: "790729e8843a44858d3eba2cfe10a572" is_pre_election: true
W20260812 06:37:07.357532 20973 leader_election.cc:336] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3760db14241b4cbd9002a1018eccd803 (127.19.104.254:43859): Timed out: RequestConsensusVote RPC to 127.19.104.254:43859 timed out after 0.515s (SENT)
I20260812 06:37:07.357703 20973 leader_election.cc:304] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d670d64a80c24cacb1d5a6cbbff302e6; no voters: 3760db14241b4cbd9002a1018eccd803, 790729e8843a44858d3eba2cfe10a572
I20260812 06:37:07.358136 21736 raft_consensus.cc:2749] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 06:37:07.886866 21738 raft_consensus.cc:493] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 790729e8843a44858d3eba2cfe10a572)
I20260812 06:37:07.887045 21738 raft_consensus.cc:515] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } }
I20260812 06:37:07.887528 21738 leader_election.cc:290] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3760db14241b4cbd9002a1018eccd803 (127.19.104.254:43859), 790729e8843a44858d3eba2cfe10a572 (127.19.104.252:43887)
I20260812 06:37:07.888054 21086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: false dest_uuid: "790729e8843a44858d3eba2cfe10a572" is_pre_election: true
W20260812 06:37:08.537789 20973 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.104.254:43859 timed out after 0.649s (SENT) [suppressed 3 similar messages]
W20260812 06:37:09.188066 20973 leader_election.cc:336] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3760db14241b4cbd9002a1018eccd803 (127.19.104.254:43859): Timed out: RequestConsensusVote RPC to 127.19.104.254:43859 timed out after 0.649s (SENT)
I20260812 06:37:09.188190 20973 leader_election.cc:304] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d670d64a80c24cacb1d5a6cbbff302e6; no voters: 3760db14241b4cbd9002a1018eccd803, 790729e8843a44858d3eba2cfe10a572
I20260812 06:37:09.188663 21741 raft_consensus.cc:2749] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 06:37:11.600237 21746 raft_consensus.cc:493] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 790729e8843a44858d3eba2cfe10a572)
I20260812 06:37:11.600385 21746 raft_consensus.cc:515] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } }
I20260812 06:37:11.601032 21746 leader_election.cc:290] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3760db14241b4cbd9002a1018eccd803 (127.19.104.254:43859), 790729e8843a44858d3eba2cfe10a572 (127.19.104.252:43887)
I20260812 06:37:11.601517 21086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: false dest_uuid: "790729e8843a44858d3eba2cfe10a572" is_pre_election: true
W20260812 06:37:12.656301 20973 leader_election.cc:336] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3760db14241b4cbd9002a1018eccd803 (127.19.104.254:43859): Timed out: RequestConsensusVote RPC to 127.19.104.254:43859 timed out after 0.526s (SENT)
I20260812 06:37:12.656428 20973 leader_election.cc:304] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d670d64a80c24cacb1d5a6cbbff302e6; no voters: 3760db14241b4cbd9002a1018eccd803, 790729e8843a44858d3eba2cfe10a572
I20260812 06:37:12.656884 21749 raft_consensus.cc:2749] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 06:37:13.465324 21752 raft_consensus.cc:493] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 790729e8843a44858d3eba2cfe10a572)
I20260812 06:37:13.465478 21752 raft_consensus.cc:515] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } }
I20260812 06:37:13.466064 21752 leader_election.cc:290] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3760db14241b4cbd9002a1018eccd803 (127.19.104.254:43859), 790729e8843a44858d3eba2cfe10a572 (127.19.104.252:43887)
I20260812 06:37:13.466549 21086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: false dest_uuid: "790729e8843a44858d3eba2cfe10a572" is_pre_election: true
W20260812 06:37:13.498719 21672 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.19.104.254:43859)
I20260812 06:37:13.501207 21072 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55010:
table { table_name: "default.test_alter_table_sync" } new_table_name: "default.test_alter_table_sync_renamed" modify_external_catalogs: true
I20260812 06:37:13.517439 21774 tablet.cc:1724] T 6d8451a4697b459c9aab7125621c33f5 P 84c89832a92343f0b279baf89d1086a5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NOT NULL,
    PRIMARY KEY (key)
) version 1
I20260812 06:37:13.517813 21775 tablet.cc:1724] T 6d8451a4697b459c9aab7125621c33f5 P 2da2eeaba58644fea7afdd531ff9fdfc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NOT NULL,
    PRIMARY KEY (key)
) version 1
I20260812 06:37:13.520886 21776 tablet.cc:1724] T 6d8451a4697b459c9aab7125621c33f5 P 50ac34da4237497e9ef12dcc51e1ec65: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NOT NULL,
    PRIMARY KEY (key)
) version 1
I20260812 06:37:13.525293 21072 catalog_manager.cc:6264] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572: default.test_alter_table_sync_renamed [id=148a3a5cfcda40c5ab122f38f5a5e333] alter complete (version 1)
I20260812 06:37:13.529281 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 21184
I20260812 06:37:13.530467 20917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "790729e8843a44858d3eba2cfe10a572" candidate_term: 2 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: false dest_uuid: "3760db14241b4cbd9002a1018eccd803" is_pre_election: true
I20260812 06:37:13.532689 20907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: false dest_uuid: "3760db14241b4cbd9002a1018eccd803" is_pre_election: true
I20260812 06:37:13.532889 20907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: false dest_uuid: "3760db14241b4cbd9002a1018eccd803" is_pre_election: true
E20260812 06:37:13.533270 20892 catalog_manager.cc:5806] TS 50ac34da4237497e9ef12dcc51e1ec65 (127.19.104.193:42435) has reported a schema version greater than the current one for tablet 6d8451a4697b459c9aab7125621c33f5 (table default.test_alter_table_sync [id=148a3a5cfcda40c5ab122f38f5a5e333]). Expected version 0 got 1 (corruption)
I20260812 06:37:13.533330 20973 leader_election.cc:304] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d670d64a80c24cacb1d5a6cbbff302e6; no voters: 3760db14241b4cbd9002a1018eccd803, 790729e8843a44858d3eba2cfe10a572
I20260812 06:37:13.533571 21752 raft_consensus.cc:2749] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 06:37:13.536038 20997 raft_consensus.cc:1240] T 00000000000000000000000000000000 P d670d64a80c24cacb1d5a6cbbff302e6 [term 2 FOLLOWER]: Rejecting Update request from peer 3760db14241b4cbd9002a1018eccd803 for earlier term 1. Current term is 2. Ops: []
I20260812 06:37:13.536645 21511 consensus_queue.cc:1059] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 }, Status: INVALID_TERM, Last received: 1.7, Next index: 8, Last known committed idx: 10, Time since last communication: 0.000s
I20260812 06:37:13.536818 21511 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 06:37:13.536856 21511 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3760db14241b4cbd9002a1018eccd803, State: Running, Role: LEADER
E20260812 06:37:13.536909 20892 catalog_manager.cc:5806] TS 2da2eeaba58644fea7afdd531ff9fdfc (127.19.104.195:36369) has reported a schema version greater than the current one for tablet 6d8451a4697b459c9aab7125621c33f5 (table default.test_alter_table_sync [id=148a3a5cfcda40c5ab122f38f5a5e333]). Expected version 0 got 1 (corruption)
I20260812 06:37:13.536988 21511 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } }
I20260812 06:37:13.537214 21511 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:37:13.537864 21086 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [term 2 LEADER]: Rejecting Update request from peer 3760db14241b4cbd9002a1018eccd803 for earlier term 1. Current term is 2. Ops: []
I20260812 06:37:13.538589 20914 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [term 2 FOLLOWER]: Refusing update from remote peer 790729e8843a44858d3eba2cfe10a572: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20260812 06:37:13.538949 20916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "790729e8843a44858d3eba2cfe10a572" candidate_term: 2 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: false dest_uuid: "3760db14241b4cbd9002a1018eccd803"
I20260812 06:37:13.539167 21753 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 790729e8843a44858d3eba2cfe10a572 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 06:37:13.540045 20916 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 790729e8843a44858d3eba2cfe10a572 in term 2.
I20260812 06:37:13.540295 20911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "790729e8843a44858d3eba2cfe10a572" candidate_term: 2 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: false dest_uuid: "3760db14241b4cbd9002a1018eccd803"
I20260812 06:37:13.540830 20909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: false dest_uuid: "3760db14241b4cbd9002a1018eccd803" is_pre_election: true
I20260812 06:37:13.541059 20913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "790729e8843a44858d3eba2cfe10a572" candidate_term: 2 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: false dest_uuid: "3760db14241b4cbd9002a1018eccd803" is_pre_election: true
I20260812 06:37:13.541494 20908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: false dest_uuid: "3760db14241b4cbd9002a1018eccd803" is_pre_election: true
I20260812 06:37:13.542148 21511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 790729e8843a44858d3eba2cfe10a572. Latest consensus state: current_term: 2 leader_uuid: "790729e8843a44858d3eba2cfe10a572" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } } }
I20260812 06:37:13.542238 21511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:13.542366 21511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "790729e8843a44858d3eba2cfe10a572" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3760db14241b4cbd9002a1018eccd803" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 43859 } } peers { permanent_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44447 } } peers { permanent_uuid: "790729e8843a44858d3eba2cfe10a572" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 43887 } } }
I20260812 06:37:13.542443 21511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3760db14241b4cbd9002a1018eccd803 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:13.542800 20905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: false dest_uuid: "3760db14241b4cbd9002a1018eccd803" is_pre_election: true
I20260812 06:37:13.542994 20910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: false dest_uuid: "3760db14241b4cbd9002a1018eccd803" is_pre_election: true
I20260812 06:37:13.543991 20917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d670d64a80c24cacb1d5a6cbbff302e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: false dest_uuid: "3760db14241b4cbd9002a1018eccd803" is_pre_election: true
W20260812 06:37:13.545701 20882 catalog_manager.cc:5123] TS 84c89832a92343f0b279baf89d1086a5 (127.19.104.194:35279): alter failed for tablet 6d8451a4697b459c9aab7125621c33f5 (table default.test_alter_table_sync [id=148a3a5cfcda40c5ab122f38f5a5e333]),no further retry: Invalid argument: Tablet has a newer schema
I20260812 06:37:13.553052 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 21352
W20260812 06:37:13.557751 20885 connection.cc:573] server connection from 127.19.104.193:42747 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 06:37:13.564702 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 21506
I20260812 06:37:13.574069 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 20864
I20260812 06:37:13.584640 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 20948
I20260812 06:37:13.592614 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 21029
2026-08-12T06:37:13Z chronyd exiting
[       OK ] HmsConfigurations/MasterFailoverTest.TestRenameTableSync/0 (11673 ms)
[ RUN      ] HmsConfigurations/MasterFailoverTest.TestMasterUUIDResolution/0
2026-08-12T06:37:13Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:37:13Z Disabled control of system clock
I20260812 06:37:13.627982 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.254:36255
--webserver_interface=127.19.104.254
--webserver_port=0
--builtin_ntp_servers=127.19.104.212:33233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.104.254:36255,127.19.104.253:44703,127.19.104.252:41225
--leader_failure_max_missed_heartbeat_periods=1.0 with env {}
W20260812 06:37:13.733332 21785 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:13.733556 21785 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:13.733596 21785 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:13.736940 21785 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:37:13.737005 21785 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:13.737026 21785 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:37:13.737043 21785 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:37:13.740891 21785 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:33233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.104.254:36255,127.19.104.253:44703,127.19.104.252:41225
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.254:36255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.104.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.21785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:13.741977 21785 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:13.743111 21785 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:13.748881 21795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:13.748925 21792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:13.748941 21785 server_base.cc:1061] running on GCE node
W20260812 06:37:13.748876 21791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:13.749768 21785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:13.750638 21785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:13.751830 21785 hybrid_clock.cc:648] HybridClock initialized: now 1786516633751810 us; error 42 us; skew 500 ppm
I20260812 06:37:13.753674 21785 webserver.cc:533] Webserver started at http://127.19.104.254:46715/ using document root <none> and password file <none>
I20260812 06:37:13.754253 21785 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:13.754345 21785 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:13.754575 21785 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:13.756111 21785 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/data/instance:
uuid: "022dbc1ff8184ac89cee789ace581616"
format_stamp: "Formatted at 2026-08-12 06:37:13 on dist-test-slave-8n49"
I20260812 06:37:13.756668 21785 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/wal/instance:
uuid: "022dbc1ff8184ac89cee789ace581616"
format_stamp: "Formatted at 2026-08-12 06:37:13 on dist-test-slave-8n49"
I20260812 06:37:13.759837 21785 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.000s
I20260812 06:37:13.762002 21801 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:13.762954 21785 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 06:37:13.763098 21785 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/wal
uuid: "022dbc1ff8184ac89cee789ace581616"
format_stamp: "Formatted at 2026-08-12 06:37:13 on dist-test-slave-8n49"
I20260812 06:37:13.763204 21785 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:13.778934 21785 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:13.779639 21785 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:13.779843 21785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:13.787362 21858 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.254:36255 every 8 connection(s)
I20260812 06:37:13.787366 21785 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.254:36255
I20260812 06:37:13.788422 21785 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/data/info.pb
I20260812 06:37:13.791222 21859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:13.793852 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 21785
I20260812 06:37:13.793974 19875 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/wal/instance
I20260812 06:37:13.794736 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.253:44703
--webserver_interface=127.19.104.253
--webserver_port=0
--builtin_ntp_servers=127.19.104.212:33233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.104.254:36255,127.19.104.253:44703,127.19.104.252:41225
--leader_failure_max_missed_heartbeat_periods=1.0 with env {}
I20260812 06:37:13.794994 21859 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:13.809441 21859 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } has no permanent_uuid. Determining permanent_uuid...
W20260812 06:37:13.810240 21804 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.104.253:44703: connect: Connection refused (error 111)
W20260812 06:37:13.811121 21859 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.253:44703: Network error: Client connection negotiation failed: client connection to 127.19.104.253:44703: connect: Connection refused (error 111)
I20260812 06:37:13.860365 21859 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } attempt: 1
W20260812 06:37:13.862267 21859 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.253:44703: Network error: Client connection negotiation failed: client connection to 127.19.104.253:44703: connect: Connection refused (error 111)
W20260812 06:37:13.904628 21861 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:13.904874 21861 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:13.904919 21861 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:13.908622 21861 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:37:13.908690 21861 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:13.908713 21861 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:37:13.908733 21861 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:37:13.913249 21861 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:33233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.104.254:36255,127.19.104.253:44703,127.19.104.252:41225
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.253:44703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.104.253
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.21861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:13.914476 21861 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:13.915692 21861 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:13.921790 21871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:13.921794 21873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:13.922057 21861 server_base.cc:1061] running on GCE node
W20260812 06:37:13.921792 21869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:13.922559 21861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:13.923460 21861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:13.924656 21861 hybrid_clock.cc:648] HybridClock initialized: now 1786516633924629 us; error 43 us; skew 500 ppm
I20260812 06:37:13.926476 21861 webserver.cc:533] Webserver started at http://127.19.104.253:37025/ using document root <none> and password file <none>
I20260812 06:37:13.927070 21861 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:13.927129 21861 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:13.927290 21861 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:13.928911 21861 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/data/instance:
uuid: "4e105824134f4416bac864e56405136e"
format_stamp: "Formatted at 2026-08-12 06:37:13 on dist-test-slave-8n49"
I20260812 06:37:13.929354 21861 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/wal/instance:
uuid: "4e105824134f4416bac864e56405136e"
format_stamp: "Formatted at 2026-08-12 06:37:13 on dist-test-slave-8n49"
I20260812 06:37:13.930461 21859 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } attempt: 2
W20260812 06:37:13.931917 21859 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.253:44703: Network error: Client connection negotiation failed: client connection to 127.19.104.253:44703: connect: Connection refused (error 111)
I20260812 06:37:13.932818 21861 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 06:37:13.934885 21879 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:13.935876 21861 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:13.936021 21861 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/wal
uuid: "4e105824134f4416bac864e56405136e"
format_stamp: "Formatted at 2026-08-12 06:37:13 on dist-test-slave-8n49"
I20260812 06:37:13.936127 21861 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:13.952010 21861 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:13.952801 21861 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:13.953001 21861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:13.960619 21941 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.253:44703 every 8 connection(s)
I20260812 06:37:13.960619 21861 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.253:44703
I20260812 06:37:13.961717 21861 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/data/info.pb
I20260812 06:37:13.961947 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 21861
I20260812 06:37:13.962059 19875 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/wal/instance
I20260812 06:37:13.962756 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.252:41225
--webserver_interface=127.19.104.252
--webserver_port=0
--builtin_ntp_servers=127.19.104.212:33233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.104.254:36255,127.19.104.253:44703,127.19.104.252:41225
--leader_failure_max_missed_heartbeat_periods=1.0 with env {}
I20260812 06:37:13.964680 21942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:13.969659 21942 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:13.979984 21942 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:13.986441 21942 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } has no permanent_uuid. Determining permanent_uuid...
W20260812 06:37:13.987294 21882 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.104.252:41225: connect: Connection refused (error 111)
W20260812 06:37:13.988250 21942 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.252:41225: Network error: Client connection negotiation failed: client connection to 127.19.104.252:41225: connect: Connection refused (error 111)
I20260812 06:37:14.023120 21859 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } attempt: 3
I20260812 06:37:14.028474 21859 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } has no permanent_uuid. Determining permanent_uuid...
W20260812 06:37:14.029917 21859 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.252:41225: Network error: Client connection negotiation failed: client connection to 127.19.104.252:41225: connect: Connection refused (error 111)
I20260812 06:37:14.037493 21942 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } attempt: 1
W20260812 06:37:14.038910 21942 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.252:41225: Network error: Client connection negotiation failed: client connection to 127.19.104.252:41225: connect: Connection refused (error 111)
I20260812 06:37:14.061103 21859 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } attempt: 1
W20260812 06:37:14.062748 21859 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.252:41225: Network error: Client connection negotiation failed: client connection to 127.19.104.252:41225: connect: Connection refused (error 111)
W20260812 06:37:14.076829 21945 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:14.077064 21945 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:14.077104 21945 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:14.080600 21945 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:37:14.080672 21945 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:14.080694 21945 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:37:14.080713 21945 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:37:14.084723 21945 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:33233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.104.254:36255,127.19.104.253:44703,127.19.104.252:41225
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.252:41225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.104.252
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.21945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:14.085912 21945 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:14.087081 21945 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:14.092572 21958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:14.092607 21956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:14.092608 21955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:14.092841 21945 server_base.cc:1061] running on GCE node
I20260812 06:37:14.093372 21945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:14.094326 21945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:14.095508 21945 hybrid_clock.cc:648] HybridClock initialized: now 1786516634095500 us; error 34 us; skew 500 ppm
I20260812 06:37:14.097316 21945 webserver.cc:533] Webserver started at http://127.19.104.252:45105/ using document root <none> and password file <none>
I20260812 06:37:14.097900 21945 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:14.097983 21945 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:14.098219 21945 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:14.099798 21945 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/data/instance:
uuid: "c4d868d998ab471eb47febaf08b59b67"
format_stamp: "Formatted at 2026-08-12 06:37:14 on dist-test-slave-8n49"
I20260812 06:37:14.100286 21945 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/wal/instance:
uuid: "c4d868d998ab471eb47febaf08b59b67"
format_stamp: "Formatted at 2026-08-12 06:37:14 on dist-test-slave-8n49"
I20260812 06:37:14.103660 21945 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.001s	sys 0.001s
I20260812 06:37:14.105865 21964 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:14.106809 21945 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 06:37:14.106962 21945 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/wal
uuid: "c4d868d998ab471eb47febaf08b59b67"
format_stamp: "Formatted at 2026-08-12 06:37:14 on dist-test-slave-8n49"
I20260812 06:37:14.107066 21945 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:14.107134 21942 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } attempt: 2
W20260812 06:37:14.108636 21942 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.252:41225: Network error: Client connection negotiation failed: client connection to 127.19.104.252:41225: connect: Connection refused (error 111)
I20260812 06:37:14.137455 21945 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:14.137969 21859 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } attempt: 2
I20260812 06:37:14.138244 21945 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:14.138473 21945 kserver.cc:163] Server-wide thread pool size limit: 3276
W20260812 06:37:14.139534 21859 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.252:41225: Network error: Client connection negotiation failed: client connection to 127.19.104.252:41225: connect: Connection refused (error 111)
I20260812 06:37:14.146129 21945 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.252:41225
I20260812 06:37:14.146127 22022 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.252:41225 every 8 connection(s)
I20260812 06:37:14.147141 21945 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/data/info.pb
I20260812 06:37:14.149947 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 21945
I20260812 06:37:14.150063 19875 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/wal/instance
I20260812 06:37:14.150098 22024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:14.154264 22024 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:14.163944 22024 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:14.167727 22024 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:14.173806 22024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67: Bootstrap starting.
I20260812 06:37:14.175997 22024 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:14.176796 22024 log.cc:826] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:14.178400 22024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67: No bootstrap required, opened a new log
I20260812 06:37:14.181731 22024 raft_consensus.cc:359] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } }
I20260812 06:37:14.181900 22024 raft_consensus.cc:385] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:14.181973 22024 raft_consensus.cc:740] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d868d998ab471eb47febaf08b59b67, State: Initialized, Role: FOLLOWER
I20260812 06:37:14.182440 22024 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } }
I20260812 06:37:14.182920 22032 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } } }
I20260812 06:37:14.183061 22032 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:14.183537 22024 sys_catalog.cc:565] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 06:37:14.188473 22044 catalog_manager.cc:1594] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:37:14.188597 22044 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:37:14.187770 21945 master_runner.cc:187] Error getting master registration for 127.19.104.254:36255: OK, instance_id { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" instance_seqno: 1786516633785278 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 06:37:14.189695 21945 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260812 06:37:14.199823 21942 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } attempt: 3
I20260812 06:37:14.206243 21942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e: Bootstrap starting.
I20260812 06:37:14.208773 21942 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:14.209502 21942 log.cc:826] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:14.211184 21942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e: No bootstrap required, opened a new log
I20260812 06:37:14.214099 21942 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } }
I20260812 06:37:14.214277 21942 raft_consensus.cc:385] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:14.214370 21942 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e105824134f4416bac864e56405136e, State: Initialized, Role: FOLLOWER
I20260812 06:37:14.214774 21942 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } }
I20260812 06:37:14.215240 22047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } } }
I20260812 06:37:14.215351 22047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:14.215924 21942 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [sys.catalog]: configured and running, proceeding with master startup.
W20260812 06:37:14.221628 22060 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:37:14.221707 22060 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:37:14.220134 21861 master_runner.cc:187] Error getting master registration for 127.19.104.254:36255: OK, instance_id { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" instance_seqno: 1786516633785278 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 06:37:14.222221 21861 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260812 06:37:14.238730 21859 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } attempt: 3
I20260812 06:37:14.245114 21859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616: Bootstrap starting.
I20260812 06:37:14.248136 21859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:14.248929 21859 log.cc:826] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:14.250607 21859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616: No bootstrap required, opened a new log
I20260812 06:37:14.253512 21859 raft_consensus.cc:359] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } }
I20260812 06:37:14.253692 21859 raft_consensus.cc:385] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:14.253767 21859 raft_consensus.cc:740] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 022dbc1ff8184ac89cee789ace581616, State: Initialized, Role: FOLLOWER
I20260812 06:37:14.254226 21859 consensus_queue.cc:260] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } }
I20260812 06:37:14.254719 22063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } } }
I20260812 06:37:14.254881 22063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:14.255415 21859 sys_catalog.cc:565] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 06:37:14.262097 22075 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:37:14.262257 22075 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:37:14.264750 21785 master_runner.cc:264] No leader master found from master 022dbc1ff8184ac89cee789ace581616
I20260812 06:37:14.264817 21785 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260812 06:37:14.281338 21808 tablet.cc:2406] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:37:14.390022 21971 tablet.cc:2406] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:37:14.454437 21886 tablet.cc:2406] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:37:14.479082 22032 raft_consensus.cc:493] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:37:14.479302 22032 raft_consensus.cc:515] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } }
I20260812 06:37:14.480295 22032 leader_election.cc:290] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 022dbc1ff8184ac89cee789ace581616 (127.19.104.254:36255), 4e105824134f4416bac864e56405136e (127.19.104.253:44703)
I20260812 06:37:14.481017 21831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c4d868d998ab471eb47febaf08b59b67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "022dbc1ff8184ac89cee789ace581616" is_pre_election: true
I20260812 06:37:14.481040 21909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c4d868d998ab471eb47febaf08b59b67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e105824134f4416bac864e56405136e" is_pre_election: true
I20260812 06:37:14.481312 21831 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d868d998ab471eb47febaf08b59b67 in term 0.
I20260812 06:37:14.481329 21909 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d868d998ab471eb47febaf08b59b67 in term 0.
I20260812 06:37:14.481778 21967 leader_election.cc:304] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 022dbc1ff8184ac89cee789ace581616, c4d868d998ab471eb47febaf08b59b67; no voters: 
I20260812 06:37:14.482225 22032 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:37:14.482319 22032 raft_consensus.cc:493] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:37:14.482363 22032 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:14.483428 22032 raft_consensus.cc:515] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } }
I20260812 06:37:14.483819 22032 leader_election.cc:290] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [CANDIDATE]: Term 1 election: Requested vote from peers 022dbc1ff8184ac89cee789ace581616 (127.19.104.254:36255), 4e105824134f4416bac864e56405136e (127.19.104.253:44703)
I20260812 06:37:14.484220 21831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c4d868d998ab471eb47febaf08b59b67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "022dbc1ff8184ac89cee789ace581616"
I20260812 06:37:14.484300 21909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c4d868d998ab471eb47febaf08b59b67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e105824134f4416bac864e56405136e"
I20260812 06:37:14.484381 21831 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:14.484442 21909 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:14.485412 21831 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d868d998ab471eb47febaf08b59b67 in term 1.
I20260812 06:37:14.485509 21909 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d868d998ab471eb47febaf08b59b67 in term 1.
I20260812 06:37:14.485813 21967 leader_election.cc:304] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 022dbc1ff8184ac89cee789ace581616, c4d868d998ab471eb47febaf08b59b67; no voters: 
I20260812 06:37:14.485989 22032 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:37:14.486243 22032 raft_consensus.cc:697] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [term 1 LEADER]: Becoming Leader. State: Replica: c4d868d998ab471eb47febaf08b59b67, State: Running, Role: LEADER
I20260812 06:37:14.486581 22032 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } }
I20260812 06:37:14.488538 22077 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4d868d998ab471eb47febaf08b59b67. Latest consensus state: current_term: 1 leader_uuid: "c4d868d998ab471eb47febaf08b59b67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } } }
I20260812 06:37:14.488670 22077 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:14.489049 22079 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:37:14.489954 22079 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:37:14.494465 21831 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [term 1 FOLLOWER]: Refusing update from remote peer c4d868d998ab471eb47febaf08b59b67: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:37:14.494601 21909 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [term 1 FOLLOWER]: Refusing update from remote peer c4d868d998ab471eb47febaf08b59b67: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:37:14.495157 22077 consensus_queue.cc:1048] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:14.495401 22032 consensus_queue.cc:1048] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:14.498939 22047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4d868d998ab471eb47febaf08b59b67. Latest consensus state: current_term: 1 leader_uuid: "c4d868d998ab471eb47febaf08b59b67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } } }
I20260812 06:37:14.499053 22047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:14.499351 22063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4d868d998ab471eb47febaf08b59b67. Latest consensus state: current_term: 1 leader_uuid: "c4d868d998ab471eb47febaf08b59b67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } } }
I20260812 06:37:14.499477 22063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:14.501648 22086 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c4d868d998ab471eb47febaf08b59b67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } } }
I20260812 06:37:14.501750 22086 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:14.501885 22032 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c4d868d998ab471eb47febaf08b59b67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } } }
I20260812 06:37:14.501964 22032 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:14.502168 22063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c4d868d998ab471eb47febaf08b59b67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } } }
I20260812 06:37:14.502157 22047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c4d868d998ab471eb47febaf08b59b67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } } }
I20260812 06:37:14.502251 22047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:14.502251 22063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:14.502522 22081 mvcc.cc:204] Tried to move back new op lower bound from 7317572134882619392 to 7317572134859845632. Current Snapshot: MvccSnapshot[applied={T|T < 7317572134882619392}]
I20260812 06:37:14.503324 22079 catalog_manager.cc:1383] Generated new cluster ID: 6c8876ec68914c3c8d78cbedf403c311
I20260812 06:37:14.503386 22079 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:37:14.515344 22079 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 06:37:14.516414 22079 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:37:14.525388 22079 catalog_manager.cc:6092] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67: Generated new TSK 0
I20260812 06:37:14.526171 22079 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:37:14.552016 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.104.193:0
--local_ip_for_outbound_sockets=127.19.104.193
--webserver_interface=127.19.104.193
--webserver_port=0
--tserver_master_addrs=127.19.104.254:36255,127.19.104.253:44703,127.19.104.252:41225
--builtin_ntp_servers=127.19.104.212:33233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
W20260812 06:37:14.657037 22090 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:14.657258 22090 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:14.657354 22090 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:14.660818 22090 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:14.660935 22090 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.104.193
I20260812 06:37:14.664815 22090 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:33233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.104.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.104.193
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.19.104.254:36255,127.19.104.253:44703,127.19.104.252:41225
--never_fsync=true
--heap_profile_path=/tmp/kudu.22090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.104.193

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:14.665922 22090 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:14.667062 22090 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:14.673322 22095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:14.673322 22099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:14.673341 22097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:14.673799 22090 server_base.cc:1061] running on GCE node
I20260812 06:37:14.674175 22090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:14.674708 22090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:14.675889 22090 hybrid_clock.cc:648] HybridClock initialized: now 1786516634675873 us; error 32 us; skew 500 ppm
I20260812 06:37:14.677613 22090 webserver.cc:533] Webserver started at http://127.19.104.193:40249/ using document root <none> and password file <none>
I20260812 06:37:14.678179 22090 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:14.678267 22090 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:14.678489 22090 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:14.679981 22090 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/data/instance:
uuid: "0d0bcbf7fecf4f0c889c06f8b8960c16"
format_stamp: "Formatted at 2026-08-12 06:37:14 on dist-test-slave-8n49"
I20260812 06:37:14.680472 22090 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/wal/instance:
uuid: "0d0bcbf7fecf4f0c889c06f8b8960c16"
format_stamp: "Formatted at 2026-08-12 06:37:14 on dist-test-slave-8n49"
I20260812 06:37:14.683643 22090 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.001s
I20260812 06:37:14.685842 22108 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:14.686846 22090 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 06:37:14.686980 22090 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/wal
uuid: "0d0bcbf7fecf4f0c889c06f8b8960c16"
format_stamp: "Formatted at 2026-08-12 06:37:14 on dist-test-slave-8n49"
I20260812 06:37:14.687077 22090 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:14.714274 22090 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:14.714951 22090 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:14.715155 22090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:14.715739 22090 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:14.716917 22090 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:14.716990 22090 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:14.717056 22090 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:14.717108 22090 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:14.726711 22090 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.193:34847
I20260812 06:37:14.726740 22235 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.193:34847 every 8 connection(s)
I20260812 06:37:14.727684 22090 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/data/info.pb
I20260812 06:37:14.728384 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 22090
I20260812 06:37:14.728513 19875 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/wal/instance
I20260812 06:37:14.731452 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.104.194:0
--local_ip_for_outbound_sockets=127.19.104.194
--webserver_interface=127.19.104.194
--webserver_port=0
--tserver_master_addrs=127.19.104.254:36255,127.19.104.253:44703,127.19.104.252:41225
--builtin_ntp_servers=127.19.104.212:33233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260812 06:37:14.738552 22238 heartbeater.cc:344] Connected to a master server at 127.19.104.252:41225
I20260812 06:37:14.738855 22238 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:14.739560 22238 heartbeater.cc:507] Master 127.19.104.252:41225 requested a full tablet report, sending...
I20260812 06:37:14.741245 22237 heartbeater.cc:344] Connected to a master server at 127.19.104.254:36255
I20260812 06:37:14.741345 22237 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:14.741569 22237 heartbeater.cc:507] Master 127.19.104.254:36255 requested a full tablet report, sending...
I20260812 06:37:14.741693 21984 ts_manager.cc:194] Registered new tserver with Master: 0d0bcbf7fecf4f0c889c06f8b8960c16 (127.19.104.193:34847)
I20260812 06:37:14.742055 22239 heartbeater.cc:344] Connected to a master server at 127.19.104.253:44703
I20260812 06:37:14.742136 22239 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:14.742318 22239 heartbeater.cc:507] Master 127.19.104.253:44703 requested a full tablet report, sending...
I20260812 06:37:14.742730 21819 ts_manager.cc:194] Registered new tserver with Master: 0d0bcbf7fecf4f0c889c06f8b8960c16 (127.19.104.193:34847)
I20260812 06:37:14.743247 21984 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.104.193:37869
I20260812 06:37:14.743559 21899 ts_manager.cc:194] Registered new tserver with Master: 0d0bcbf7fecf4f0c889c06f8b8960c16 (127.19.104.193:34847)
W20260812 06:37:14.839514 22247 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:14.839722 22247 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:14.839773 22247 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:14.843133 22247 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:14.843286 22247 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.104.194
I20260812 06:37:14.847069 22247 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:33233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.104.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.104.194
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.19.104.254:36255,127.19.104.253:44703,127.19.104.252:41225
--never_fsync=true
--heap_profile_path=/tmp/kudu.22247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.104.194

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:14.848217 22247 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:14.849334 22247 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:14.855713 22254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:14.855759 22253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:14.855885 22247 server_base.cc:1061] running on GCE node
W20260812 06:37:14.855713 22256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:14.856376 22247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:14.857019 22247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:14.858183 22247 hybrid_clock.cc:648] HybridClock initialized: now 1786516634858171 us; error 34 us; skew 500 ppm
I20260812 06:37:14.859918 22247 webserver.cc:533] Webserver started at http://127.19.104.194:44831/ using document root <none> and password file <none>
I20260812 06:37:14.860494 22247 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:14.860605 22247 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:14.860827 22247 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:14.862365 22247 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/data/instance:
uuid: "6b28823152394f21ac8a9ad43f660111"
format_stamp: "Formatted at 2026-08-12 06:37:14 on dist-test-slave-8n49"
I20260812 06:37:14.862855 22247 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/wal/instance:
uuid: "6b28823152394f21ac8a9ad43f660111"
format_stamp: "Formatted at 2026-08-12 06:37:14 on dist-test-slave-8n49"
I20260812 06:37:14.866104 22247 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.000s	sys 0.005s
I20260812 06:37:14.868391 22264 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:14.869524 22247 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.002s
I20260812 06:37:14.869652 22247 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/wal
uuid: "6b28823152394f21ac8a9ad43f660111"
format_stamp: "Formatted at 2026-08-12 06:37:14 on dist-test-slave-8n49"
I20260812 06:37:14.869751 22247 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:14.882114 22247 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:14.882745 22247 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:14.882953 22247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:14.883528 22247 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:14.884604 22247 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:14.884675 22247 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:14.884780 22247 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:14.884817 22247 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:14.894433 22247 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.194:38507
I20260812 06:37:14.894469 22395 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.194:38507 every 8 connection(s)
I20260812 06:37:14.895354 22247 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/data/info.pb
I20260812 06:37:14.898681 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 22247
I20260812 06:37:14.898810 19875 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/wal/instance
I20260812 06:37:14.901700 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.104.195:0
--local_ip_for_outbound_sockets=127.19.104.195
--webserver_interface=127.19.104.195
--webserver_port=0
--tserver_master_addrs=127.19.104.254:36255,127.19.104.253:44703,127.19.104.252:41225
--builtin_ntp_servers=127.19.104.212:33233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260812 06:37:14.905526 22397 heartbeater.cc:344] Connected to a master server at 127.19.104.252:41225
I20260812 06:37:14.905609 22398 heartbeater.cc:344] Connected to a master server at 127.19.104.253:44703
I20260812 06:37:14.905786 22398 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:14.905786 22397 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:14.906401 22398 heartbeater.cc:507] Master 127.19.104.253:44703 requested a full tablet report, sending...
I20260812 06:37:14.906402 22397 heartbeater.cc:507] Master 127.19.104.252:41225 requested a full tablet report, sending...
I20260812 06:37:14.906805 22396 heartbeater.cc:344] Connected to a master server at 127.19.104.254:36255
I20260812 06:37:14.906870 22396 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:14.907061 22396 heartbeater.cc:507] Master 127.19.104.254:36255 requested a full tablet report, sending...
I20260812 06:37:14.907617 21984 ts_manager.cc:194] Registered new tserver with Master: 6b28823152394f21ac8a9ad43f660111 (127.19.104.194:38507)
I20260812 06:37:14.907662 21899 ts_manager.cc:194] Registered new tserver with Master: 6b28823152394f21ac8a9ad43f660111 (127.19.104.194:38507)
I20260812 06:37:14.908355 21984 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.104.194:41361
I20260812 06:37:14.908870 21819 ts_manager.cc:194] Registered new tserver with Master: 6b28823152394f21ac8a9ad43f660111 (127.19.104.194:38507)
I20260812 06:37:15.026748 22413 raft_consensus.cc:493] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c4d868d998ab471eb47febaf08b59b67)
I20260812 06:37:15.026911 22413 raft_consensus.cc:515] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } }
I20260812 06:37:15.027756 22413 leader_election.cc:290] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e105824134f4416bac864e56405136e (127.19.104.253:44703), c4d868d998ab471eb47febaf08b59b67 (127.19.104.252:41225)
I20260812 06:37:15.028323 21909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "022dbc1ff8184ac89cee789ace581616" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e105824134f4416bac864e56405136e" is_pre_election: true
I20260812 06:37:15.028357 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "022dbc1ff8184ac89cee789ace581616" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c4d868d998ab471eb47febaf08b59b67" is_pre_election: true
I20260812 06:37:15.028929 21804 leader_election.cc:304] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 022dbc1ff8184ac89cee789ace581616; no voters: 4e105824134f4416bac864e56405136e, c4d868d998ab471eb47febaf08b59b67
I20260812 06:37:15.029171 22413 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 06:37:15.030195 22403 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:15.030385 22403 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:15.030498 22403 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:15.033903 22403 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:15.034057 22403 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.104.195
I20260812 06:37:15.037912 22403 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:33233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.104.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.104.195
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.19.104.254:36255,127.19.104.253:44703,127.19.104.252:41225
--never_fsync=true
--heap_profile_path=/tmp/kudu.22403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.104.195

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:15.038985 22403 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:15.040094 22403 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:15.046275 22417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:15.046255 22416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:15.046451 22403 server_base.cc:1061] running on GCE node
W20260812 06:37:15.046285 22419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:15.046868 22403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:15.047495 22403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:15.048728 22403 hybrid_clock.cc:648] HybridClock initialized: now 1786516635048712 us; error 36 us; skew 500 ppm
I20260812 06:37:15.050695 22403 webserver.cc:533] Webserver started at http://127.19.104.195:34857/ using document root <none> and password file <none>
I20260812 06:37:15.051316 22403 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:15.051369 22403 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:15.051594 22403 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:15.053184 22403 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/data/instance:
uuid: "f5543c77410446cea27a857293f6127e"
format_stamp: "Formatted at 2026-08-12 06:37:15 on dist-test-slave-8n49"
I20260812 06:37:15.053656 22403 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/wal/instance:
uuid: "f5543c77410446cea27a857293f6127e"
format_stamp: "Formatted at 2026-08-12 06:37:15 on dist-test-slave-8n49"
I20260812 06:37:15.056860 22403 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 06:37:15.058983 22425 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:15.059937 22403 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:15.060067 22403 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/wal
uuid: "f5543c77410446cea27a857293f6127e"
format_stamp: "Formatted at 2026-08-12 06:37:15 on dist-test-slave-8n49"
I20260812 06:37:15.060165 22403 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:15.070761 22403 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:15.071377 22403 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:15.071569 22403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:15.072101 22403 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:15.073221 22403 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:15.073292 22403 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:15.073359 22403 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:15.073410 22403 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:15.082569 22403 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.195:46883
I20260812 06:37:15.082597 22559 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.195:46883 every 8 connection(s)
I20260812 06:37:15.083477 22403 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/data/info.pb
I20260812 06:37:15.091995 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 22403
I20260812 06:37:15.092147 19875 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/wal/instance
I20260812 06:37:15.092764 22561 heartbeater.cc:344] Connected to a master server at 127.19.104.252:41225
I20260812 06:37:15.092861 22560 heartbeater.cc:344] Connected to a master server at 127.19.104.254:36255
I20260812 06:37:15.093001 22561 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:15.093001 22560 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:15.093645 22561 heartbeater.cc:507] Master 127.19.104.252:41225 requested a full tablet report, sending...
I20260812 06:37:15.093645 22560 heartbeater.cc:507] Master 127.19.104.254:36255 requested a full tablet report, sending...
I20260812 06:37:15.093884 22562 heartbeater.cc:344] Connected to a master server at 127.19.104.253:44703
I20260812 06:37:15.093964 22562 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:15.094120 22562 heartbeater.cc:507] Master 127.19.104.253:44703 requested a full tablet report, sending...
I20260812 06:37:15.094732 21984 ts_manager.cc:194] Registered new tserver with Master: f5543c77410446cea27a857293f6127e (127.19.104.195:46883)
I20260812 06:37:15.094833 21819 ts_manager.cc:194] Registered new tserver with Master: f5543c77410446cea27a857293f6127e (127.19.104.195:46883)
I20260812 06:37:15.095465 21984 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.104.195:45171
I20260812 06:37:15.096004 21899 ts_manager.cc:194] Registered new tserver with Master: f5543c77410446cea27a857293f6127e (127.19.104.195:46883)
I20260812 06:37:15.098613 19875 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 06:37:15.119472 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 22090
I20260812 06:37:15.132026 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 22247
I20260812 06:37:15.139343 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 22403
I20260812 06:37:15.149865 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 21785
I20260812 06:37:15.158049 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 21861
I20260812 06:37:15.168268 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 21945
I20260812 06:37:15.177716 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.254:36255
--webserver_interface=127.19.104.254
--webserver_port=46715
--builtin_ntp_servers=127.19.104.212:33233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.104.254:36255,127.19.104.253:44703,127.19.104.252:41225
--leader_failure_max_missed_heartbeat_periods=1.0 with env {}
W20260812 06:37:15.281095 22582 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:15.281354 22582 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:15.281443 22582 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:15.284770 22582 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:37:15.284847 22582 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:15.284883 22582 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:37:15.284943 22582 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:37:15.288769 22582 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:33233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.104.254:36255,127.19.104.253:44703,127.19.104.252:41225
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.254:36255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.104.254
--webserver_port=46715
--never_fsync=true
--heap_profile_path=/tmp/kudu.22582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:15.289862 22582 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:15.291052 22582 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:15.296794 22587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:15.296782 22588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:15.296757 22591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:15.297055 22582 server_base.cc:1061] running on GCE node
I20260812 06:37:15.297637 22582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:15.298563 22582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:15.299793 22582 hybrid_clock.cc:648] HybridClock initialized: now 1786516635299779 us; error 38 us; skew 500 ppm
I20260812 06:37:15.301743 22582 webserver.cc:533] Webserver started at http://127.19.104.254:46715/ using document root <none> and password file <none>
I20260812 06:37:15.302335 22582 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:15.302448 22582 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:15.305949 22582 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.002s	sys 0.002s
I20260812 06:37:15.307982 22597 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:15.309135 22582 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:15.309288 22582 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/wal
uuid: "022dbc1ff8184ac89cee789ace581616"
format_stamp: "Formatted at 2026-08-12 06:37:13 on dist-test-slave-8n49"
I20260812 06:37:15.309723 22582 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:15.331040 22582 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:15.331835 22582 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:15.332067 22582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:15.340338 22658 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.254:36255 every 8 connection(s)
I20260812 06:37:15.340339 22582 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.254:36255
I20260812 06:37:15.341603 22582 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-0/data/info.pb
I20260812 06:37:15.343466 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 22582
I20260812 06:37:15.343889 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.253:44703
--webserver_interface=127.19.104.253
--webserver_port=37025
--builtin_ntp_servers=127.19.104.212:33233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.104.254:36255,127.19.104.253:44703,127.19.104.252:41225
--leader_failure_max_missed_heartbeat_periods=1.0 with env {}
I20260812 06:37:15.345759 22659 sys_catalog.cc:263] Verifying existing consensus state
I20260812 06:37:15.347373 22659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616: Bootstrap starting.
I20260812 06:37:15.361284 22659 log.cc:826] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:15.366905 22659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:37:15.367285 22659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616: Bootstrap complete.
I20260812 06:37:15.372450 22659 raft_consensus.cc:359] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } }
I20260812 06:37:15.372977 22659 raft_consensus.cc:740] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 022dbc1ff8184ac89cee789ace581616, State: Initialized, Role: FOLLOWER
I20260812 06:37:15.373579 22659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } }
I20260812 06:37:15.374106 22664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } } }
I20260812 06:37:15.374250 22664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:15.374892 22659 sys_catalog.cc:565] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:37:15.379906 22676 catalog_manager.cc:1295] Loaded cluster ID: 6c8876ec68914c3c8d78cbedf403c311
I20260812 06:37:15.380021 22676 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616: loading cluster ID for follower catalog manager: success
I20260812 06:37:15.382313 22676 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616: acquiring CA information for follower catalog manager: success
I20260812 06:37:15.383524 22676 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:37:15.388396 22582 master_runner.cc:187] Error getting master registration for 127.19.104.253:44703: Network error: Client connection negotiation failed: client connection to 127.19.104.253:44703: connect: Connection refused (error 111), 
I20260812 06:37:15.389899 22582 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260812 06:37:15.473301 22662 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:15.473529 22662 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:15.473575 22662 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:15.477481 22662 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:37:15.477545 22662 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:15.477567 22662 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:37:15.477615 22662 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:37:15.481979 22662 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:33233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.104.254:36255,127.19.104.253:44703,127.19.104.252:41225
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.253:44703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.104.253
--webserver_port=37025
--never_fsync=true
--heap_profile_path=/tmp/kudu.22662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:15.483147 22662 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:15.484350 22662 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:15.490173 22685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:15.490249 22686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:15.490417 22662 server_base.cc:1061] running on GCE node
W20260812 06:37:15.490506 22689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:15.491034 22662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:15.491905 22662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:15.493106 22662 hybrid_clock.cc:648] HybridClock initialized: now 1786516635493080 us; error 41 us; skew 500 ppm
I20260812 06:37:15.495074 22662 webserver.cc:533] Webserver started at http://127.19.104.253:37025/ using document root <none> and password file <none>
I20260812 06:37:15.495666 22662 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:15.495779 22662 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:15.499214 22662 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:15.501238 22696 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:15.502251 22662 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:15.502368 22662 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/wal
uuid: "4e105824134f4416bac864e56405136e"
format_stamp: "Formatted at 2026-08-12 06:37:13 on dist-test-slave-8n49"
I20260812 06:37:15.502735 22662 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:15.513411 22662 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:15.514037 22662 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:15.514238 22662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:15.522451 22760 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.253:44703 every 8 connection(s)
I20260812 06:37:15.522456 22662 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.253:44703
I20260812 06:37:15.523564 22662 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-1/data/info.pb
I20260812 06:37:15.527379 22761 sys_catalog.cc:263] Verifying existing consensus state
I20260812 06:37:15.528990 22761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e: Bootstrap starting.
I20260812 06:37:15.531001 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 22662
I20260812 06:37:15.531427 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.252:41225
--webserver_interface=127.19.104.252
--webserver_port=45105
--builtin_ntp_servers=127.19.104.212:33233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.104.254:36255,127.19.104.253:44703,127.19.104.252:41225
--leader_failure_max_missed_heartbeat_periods=1.0 with env {}
I20260812 06:37:15.546530 22761 log.cc:826] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:15.554095 22761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:37:15.554535 22761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e: Bootstrap complete.
I20260812 06:37:15.559168 22761 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } }
I20260812 06:37:15.559579 22761 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e105824134f4416bac864e56405136e, State: Initialized, Role: FOLLOWER
I20260812 06:37:15.560130 22761 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } }
I20260812 06:37:15.560631 22766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } } }
I20260812 06:37:15.560762 22766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:15.561344 22761 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:37:15.566390 22779 catalog_manager.cc:1295] Loaded cluster ID: 6c8876ec68914c3c8d78cbedf403c311
I20260812 06:37:15.566454 22779 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e: loading cluster ID for follower catalog manager: success
I20260812 06:37:15.568789 22779 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e: acquiring CA information for follower catalog manager: success
I20260812 06:37:15.569965 22779 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:37:15.586380 22662 master_runner.cc:187] Error getting master registration for 127.19.104.252:41225: Network error: Client connection negotiation failed: client connection to 127.19.104.252:41225: connect: Connection refused (error 111), 
I20260812 06:37:15.588410 22662 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260812 06:37:15.650543 22763 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:15.650810 22763 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:15.650882 22763 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:15.654248 22763 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:37:15.654336 22763 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:15.654376 22763 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:37:15.654435 22763 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:37:15.658308 22763 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:33233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.104.254:36255,127.19.104.253:44703,127.19.104.252:41225
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.252:41225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.104.252
--webserver_port=45105
--never_fsync=true
--heap_profile_path=/tmp/kudu.22763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:15.659401 22763 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:15.660528 22763 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:15.666003 22788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:15.666050 22787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:15.666046 22791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:15.666851 22763 server_base.cc:1061] running on GCE node
I20260812 06:37:15.667367 22763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:15.668390 22763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:15.669584 22763 hybrid_clock.cc:648] HybridClock initialized: now 1786516635669564 us; error 30 us; skew 500 ppm
I20260812 06:37:15.671586 22763 webserver.cc:533] Webserver started at http://127.19.104.252:45105/ using document root <none> and password file <none>
I20260812 06:37:15.672228 22763 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:15.672312 22763 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:15.676446 22763 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 06:37:15.678714 22797 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:15.679831 22763 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:15.679942 22763 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/wal
uuid: "c4d868d998ab471eb47febaf08b59b67"
format_stamp: "Formatted at 2026-08-12 06:37:14 on dist-test-slave-8n49"
I20260812 06:37:15.680393 22763 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:15.708958 22763 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:15.709662 22763 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:15.709860 22763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:15.718191 22763 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.252:41225
I20260812 06:37:15.718191 22856 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.252:41225 every 8 connection(s)
I20260812 06:37:15.719390 22763 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/master-2/data/info.pb
I20260812 06:37:15.723340 22857 sys_catalog.cc:263] Verifying existing consensus state
I20260812 06:37:15.725013 22857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67: Bootstrap starting.
I20260812 06:37:15.729130 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 22763
I20260812 06:37:15.729548 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.104.193:34847
--local_ip_for_outbound_sockets=127.19.104.193
--tserver_master_addrs=127.19.104.254:36255,127.19.104.253:44703,127.19.104.252:41225
--webserver_port=40249
--webserver_interface=127.19.104.193
--builtin_ntp_servers=127.19.104.212:33233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260812 06:37:15.738963 22857 log.cc:826] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:15.744989 22857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:37:15.745365 22857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67: Bootstrap complete.
I20260812 06:37:15.748808 22857 raft_consensus.cc:359] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } }
I20260812 06:37:15.749174 22857 raft_consensus.cc:740] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d868d998ab471eb47febaf08b59b67, State: Initialized, Role: FOLLOWER
I20260812 06:37:15.749876 22857 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } }
I20260812 06:37:15.750329 22861 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } } }
I20260812 06:37:15.750489 22861 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:15.750981 22857 sys_catalog.cc:565] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:37:15.756090 22872 catalog_manager.cc:1295] Loaded cluster ID: 6c8876ec68914c3c8d78cbedf403c311
I20260812 06:37:15.756208 22872 catalog_manager.cc:1588] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67: loading cluster ID for follower catalog manager: success
I20260812 06:37:15.758605 22872 catalog_manager.cc:1610] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67: acquiring CA information for follower catalog manager: success
I20260812 06:37:15.759845 22872 catalog_manager.cc:1638] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:37:15.780478 22763 master_runner.cc:264] No leader master found from master c4d868d998ab471eb47febaf08b59b67
I20260812 06:37:15.780599 22763 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260812 06:37:15.812940 22664 raft_consensus.cc:493] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:37:15.813185 22664 raft_consensus.cc:515] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } }
I20260812 06:37:15.814407 22664 leader_election.cc:290] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e105824134f4416bac864e56405136e (127.19.104.253:44703), c4d868d998ab471eb47febaf08b59b67 (127.19.104.252:41225)
I20260812 06:37:15.819306 22728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "022dbc1ff8184ac89cee789ace581616" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e105824134f4416bac864e56405136e" is_pre_election: true
I20260812 06:37:15.819708 22728 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 022dbc1ff8184ac89cee789ace581616 in term 1.
I20260812 06:37:15.819983 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "022dbc1ff8184ac89cee789ace581616" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c4d868d998ab471eb47febaf08b59b67" is_pre_election: true
I20260812 06:37:15.820221 22601 leader_election.cc:304] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 022dbc1ff8184ac89cee789ace581616, 4e105824134f4416bac864e56405136e; no voters: 
I20260812 06:37:15.820282 22827 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 022dbc1ff8184ac89cee789ace581616 in term 1.
I20260812 06:37:15.820489 22664 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 06:37:15.820595 22664 raft_consensus.cc:493] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:37:15.820655 22664 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:37:15.822425 22664 raft_consensus.cc:515] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } }
I20260812 06:37:15.822870 22664 leader_election.cc:290] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [CANDIDATE]: Term 2 election: Requested vote from peers 4e105824134f4416bac864e56405136e (127.19.104.253:44703), c4d868d998ab471eb47febaf08b59b67 (127.19.104.252:41225)
I20260812 06:37:15.823215 22728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "022dbc1ff8184ac89cee789ace581616" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e105824134f4416bac864e56405136e"
I20260812 06:37:15.823294 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "022dbc1ff8184ac89cee789ace581616" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c4d868d998ab471eb47febaf08b59b67"
I20260812 06:37:15.823343 22728 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:37:15.823398 22827 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:37:15.825316 22728 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 022dbc1ff8184ac89cee789ace581616 in term 2.
I20260812 06:37:15.825316 22827 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 022dbc1ff8184ac89cee789ace581616 in term 2.
I20260812 06:37:15.825663 22601 leader_election.cc:304] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 022dbc1ff8184ac89cee789ace581616, 4e105824134f4416bac864e56405136e; no voters: 
I20260812 06:37:15.825875 22664 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 06:37:15.826113 22664 raft_consensus.cc:697] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [term 2 LEADER]: Becoming Leader. State: Replica: 022dbc1ff8184ac89cee789ace581616, State: Running, Role: LEADER
I20260812 06:37:15.826392 22664 consensus_queue.cc:237] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } }
I20260812 06:37:15.828068 22881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 022dbc1ff8184ac89cee789ace581616. Latest consensus state: current_term: 2 leader_uuid: "022dbc1ff8184ac89cee789ace581616" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } } }
I20260812 06:37:15.828184 22881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:15.828514 22883 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:37:15.829514 22883 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:37:15.829906 22883 catalog_manager.cc:1295] Loaded cluster ID: 6c8876ec68914c3c8d78cbedf403c311
I20260812 06:37:15.829988 22883 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:37:15.830502 22883 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:37:15.830890 22883 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616: Loaded TSK: 0
I20260812 06:37:15.831406 22883 catalog_manager.cc:1550] Initializing in-progress tserver states...
W20260812 06:37:15.862208 22859 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:15.862483 22859 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:15.862573 22859 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:15.867213 22859 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:15.867368 22859 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.104.193
I20260812 06:37:15.872690 22859 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:33233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.104.193:34847
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.104.193
--webserver_port=40249
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.19.104.254:36255,127.19.104.253:44703,127.19.104.252:41225
--never_fsync=true
--heap_profile_path=/tmp/kudu.22859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.104.193

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:15.874045 22859 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:15.875409 22859 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:15.883049 22888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:15.883057 22889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:15.883047 22891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:15.883620 22859 server_base.cc:1061] running on GCE node
I20260812 06:37:15.884006 22859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:15.884644 22859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:15.885828 22859 hybrid_clock.cc:648] HybridClock initialized: now 1786516635885810 us; error 37 us; skew 500 ppm
I20260812 06:37:15.887899 22859 webserver.cc:533] Webserver started at http://127.19.104.193:40249/ using document root <none> and password file <none>
I20260812 06:37:15.888600 22859 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:15.888691 22859 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:15.892872 22859 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 06:37:15.894964 22898 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:15.896183 22859 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:15.896288 22859 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/wal
uuid: "0d0bcbf7fecf4f0c889c06f8b8960c16"
format_stamp: "Formatted at 2026-08-12 06:37:14 on dist-test-slave-8n49"
I20260812 06:37:15.896776 22859 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:15.911391 22859 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:15.912153 22859 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:15.912348 22859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:15.913112 22859 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:15.914368 22859 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:15.914439 22859 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:15.914494 22859 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:15.914528 22859 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:15.926045 22859 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.193:34847
I20260812 06:37:15.926119 23036 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.193:34847 every 8 connection(s)
I20260812 06:37:15.927285 22859 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-0/data/info.pb
I20260812 06:37:15.927757 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 22859
I20260812 06:37:15.928215 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.104.194:38507
--local_ip_for_outbound_sockets=127.19.104.194
--tserver_master_addrs=127.19.104.254:36255,127.19.104.253:44703,127.19.104.252:41225
--webserver_port=44831
--webserver_interface=127.19.104.194
--builtin_ntp_servers=127.19.104.212:33233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260812 06:37:15.941941 23038 heartbeater.cc:344] Connected to a master server at 127.19.104.252:41225
I20260812 06:37:15.941982 23039 heartbeater.cc:344] Connected to a master server at 127.19.104.253:44703
I20260812 06:37:15.941965 23037 heartbeater.cc:344] Connected to a master server at 127.19.104.254:36255
I20260812 06:37:15.942235 23038 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:15.942358 23037 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:15.942237 23039 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:15.942953 23037 heartbeater.cc:507] Master 127.19.104.254:36255 requested a full tablet report, sending...
I20260812 06:37:15.943132 23039 heartbeater.cc:507] Master 127.19.104.253:44703 requested a full tablet report, sending...
I20260812 06:37:15.943543 23038 heartbeater.cc:507] Master 127.19.104.252:41225 requested a full tablet report, sending...
I20260812 06:37:15.944870 22715 ts_manager.cc:194] Registered new tserver with Master: 0d0bcbf7fecf4f0c889c06f8b8960c16 (127.19.104.193:34847)
I20260812 06:37:15.944952 22619 ts_manager.cc:194] Registered new tserver with Master: 0d0bcbf7fecf4f0c889c06f8b8960c16 (127.19.104.193:34847)
I20260812 06:37:15.946630 22619 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.104.193:47679
I20260812 06:37:15.948345 22816 ts_manager.cc:194] Registered new tserver with Master: 0d0bcbf7fecf4f0c889c06f8b8960c16 (127.19.104.193:34847)
W20260812 06:37:16.040455 23041 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:16.040726 23041 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:16.040815 23041 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:16.044087 23041 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:16.044203 23041 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.104.194
I20260812 06:37:16.048120 23041 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:33233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.104.194:38507
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.104.194
--webserver_port=44831
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.19.104.254:36255,127.19.104.253:44703,127.19.104.252:41225
--never_fsync=true
--heap_profile_path=/tmp/kudu.23041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.104.194

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:16.049250 23041 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:16.050444 23041 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:16.057188 23054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:16.057210 23051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:16.057210 23050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:16.057368 23041 server_base.cc:1061] running on GCE node
I20260812 06:37:16.057766 23041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:16.058419 23041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:16.059587 23041 hybrid_clock.cc:648] HybridClock initialized: now 1786516636059572 us; error 38 us; skew 500 ppm
I20260812 06:37:16.061358 23041 webserver.cc:533] Webserver started at http://127.19.104.194:44831/ using document root <none> and password file <none>
I20260812 06:37:16.061929 23041 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:16.062033 23041 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:16.065477 23041 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.001s	sys 0.003s
I20260812 06:37:16.067205 23064 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:16.068172 23041 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:16.068305 23041 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/wal
uuid: "6b28823152394f21ac8a9ad43f660111"
format_stamp: "Formatted at 2026-08-12 06:37:14 on dist-test-slave-8n49"
I20260812 06:37:16.068759 23041 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:16.079666 23041 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:16.080273 23041 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:16.080468 23041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:16.081069 23041 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:16.082062 23041 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:16.082135 23041 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:16.082204 23041 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:16.082257 23041 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:16.091180 23041 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.194:38507
I20260812 06:37:16.091209 23189 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.194:38507 every 8 connection(s)
I20260812 06:37:16.092149 23041 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-1/data/info.pb
I20260812 06:37:16.095932 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 23041
I20260812 06:37:16.096421 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.104.195:46883
--local_ip_for_outbound_sockets=127.19.104.195
--tserver_master_addrs=127.19.104.254:36255,127.19.104.253:44703,127.19.104.252:41225
--webserver_port=34857
--webserver_interface=127.19.104.195
--builtin_ntp_servers=127.19.104.212:33233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260812 06:37:16.102008 23190 heartbeater.cc:344] Connected to a master server at 127.19.104.254:36255
I20260812 06:37:16.102095 23192 heartbeater.cc:344] Connected to a master server at 127.19.104.252:41225
I20260812 06:37:16.102252 23192 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:16.102928 23192 heartbeater.cc:507] Master 127.19.104.252:41225 requested a full tablet report, sending...
I20260812 06:37:16.103446 23193 heartbeater.cc:344] Connected to a master server at 127.19.104.253:44703
I20260812 06:37:16.102253 23190 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:16.103538 23193 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:16.103721 23190 heartbeater.cc:507] Master 127.19.104.254:36255 requested a full tablet report, sending...
I20260812 06:37:16.103744 23193 heartbeater.cc:507] Master 127.19.104.253:44703 requested a full tablet report, sending...
I20260812 06:37:16.104068 22816 ts_manager.cc:194] Registered new tserver with Master: 6b28823152394f21ac8a9ad43f660111 (127.19.104.194:38507)
I20260812 06:37:16.104545 22715 ts_manager.cc:194] Registered new tserver with Master: 6b28823152394f21ac8a9ad43f660111 (127.19.104.194:38507)
I20260812 06:37:16.104607 22619 ts_manager.cc:194] Registered new tserver with Master: 6b28823152394f21ac8a9ad43f660111 (127.19.104.194:38507)
I20260812 06:37:16.105227 22619 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.104.194:41905
W20260812 06:37:16.204541 23198 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:16.204777 23198 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:16.204829 23198 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:16.208096 23198 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:16.208210 23198 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.104.195
I20260812 06:37:16.208933 22827 raft_consensus.cc:1275] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [term 2 FOLLOWER]: Refusing update from remote peer 022dbc1ff8184ac89cee789ace581616: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 06:37:16.209712 22881 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 06:37:16.211835 22861 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 022dbc1ff8184ac89cee789ace581616. Latest consensus state: current_term: 2 leader_uuid: "022dbc1ff8184ac89cee789ace581616" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } } }
I20260812 06:37:16.211962 22861 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:16.212282 23198 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:33233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.104.195:46883
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.104.195
--webserver_port=34857
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.19.104.254:36255,127.19.104.253:44703,127.19.104.252:41225
--never_fsync=true
--heap_profile_path=/tmp/kudu.23198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.104.195

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:16.213536 23198 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:16.213591 22881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "022dbc1ff8184ac89cee789ace581616" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } } }
I20260812 06:37:16.213722 22881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:16.214236 22861 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "022dbc1ff8184ac89cee789ace581616" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } } }
I20260812 06:37:16.214334 22861 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4d868d998ab471eb47febaf08b59b67 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:16.214958 23198 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 06:37:16.215385 22728 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [term 2 FOLLOWER]: Refusing update from remote peer 022dbc1ff8184ac89cee789ace581616: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 06:37:16.216002 22664 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 022dbc1ff8184ac89cee789ace581616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 06:37:16.218321 23213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "022dbc1ff8184ac89cee789ace581616" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } } }
I20260812 06:37:16.218461 23213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:16.218914 23208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 022dbc1ff8184ac89cee789ace581616. Latest consensus state: current_term: 2 leader_uuid: "022dbc1ff8184ac89cee789ace581616" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022dbc1ff8184ac89cee789ace581616" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 36255 } } peers { permanent_uuid: "4e105824134f4416bac864e56405136e" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 44703 } } peers { permanent_uuid: "c4d868d998ab471eb47febaf08b59b67" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 41225 } } }
I20260812 06:37:16.219025 23208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e105824134f4416bac864e56405136e [sys.catalog]: This master's current role is: FOLLOWER
W20260812 06:37:16.222538 23217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:16.222465 23216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:16.222668 23198 server_base.cc:1061] running on GCE node
W20260812 06:37:16.222466 23220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:16.223227 23198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:16.223755 23198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:16.224921 23198 hybrid_clock.cc:648] HybridClock initialized: now 1786516636224906 us; error 32 us; skew 500 ppm
I20260812 06:37:16.226527 23198 webserver.cc:533] Webserver started at http://127.19.104.195:34857/ using document root <none> and password file <none>
I20260812 06:37:16.227093 23198 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:16.227190 23198 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:16.230345 23198 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.003s	sys 0.001s
I20260812 06:37:16.232043 23227 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:16.233094 23198 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:16.233229 23198 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/wal
uuid: "f5543c77410446cea27a857293f6127e"
format_stamp: "Formatted at 2026-08-12 06:37:15 on dist-test-slave-8n49"
I20260812 06:37:16.233642 23198 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:16.247627 23198 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:16.248353 23198 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:16.248615 23198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:16.249225 23198 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:16.250283 23198 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:16.250356 23198 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:16.250437 23198 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:16.250491 23198 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:16.260198 23198 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.195:46883
I20260812 06:37:16.260221 23363 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.195:46883 every 8 connection(s)
I20260812 06:37:16.261298 23198 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestMasterUUIDResolution_0.1786516586160649-19875-0/minicluster-data/ts-2/data/info.pb
I20260812 06:37:16.263796 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 23198
I20260812 06:37:16.272300 23366 heartbeater.cc:344] Connected to a master server at 127.19.104.253:44703
I20260812 06:37:16.272459 23364 heartbeater.cc:344] Connected to a master server at 127.19.104.254:36255
I20260812 06:37:16.272475 23365 heartbeater.cc:344] Connected to a master server at 127.19.104.252:41225
I20260812 06:37:16.272636 23366 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:16.272749 23365 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:16.272635 23364 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:16.273337 23366 heartbeater.cc:507] Master 127.19.104.253:44703 requested a full tablet report, sending...
I20260812 06:37:16.273337 23365 heartbeater.cc:507] Master 127.19.104.252:41225 requested a full tablet report, sending...
I20260812 06:37:16.273512 23364 heartbeater.cc:507] Master 127.19.104.254:36255 requested a full tablet report, sending...
I20260812 06:37:16.274551 22619 ts_manager.cc:194] Registered new tserver with Master: f5543c77410446cea27a857293f6127e (127.19.104.195:46883)
I20260812 06:37:16.274647 22715 ts_manager.cc:194] Registered new tserver with Master: f5543c77410446cea27a857293f6127e (127.19.104.195:46883)
I20260812 06:37:16.274896 22816 ts_manager.cc:194] Registered new tserver with Master: f5543c77410446cea27a857293f6127e (127.19.104.195:46883)
I20260812 06:37:16.275247 22619 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.104.195:53229
I20260812 06:37:16.283486 19875 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 06:37:16.295429 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 22859
I20260812 06:37:16.304701 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 23041
I20260812 06:37:16.316812 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 23198
I20260812 06:37:16.325279 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 22582
I20260812 06:37:16.335009 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 22662
I20260812 06:37:16.343919 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 22763
2026-08-12T06:37:16Z chronyd exiting
[       OK ] HmsConfigurations/MasterFailoverTest.TestMasterUUIDResolution/0 (2751 ms)
[ RUN      ] HmsConfigurations/MasterFailoverTest.TestClusterIdOnFailover/0
2026-08-12T06:37:16Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:37:16Z Disabled control of system clock
I20260812 06:37:16.379379 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.254:40901
--webserver_interface=127.19.104.254
--webserver_port=0
--builtin_ntp_servers=127.19.104.212:39791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.104.254:40901,127.19.104.253:36501,127.19.104.252:40219
--leader_failure_max_missed_heartbeat_periods=1.0 with env {}
W20260812 06:37:16.487534 23381 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:16.487777 23381 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:16.487818 23381 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:16.491120 23381 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:37:16.491184 23381 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:16.491204 23381 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:37:16.491222 23381 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:37:16.495167 23381 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:39791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.104.254:40901,127.19.104.253:36501,127.19.104.252:40219
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.254:40901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.104.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.23381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:16.496227 23381 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:16.497350 23381 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:16.502950 23387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:16.502947 23392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:16.502945 23390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:16.503142 23381 server_base.cc:1061] running on GCE node
I20260812 06:37:16.503672 23381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:16.504650 23381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:16.505817 23381 hybrid_clock.cc:648] HybridClock initialized: now 1786516636505803 us; error 35 us; skew 500 ppm
I20260812 06:37:16.507512 23381 webserver.cc:533] Webserver started at http://127.19.104.254:45639/ using document root <none> and password file <none>
I20260812 06:37:16.508087 23381 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:16.508148 23381 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:16.508355 23381 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:16.509953 23381 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-0/data/instance:
uuid: "884e3c5e5de4458495dd8f10137050e3"
format_stamp: "Formatted at 2026-08-12 06:37:16 on dist-test-slave-8n49"
I20260812 06:37:16.510432 23381 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-0/wal/instance:
uuid: "884e3c5e5de4458495dd8f10137050e3"
format_stamp: "Formatted at 2026-08-12 06:37:16 on dist-test-slave-8n49"
I20260812 06:37:16.513967 23381 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 06:37:16.516111 23398 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:16.517159 23381 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 06:37:16.517284 23381 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-0/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-0/wal
uuid: "884e3c5e5de4458495dd8f10137050e3"
format_stamp: "Formatted at 2026-08-12 06:37:16 on dist-test-slave-8n49"
I20260812 06:37:16.517385 23381 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:16.530146 23381 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:16.530812 23381 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:16.531013 23381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:16.539063 23381 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.254:40901
I20260812 06:37:16.539088 23461 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.254:40901 every 8 connection(s)
I20260812 06:37:16.540131 23381 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-0/data/info.pb
I20260812 06:37:16.543078 23462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:16.544828 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 23381
I20260812 06:37:16.544938 19875 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-0/wal/instance
I20260812 06:37:16.545621 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.253:36501
--webserver_interface=127.19.104.253
--webserver_port=0
--builtin_ntp_servers=127.19.104.212:39791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.104.254:40901,127.19.104.253:36501,127.19.104.252:40219
--leader_failure_max_missed_heartbeat_periods=1.0 with env {}
I20260812 06:37:16.546916 23462 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:16.557686 23462 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } has no permanent_uuid. Determining permanent_uuid...
W20260812 06:37:16.558804 23401 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.104.253:36501: connect: Connection refused (error 111)
W20260812 06:37:16.559815 23462 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.253:36501: Network error: Client connection negotiation failed: client connection to 127.19.104.253:36501: connect: Connection refused (error 111)
I20260812 06:37:16.609102 23462 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } attempt: 1
W20260812 06:37:16.610777 23462 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.253:36501: Network error: Client connection negotiation failed: client connection to 127.19.104.253:36501: connect: Connection refused (error 111)
W20260812 06:37:16.650574 23464 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:16.650789 23464 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:16.650828 23464 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:16.654209 23464 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:37:16.654273 23464 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:16.654294 23464 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:37:16.654310 23464 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:37:16.659427 23464 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:39791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.104.254:40901,127.19.104.253:36501,127.19.104.252:40219
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.253:36501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.104.253
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.23464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:16.660943 23464 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:16.662540 23464 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:16.669350 23477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:16.669348 23476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:16.669417 23479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:16.670405 23464 server_base.cc:1061] running on GCE node
I20260812 06:37:16.670861 23464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:16.671772 23464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:16.672950 23464 hybrid_clock.cc:648] HybridClock initialized: now 1786516636672925 us; error 44 us; skew 500 ppm
I20260812 06:37:16.674949 23464 webserver.cc:533] Webserver started at http://127.19.104.253:42233/ using document root <none> and password file <none>
I20260812 06:37:16.675527 23464 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:16.675613 23464 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:16.675841 23464 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:16.677472 23464 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-1/data/instance:
uuid: "d5f910a7236c45f3af3ce03936252ee8"
format_stamp: "Formatted at 2026-08-12 06:37:16 on dist-test-slave-8n49"
I20260812 06:37:16.677984 23464 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-1/wal/instance:
uuid: "d5f910a7236c45f3af3ce03936252ee8"
format_stamp: "Formatted at 2026-08-12 06:37:16 on dist-test-slave-8n49"
I20260812 06:37:16.678968 23462 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } attempt: 2
W20260812 06:37:16.680433 23462 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.253:36501: Network error: Client connection negotiation failed: client connection to 127.19.104.253:36501: connect: Connection refused (error 111)
I20260812 06:37:16.681439 23464 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 06:37:16.683736 23486 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:16.684878 23464 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:16.684983 23464 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-1/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-1/wal
uuid: "d5f910a7236c45f3af3ce03936252ee8"
format_stamp: "Formatted at 2026-08-12 06:37:16 on dist-test-slave-8n49"
I20260812 06:37:16.685063 23464 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:16.714766 23464 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:16.715418 23464 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:16.715580 23464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:16.723170 23546 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.253:36501 every 8 connection(s)
I20260812 06:37:16.723179 23464 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.253:36501
I20260812 06:37:16.724243 23464 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-1/data/info.pb
I20260812 06:37:16.727174 23547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:16.730476 23547 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:16.733012 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 23464
I20260812 06:37:16.733130 19875 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-1/wal/instance
I20260812 06:37:16.733934 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.252:40219
--webserver_interface=127.19.104.252
--webserver_port=0
--builtin_ntp_servers=127.19.104.212:39791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.104.254:40901,127.19.104.253:36501,127.19.104.252:40219
--leader_failure_max_missed_heartbeat_periods=1.0 with env {}
I20260812 06:37:16.739821 23547 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:16.746582 23547 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } has no permanent_uuid. Determining permanent_uuid...
W20260812 06:37:16.747385 23487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.104.252:40219: connect: Connection refused (error 111)
W20260812 06:37:16.748309 23547 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.252:40219: Network error: Client connection negotiation failed: client connection to 127.19.104.252:40219: connect: Connection refused (error 111)
I20260812 06:37:16.771636 23462 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } attempt: 3
I20260812 06:37:16.776582 23462 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } has no permanent_uuid. Determining permanent_uuid...
W20260812 06:37:16.777980 23462 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.252:40219: Network error: Client connection negotiation failed: client connection to 127.19.104.252:40219: connect: Connection refused (error 111)
I20260812 06:37:16.797559 23547 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } attempt: 1
W20260812 06:37:16.799132 23547 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.252:40219: Network error: Client connection negotiation failed: client connection to 127.19.104.252:40219: connect: Connection refused (error 111)
I20260812 06:37:16.809177 23462 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } attempt: 1
W20260812 06:37:16.810756 23462 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.252:40219: Network error: Client connection negotiation failed: client connection to 127.19.104.252:40219: connect: Connection refused (error 111)
W20260812 06:37:16.843223 23550 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:16.843452 23550 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:16.843495 23550 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:16.846819 23550 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:37:16.846881 23550 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:16.846902 23550 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:37:16.846920 23550 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:37:16.850903 23550 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:39791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.104.254:40901,127.19.104.253:36501,127.19.104.252:40219
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.104.252:40219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.104.252
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.23550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:16.852001 23550 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:16.853127 23550 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:16.858595 23558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:16.858608 23557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:16.858616 23561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:16.858954 23550 server_base.cc:1061] running on GCE node
I20260812 06:37:16.859460 23550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:16.860363 23550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:16.861562 23550 hybrid_clock.cc:648] HybridClock initialized: now 1786516636861546 us; error 36 us; skew 500 ppm
I20260812 06:37:16.863317 23550 webserver.cc:533] Webserver started at http://127.19.104.252:32839/ using document root <none> and password file <none>
I20260812 06:37:16.863878 23550 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:16.863972 23550 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:16.864192 23550 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:16.865780 23550 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-2/data/instance:
uuid: "75c24f176e7f4b22a21593548438edc7"
format_stamp: "Formatted at 2026-08-12 06:37:16 on dist-test-slave-8n49"
I20260812 06:37:16.866299 23550 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-2/wal/instance:
uuid: "75c24f176e7f4b22a21593548438edc7"
format_stamp: "Formatted at 2026-08-12 06:37:16 on dist-test-slave-8n49"
I20260812 06:37:16.867324 23547 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } attempt: 2
W20260812 06:37:16.868970 23547 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.252:40219: Network error: Client connection negotiation failed: client connection to 127.19.104.252:40219: connect: Connection refused (error 111)
I20260812 06:37:16.870213 23550 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.002s
I20260812 06:37:16.872360 23568 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:16.873485 23550 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:16.873646 23550 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-2/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-2/wal
uuid: "75c24f176e7f4b22a21593548438edc7"
format_stamp: "Formatted at 2026-08-12 06:37:16 on dist-test-slave-8n49"
I20260812 06:37:16.873759 23550 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:16.885247 23550 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:16.885962 23550 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:16.885959 23462 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } attempt: 2
I20260812 06:37:16.886168 23550 kserver.cc:163] Server-wide thread pool size limit: 3276
W20260812 06:37:16.887604 23462 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.104.252:40219: Network error: Client connection negotiation failed: client connection to 127.19.104.252:40219: connect: Connection refused (error 111)
I20260812 06:37:16.893740 23627 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.252:40219 every 8 connection(s)
I20260812 06:37:16.893748 23550 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.252:40219
I20260812 06:37:16.894765 23550 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-2/data/info.pb
I20260812 06:37:16.897555 23628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:16.900803 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 23550
I20260812 06:37:16.900907 19875 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/master-2/wal/instance
I20260812 06:37:16.900906 23628 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:16.909952 23628 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:16.914475 23628 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:16.920266 23628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7: Bootstrap starting.
I20260812 06:37:16.923246 23628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:16.923959 23628 log.cc:826] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:16.925560 23628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7: No bootstrap required, opened a new log
I20260812 06:37:16.928399 23628 raft_consensus.cc:359] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } }
I20260812 06:37:16.928634 23628 raft_consensus.cc:385] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:16.928687 23628 raft_consensus.cc:740] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75c24f176e7f4b22a21593548438edc7, State: Initialized, Role: FOLLOWER
I20260812 06:37:16.929107 23628 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } }
I20260812 06:37:16.929576 23636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } } }
I20260812 06:37:16.929834 23636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:16.930204 23628 sys_catalog.cc:565] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 06:37:16.935557 23649 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:37:16.935639 23649 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:37:16.934263 23550 master_runner.cc:187] Error getting master registration for 127.19.104.254:40901: OK, instance_id { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" instance_seqno: 1786516636536815 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 06:37:16.936147 23550 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260812 06:37:16.960258 23547 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } attempt: 3
I20260812 06:37:16.966951 23547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8: Bootstrap starting.
I20260812 06:37:16.969681 23547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:16.971168 23547 log.cc:826] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:16.973203 23547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8: No bootstrap required, opened a new log
I20260812 06:37:16.976972 23547 raft_consensus.cc:359] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } }
I20260812 06:37:16.977166 23547 raft_consensus.cc:385] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:16.977233 23547 raft_consensus.cc:740] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5f910a7236c45f3af3ce03936252ee8, State: Initialized, Role: FOLLOWER
I20260812 06:37:16.977716 23547 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } }
I20260812 06:37:16.978253 23652 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } } }
I20260812 06:37:16.978436 23652 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:16.979043 23547 sys_catalog.cc:565] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 06:37:16.984650 23667 catalog_manager.cc:1594] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:37:16.984731 23667 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:37:16.983165 23464 master_runner.cc:187] Error getting master registration for 127.19.104.254:40901: OK, instance_id { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" instance_seqno: 1786516636536815 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 06:37:16.985428 23464 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260812 06:37:16.986795 23462 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } attempt: 3
I20260812 06:37:16.994114 23462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3: Bootstrap starting.
I20260812 06:37:16.996891 23462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:16.998047 23462 log.cc:826] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:17.000048 23462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3: No bootstrap required, opened a new log
I20260812 06:37:17.003396 23462 raft_consensus.cc:359] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } }
I20260812 06:37:17.003592 23462 raft_consensus.cc:385] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:17.003652 23462 raft_consensus.cc:740] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884e3c5e5de4458495dd8f10137050e3, State: Initialized, Role: FOLLOWER
I20260812 06:37:17.004112 23462 consensus_queue.cc:260] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } }
I20260812 06:37:17.004654 23670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } } }
I20260812 06:37:17.004834 23670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:17.005405 23462 sys_catalog.cc:565] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 06:37:17.013146 23681 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:37:17.013249 23681 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:37:17.015511 23381 master_runner.cc:264] No leader master found from master 884e3c5e5de4458495dd8f10137050e3
I20260812 06:37:17.015590 23381 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260812 06:37:17.032413 23405 tablet.cc:2406] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:37:17.076503 23636 raft_consensus.cc:493] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:37:17.076768 23636 raft_consensus.cc:515] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } }
I20260812 06:37:17.078018 23636 leader_election.cc:290] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884e3c5e5de4458495dd8f10137050e3 (127.19.104.254:40901), d5f910a7236c45f3af3ce03936252ee8 (127.19.104.253:36501)
I20260812 06:37:17.078936 23430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75c24f176e7f4b22a21593548438edc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884e3c5e5de4458495dd8f10137050e3" is_pre_election: true
I20260812 06:37:17.079314 23430 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75c24f176e7f4b22a21593548438edc7 in term 0.
I20260812 06:37:17.079878 23571 leader_election.cc:304] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75c24f176e7f4b22a21593548438edc7, 884e3c5e5de4458495dd8f10137050e3; no voters: 
I20260812 06:37:17.081023 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75c24f176e7f4b22a21593548438edc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5f910a7236c45f3af3ce03936252ee8" is_pre_election: true
I20260812 06:37:17.081334 23518 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75c24f176e7f4b22a21593548438edc7 in term 0.
I20260812 06:37:17.081995 23636 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:37:17.082089 23636 raft_consensus.cc:493] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:37:17.082144 23636 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:17.083424 23636 raft_consensus.cc:515] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } }
I20260812 06:37:17.083848 23636 leader_election.cc:290] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [CANDIDATE]: Term 1 election: Requested vote from peers 884e3c5e5de4458495dd8f10137050e3 (127.19.104.254:40901), d5f910a7236c45f3af3ce03936252ee8 (127.19.104.253:36501)
I20260812 06:37:17.084357 23430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75c24f176e7f4b22a21593548438edc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884e3c5e5de4458495dd8f10137050e3"
I20260812 06:37:17.084957 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75c24f176e7f4b22a21593548438edc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5f910a7236c45f3af3ce03936252ee8"
I20260812 06:37:17.085107 23518 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:17.086406 23518 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75c24f176e7f4b22a21593548438edc7 in term 1.
I20260812 06:37:17.086838 23570 leader_election.cc:304] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75c24f176e7f4b22a21593548438edc7, d5f910a7236c45f3af3ce03936252ee8; no voters: 
I20260812 06:37:17.084506 23430 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:17.088285 23430 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75c24f176e7f4b22a21593548438edc7 in term 1.
I20260812 06:37:17.090127 23636 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:37:17.090410 23636 raft_consensus.cc:697] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [term 1 LEADER]: Becoming Leader. State: Replica: 75c24f176e7f4b22a21593548438edc7, State: Running, Role: LEADER
I20260812 06:37:17.090781 23636 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } }
I20260812 06:37:17.095829 23636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75c24f176e7f4b22a21593548438edc7. Latest consensus state: current_term: 1 leader_uuid: "75c24f176e7f4b22a21593548438edc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } } }
I20260812 06:37:17.095978 23636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:17.096589 23685 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:37:17.097760 23685 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:37:17.104871 23518 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [term 1 FOLLOWER]: Refusing update from remote peer 75c24f176e7f4b22a21593548438edc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:37:17.105602 23683 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:17.110059 23652 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75c24f176e7f4b22a21593548438edc7. Latest consensus state: current_term: 1 leader_uuid: "75c24f176e7f4b22a21593548438edc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } } }
I20260812 06:37:17.110191 23652 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:17.111163 23430 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [term 1 FOLLOWER]: Refusing update from remote peer 75c24f176e7f4b22a21593548438edc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:37:17.111721 23636 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:17.115142 23670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75c24f176e7f4b22a21593548438edc7. Latest consensus state: current_term: 1 leader_uuid: "75c24f176e7f4b22a21593548438edc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } } }
I20260812 06:37:17.115269 23670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:17.122066 23683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "75c24f176e7f4b22a21593548438edc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } } }
I20260812 06:37:17.122200 23683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:17.123386 23670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "75c24f176e7f4b22a21593548438edc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } } }
I20260812 06:37:17.123505 23670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:17.124289 23689 mvcc.cc:204] Tried to move back new op lower bound from 7317572145573130240 to 7317572145535197184. Current Snapshot: MvccSnapshot[applied={T|T < 7317572145573130240}]
I20260812 06:37:17.129161 23636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "75c24f176e7f4b22a21593548438edc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } } }
I20260812 06:37:17.129297 23636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:17.130426 23652 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "75c24f176e7f4b22a21593548438edc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } } }
I20260812 06:37:17.130558 23652 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:17.131533 23685 catalog_manager.cc:1383] Generated new cluster ID: 8834243fea2544cf8e7adad60e39b5bc
I20260812 06:37:17.131618 23685 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:37:17.149082 23687 mvcc.cc:204] Tried to move back new op lower bound from 7317572145573130240 to 7317572145535197184. Current Snapshot: MvccSnapshot[applied={T|T < 7317572145573130240}]
I20260812 06:37:17.168849 23685 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 06:37:17.170047 23685 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:37:17.182658 23685 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7: Generated new TSK 0
I20260812 06:37:17.183597 23685 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:37:17.189704 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.104.193:0
--local_ip_for_outbound_sockets=127.19.104.193
--webserver_interface=127.19.104.193
--webserver_port=0
--tserver_master_addrs=127.19.104.254:40901,127.19.104.253:36501,127.19.104.252:40219
--builtin_ntp_servers=127.19.104.212:39791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
W20260812 06:37:17.298763 23695 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:17.298991 23695 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:17.299043 23695 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:17.302386 23695 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:17.302579 23695 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.104.193
I20260812 06:37:17.306589 23695 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:39791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.104.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.104.193
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.19.104.254:40901,127.19.104.253:36501,127.19.104.252:40219
--never_fsync=true
--heap_profile_path=/tmp/kudu.23695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.104.193

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:17.307878 23695 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:17.309052 23695 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:17.315625 23708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:17.315625 23704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:17.315738 23695 server_base.cc:1061] running on GCE node
W20260812 06:37:17.315752 23702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:17.316149 23695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:17.316805 23695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:17.318003 23695 hybrid_clock.cc:648] HybridClock initialized: now 1786516637317985 us; error 35 us; skew 500 ppm
I20260812 06:37:17.319787 23695 webserver.cc:533] Webserver started at http://127.19.104.193:42793/ using document root <none> and password file <none>
I20260812 06:37:17.320384 23695 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:17.320474 23695 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:17.320735 23695 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:17.322284 23695 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-0/data/instance:
uuid: "e6ed6080e6b34ebf8feebe76f37409b8"
format_stamp: "Formatted at 2026-08-12 06:37:17 on dist-test-slave-8n49"
I20260812 06:37:17.322808 23695 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-0/wal/instance:
uuid: "e6ed6080e6b34ebf8feebe76f37409b8"
format_stamp: "Formatted at 2026-08-12 06:37:17 on dist-test-slave-8n49"
I20260812 06:37:17.326124 23695 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.001s	sys 0.004s
I20260812 06:37:17.328156 23714 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:17.329160 23695 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 06:37:17.329293 23695 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-0/wal
uuid: "e6ed6080e6b34ebf8feebe76f37409b8"
format_stamp: "Formatted at 2026-08-12 06:37:17 on dist-test-slave-8n49"
I20260812 06:37:17.329393 23695 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:17.343257 23695 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:17.343905 23695 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:17.344105 23695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:17.344707 23695 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:17.345719 23695 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:17.345791 23695 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:17.345865 23695 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:17.345918 23695 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:17.355014 23695 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.193:39173
I20260812 06:37:17.355044 23844 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.193:39173 every 8 connection(s)
I20260812 06:37:17.355957 23695 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-0/data/info.pb
I20260812 06:37:17.356276 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 23695
I20260812 06:37:17.356403 19875 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-0/wal/instance
I20260812 06:37:17.360404 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.104.194:0
--local_ip_for_outbound_sockets=127.19.104.194
--webserver_interface=127.19.104.194
--webserver_port=0
--tserver_master_addrs=127.19.104.254:40901,127.19.104.253:36501,127.19.104.252:40219
--builtin_ntp_servers=127.19.104.212:39791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260812 06:37:17.367565 23847 heartbeater.cc:344] Connected to a master server at 127.19.104.254:40901
I20260812 06:37:17.367551 23848 heartbeater.cc:344] Connected to a master server at 127.19.104.253:36501
I20260812 06:37:17.367527 23846 heartbeater.cc:344] Connected to a master server at 127.19.104.252:40219
I20260812 06:37:17.367826 23848 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:17.367823 23846 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:17.367825 23847 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:17.368454 23847 heartbeater.cc:507] Master 127.19.104.254:40901 requested a full tablet report, sending...
I20260812 06:37:17.368474 23848 heartbeater.cc:507] Master 127.19.104.253:36501 requested a full tablet report, sending...
I20260812 06:37:17.368454 23846 heartbeater.cc:507] Master 127.19.104.252:40219 requested a full tablet report, sending...
I20260812 06:37:17.370294 23505 ts_manager.cc:194] Registered new tserver with Master: e6ed6080e6b34ebf8feebe76f37409b8 (127.19.104.193:39173)
I20260812 06:37:17.370311 23419 ts_manager.cc:194] Registered new tserver with Master: e6ed6080e6b34ebf8feebe76f37409b8 (127.19.104.193:39173)
I20260812 06:37:17.374151 23585 ts_manager.cc:194] Registered new tserver with Master: e6ed6080e6b34ebf8feebe76f37409b8 (127.19.104.193:39173)
I20260812 06:37:17.375916 23585 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.104.193:34791
W20260812 06:37:17.476168 23855 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:17.476387 23855 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:17.476490 23855 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:17.480033 23855 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:17.480187 23855 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.104.194
I20260812 06:37:17.484129 23855 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:39791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.104.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.104.194
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.19.104.254:40901,127.19.104.253:36501,127.19.104.252:40219
--never_fsync=true
--heap_profile_path=/tmp/kudu.23855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.104.194

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:17.485291 23855 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:17.486443 23855 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:17.492810 23863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:17.492859 23861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:17.492859 23860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:17.493173 23855 server_base.cc:1061] running on GCE node
I20260812 06:37:17.493515 23855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:17.494102 23855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:17.495280 23855 hybrid_clock.cc:648] HybridClock initialized: now 1786516637495266 us; error 35 us; skew 500 ppm
I20260812 06:37:17.497159 23855 webserver.cc:533] Webserver started at http://127.19.104.194:43271/ using document root <none> and password file <none>
I20260812 06:37:17.497736 23855 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:17.497814 23855 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:17.498040 23855 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:17.499583 23855 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-1/data/instance:
uuid: "4c86452892b149c0b97f0d869965dcc5"
format_stamp: "Formatted at 2026-08-12 06:37:17 on dist-test-slave-8n49"
I20260812 06:37:17.500063 23855 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-1/wal/instance:
uuid: "4c86452892b149c0b97f0d869965dcc5"
format_stamp: "Formatted at 2026-08-12 06:37:17 on dist-test-slave-8n49"
I20260812 06:37:17.503321 23855 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 06:37:17.505416 23871 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:17.506474 23855 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:17.506583 23855 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-1/wal
uuid: "4c86452892b149c0b97f0d869965dcc5"
format_stamp: "Formatted at 2026-08-12 06:37:17 on dist-test-slave-8n49"
I20260812 06:37:17.506687 23855 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:17.521826 23855 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:17.522511 23855 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:17.522707 23855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:17.523324 23855 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:17.524339 23855 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:17.524389 23855 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:17.524425 23855 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:17.524482 23855 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:17.533550 23855 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.194:44771
I20260812 06:37:17.533592 24000 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.194:44771 every 8 connection(s)
I20260812 06:37:17.534590 23855 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-1/data/info.pb
I20260812 06:37:17.537822 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 23855
I20260812 06:37:17.537920 19875 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-1/wal/instance
I20260812 06:37:17.541447 19875 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
/tmp/dist-test-taskaCuUsW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.104.195:0
--local_ip_for_outbound_sockets=127.19.104.195
--webserver_interface=127.19.104.195
--webserver_port=0
--tserver_master_addrs=127.19.104.254:40901,127.19.104.253:36501,127.19.104.252:40219
--builtin_ntp_servers=127.19.104.212:39791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260812 06:37:17.545333 24003 heartbeater.cc:344] Connected to a master server at 127.19.104.253:36501
I20260812 06:37:17.545578 24003 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:17.546136 24003 heartbeater.cc:507] Master 127.19.104.253:36501 requested a full tablet report, sending...
I20260812 06:37:17.546656 24001 heartbeater.cc:344] Connected to a master server at 127.19.104.252:40219
I20260812 06:37:17.546730 24001 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:17.546885 24001 heartbeater.cc:507] Master 127.19.104.252:40219 requested a full tablet report, sending...
I20260812 06:37:17.547401 23505 ts_manager.cc:194] Registered new tserver with Master: 4c86452892b149c0b97f0d869965dcc5 (127.19.104.194:44771)
I20260812 06:37:17.547468 23585 ts_manager.cc:194] Registered new tserver with Master: 4c86452892b149c0b97f0d869965dcc5 (127.19.104.194:44771)
I20260812 06:37:17.548146 23585 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.104.194:44341
I20260812 06:37:17.548833 24002 heartbeater.cc:344] Connected to a master server at 127.19.104.254:40901
I20260812 06:37:17.548926 24002 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:17.549361 24002 heartbeater.cc:507] Master 127.19.104.254:40901 requested a full tablet report, sending...
I20260812 06:37:17.550071 23419 ts_manager.cc:194] Registered new tserver with Master: 4c86452892b149c0b97f0d869965dcc5 (127.19.104.194:44771)
W20260812 06:37:17.657611 24008 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:17.657835 24008 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:17.657910 24008 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:17.661340 24008 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:17.661496 24008 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.104.195
I20260812 06:37:17.665369 24008 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.104.212:39791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.104.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.104.195
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.19.104.254:40901,127.19.104.253:36501,127.19.104.252:40219
--never_fsync=true
--heap_profile_path=/tmp/kudu.24008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.104.195

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:17.666440 24008 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:17.667575 24008 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:17.674063 24016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:17.674223 24008 server_base.cc:1061] running on GCE node
W20260812 06:37:17.674151 24018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:17.674081 24015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:17.674786 24008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:17.675275 24008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:17.676407 24008 hybrid_clock.cc:648] HybridClock initialized: now 1786516637676390 us; error 30 us; skew 500 ppm
I20260812 06:37:17.678177 24008 webserver.cc:533] Webserver started at http://127.19.104.195:38195/ using document root <none> and password file <none>
I20260812 06:37:17.678683 24008 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:17.678735 24008 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:17.678973 24008 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:17.680621 24008 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-2/data/instance:
uuid: "9f3aa114cdbb404f93f1ab99b5ee0b48"
format_stamp: "Formatted at 2026-08-12 06:37:17 on dist-test-slave-8n49"
I20260812 06:37:17.681187 24008 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-2/wal/instance:
uuid: "9f3aa114cdbb404f93f1ab99b5ee0b48"
format_stamp: "Formatted at 2026-08-12 06:37:17 on dist-test-slave-8n49"
I20260812 06:37:17.683952 24026 raft_consensus.cc:493] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 75c24f176e7f4b22a21593548438edc7)
I20260812 06:37:17.684062 24026 raft_consensus.cc:515] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } }
I20260812 06:37:17.684374 24028 raft_consensus.cc:493] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 75c24f176e7f4b22a21593548438edc7)
I20260812 06:37:17.684600 24028 raft_consensus.cc:515] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } }
I20260812 06:37:17.685081 24008 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.001s
I20260812 06:37:17.685601 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "884e3c5e5de4458495dd8f10137050e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d5f910a7236c45f3af3ce03936252ee8" is_pre_election: true
I20260812 06:37:17.685657 24026 leader_election.cc:290] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d5f910a7236c45f3af3ce03936252ee8 (127.19.104.253:36501), 75c24f176e7f4b22a21593548438edc7 (127.19.104.252:40219)
I20260812 06:37:17.685693 24028 leader_election.cc:290] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884e3c5e5de4458495dd8f10137050e3 (127.19.104.254:40901), 75c24f176e7f4b22a21593548438edc7 (127.19.104.252:40219)
I20260812 06:37:17.685722 23518 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884e3c5e5de4458495dd8f10137050e3 in term 1.
I20260812 06:37:17.686148 23430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d5f910a7236c45f3af3ce03936252ee8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "884e3c5e5de4458495dd8f10137050e3" is_pre_election: true
I20260812 06:37:17.686275 23430 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5f910a7236c45f3af3ce03936252ee8 in term 1.
I20260812 06:37:17.686301 23595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d5f910a7236c45f3af3ce03936252ee8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "75c24f176e7f4b22a21593548438edc7" is_pre_election: true
I20260812 06:37:17.686440 23401 leader_election.cc:304] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884e3c5e5de4458495dd8f10137050e3, d5f910a7236c45f3af3ce03936252ee8; no voters: 
I20260812 06:37:17.686301 23596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "884e3c5e5de4458495dd8f10137050e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "75c24f176e7f4b22a21593548438edc7" is_pre_election: true
I20260812 06:37:17.686666 23489 leader_election.cc:304] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884e3c5e5de4458495dd8f10137050e3, d5f910a7236c45f3af3ce03936252ee8; no voters: 
I20260812 06:37:17.686942 24026 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 06:37:17.687052 24026 raft_consensus.cc:493] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 75c24f176e7f4b22a21593548438edc7)
I20260812 06:37:17.687135 24026 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:37:17.687471 24028 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 06:37:17.687541 24028 raft_consensus.cc:493] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 75c24f176e7f4b22a21593548438edc7)
I20260812 06:37:17.687588 24028 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:37:17.688290 24026 raft_consensus.cc:515] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } }
I20260812 06:37:17.688499 23430 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [term 2 FOLLOWER]: Rejecting Update request from peer 75c24f176e7f4b22a21593548438edc7 for earlier term 1. Current term is 2. Ops: []
I20260812 06:37:17.688740 24028 raft_consensus.cc:515] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } }
I20260812 06:37:17.688831 24026 leader_election.cc:290] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [CANDIDATE]: Term 2 election: Requested vote from peers d5f910a7236c45f3af3ce03936252ee8 (127.19.104.253:36501), 75c24f176e7f4b22a21593548438edc7 (127.19.104.252:40219)
I20260812 06:37:17.689019 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "884e3c5e5de4458495dd8f10137050e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d5f910a7236c45f3af3ce03936252ee8"
I20260812 06:37:17.689144 24028 leader_election.cc:290] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [CANDIDATE]: Term 2 election: Requested vote from peers 884e3c5e5de4458495dd8f10137050e3 (127.19.104.254:40901), 75c24f176e7f4b22a21593548438edc7 (127.19.104.252:40219)
I20260812 06:37:17.689124 23596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "884e3c5e5de4458495dd8f10137050e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "75c24f176e7f4b22a21593548438edc7"
I20260812 06:37:17.689347 24029 consensus_queue.cc:1059] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 06:37:17.689399 23430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d5f910a7236c45f3af3ce03936252ee8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "884e3c5e5de4458495dd8f10137050e3"
I20260812 06:37:17.689548 23430 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d5f910a7236c45f3af3ce03936252ee8 in current term 2: Already voted for candidate 884e3c5e5de4458495dd8f10137050e3 in this term.
I20260812 06:37:17.689165 23518 raft_consensus.cc:2393] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 884e3c5e5de4458495dd8f10137050e3 in current term 2: Already voted for candidate d5f910a7236c45f3af3ce03936252ee8 in this term.
I20260812 06:37:17.689608 23596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d5f910a7236c45f3af3ce03936252ee8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "75c24f176e7f4b22a21593548438edc7"
I20260812 06:37:17.689661 24029 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 06:37:17.689716 24029 raft_consensus.cc:740] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75c24f176e7f4b22a21593548438edc7, State: Running, Role: LEADER
I20260812 06:37:17.689903 24029 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } }
I20260812 06:37:17.690073 24030 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:17.690092 23487 leader_election.cc:304] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d5f910a7236c45f3af3ce03936252ee8; no voters: 75c24f176e7f4b22a21593548438edc7, 884e3c5e5de4458495dd8f10137050e3
I20260812 06:37:17.690219 24029 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:37:17.690248 24028 raft_consensus.cc:2749] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 06:37:17.690358 23401 leader_election.cc:304] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884e3c5e5de4458495dd8f10137050e3; no voters: 75c24f176e7f4b22a21593548438edc7, d5f910a7236c45f3af3ce03936252ee8
I20260812 06:37:17.690644 24026 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 06:37:17.691231 24008 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.003s	sys 0.000s
I20260812 06:37:17.691596 24008 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-2/wal
uuid: "9f3aa114cdbb404f93f1ab99b5ee0b48"
format_stamp: "Formatted at 2026-08-12 06:37:17 on dist-test-slave-8n49"
I20260812 06:37:17.691708 24008 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:17.709961 24008 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:17.710660 24008 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:17.710862 24008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:17.711460 24008 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:17.712641 24008 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:17.712713 24008 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:17.712788 24008 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:17.712826 24008 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:17.722584 24008 rpc_server.cc:307] RPC server started. Bound to: 127.19.104.195:39567
I20260812 06:37:17.722644 24168 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.104.195:39567 every 8 connection(s)
I20260812 06:37:17.723758 24008 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-2/data/info.pb
I20260812 06:37:17.728382 19875 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu as pid 24008
I20260812 06:37:17.728504 19875 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskaCuUsW/test-tmp/master_failover-itest.2.HmsConfigurations_MasterFailoverTest.TestClusterIdOnFailover_0.1786516586160649-19875-0/minicluster-data/ts-2/wal/instance
I20260812 06:37:17.734344 24170 heartbeater.cc:344] Connected to a master server at 127.19.104.254:40901
I20260812 06:37:17.734607 24170 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:17.735226 24170 heartbeater.cc:507] Master 127.19.104.254:40901 requested a full tablet report, sending...
I20260812 06:37:17.735242 24169 heartbeater.cc:344] Connected to a master server at 127.19.104.252:40219
I20260812 06:37:17.735357 24169 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:17.735682 24169 heartbeater.cc:507] Master 127.19.104.252:40219 requested a full tablet report, sending...
I20260812 06:37:17.735770 24172 heartbeater.cc:344] Connected to a master server at 127.19.104.253:36501
I20260812 06:37:17.736373 23419 ts_manager.cc:194] Registered new tserver with Master: 9f3aa114cdbb404f93f1ab99b5ee0b48 (127.19.104.195:39567)
I20260812 06:37:17.736373 23585 ts_manager.cc:194] Registered new tserver with Master: 9f3aa114cdbb404f93f1ab99b5ee0b48 (127.19.104.195:39567)
I20260812 06:37:17.736637 24172 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:17.736824 24172 heartbeater.cc:507] Master 127.19.104.253:36501 requested a full tablet report, sending...
I20260812 06:37:17.737437 23505 ts_manager.cc:194] Registered new tserver with Master: 9f3aa114cdbb404f93f1ab99b5ee0b48 (127.19.104.195:39567)
I20260812 06:37:17.750641 19875 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 06:37:17.878232 23846 heartbeater.cc:499] Master 127.19.104.252:40219 was elected leader, sending a full tablet report...
I20260812 06:37:17.936807 23649 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:37:17.985960 23667 catalog_manager.cc:1295] Loaded cluster ID: 8834243fea2544cf8e7adad60e39b5bc
I20260812 06:37:17.986085 23667 catalog_manager.cc:1588] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8: loading cluster ID for follower catalog manager: success
I20260812 06:37:17.987785 23667 catalog_manager.cc:1610] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8: acquiring CA information for follower catalog manager: success
I20260812 06:37:17.988864 23667 catalog_manager.cc:1638] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:37:18.014510 23681 catalog_manager.cc:1295] Loaded cluster ID: 8834243fea2544cf8e7adad60e39b5bc
I20260812 06:37:18.014606 23681 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3: loading cluster ID for follower catalog manager: success
I20260812 06:37:18.016252 23681 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3: acquiring CA information for follower catalog manager: success
I20260812 06:37:18.017325 23681 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:37:18.049954 24001 heartbeater.cc:499] Master 127.19.104.252:40219 was elected leader, sending a full tablet report...
I20260812 06:37:18.215976 24186 raft_consensus.cc:493] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:37:18.216111 24186 raft_consensus.cc:515] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } }
I20260812 06:37:18.216586 24186 leader_election.cc:290] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d5f910a7236c45f3af3ce03936252ee8 (127.19.104.253:36501), 75c24f176e7f4b22a21593548438edc7 (127.19.104.252:40219)
I20260812 06:37:18.216964 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "884e3c5e5de4458495dd8f10137050e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d5f910a7236c45f3af3ce03936252ee8" is_pre_election: true
I20260812 06:37:18.217017 23596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "884e3c5e5de4458495dd8f10137050e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "75c24f176e7f4b22a21593548438edc7" is_pre_election: true
I20260812 06:37:18.217120 23518 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884e3c5e5de4458495dd8f10137050e3 in term 2.
I20260812 06:37:18.217222 23596 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884e3c5e5de4458495dd8f10137050e3 in term 2.
I20260812 06:37:18.217514 23401 leader_election.cc:304] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884e3c5e5de4458495dd8f10137050e3, d5f910a7236c45f3af3ce03936252ee8; no voters: 
I20260812 06:37:18.217692 24186 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 06:37:18.217758 24186 raft_consensus.cc:493] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:37:18.217805 24186 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:37:18.218680 24186 raft_consensus.cc:515] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } }
I20260812 06:37:18.219105 24186 leader_election.cc:290] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [CANDIDATE]: Term 3 election: Requested vote from peers d5f910a7236c45f3af3ce03936252ee8 (127.19.104.253:36501), 75c24f176e7f4b22a21593548438edc7 (127.19.104.252:40219)
I20260812 06:37:18.219441 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "884e3c5e5de4458495dd8f10137050e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d5f910a7236c45f3af3ce03936252ee8"
I20260812 06:37:18.219484 23596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "884e3c5e5de4458495dd8f10137050e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "75c24f176e7f4b22a21593548438edc7"
I20260812 06:37:18.219578 23518 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:37:18.219587 23596 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:37:18.220650 23518 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884e3c5e5de4458495dd8f10137050e3 in term 3.
I20260812 06:37:18.220650 23596 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884e3c5e5de4458495dd8f10137050e3 in term 3.
I20260812 06:37:18.220956 23399 leader_election.cc:304] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75c24f176e7f4b22a21593548438edc7, 884e3c5e5de4458495dd8f10137050e3; no voters: 
I20260812 06:37:18.221169 24186 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 06:37:18.221401 24186 raft_consensus.cc:697] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [term 3 LEADER]: Becoming Leader. State: Replica: 884e3c5e5de4458495dd8f10137050e3, State: Running, Role: LEADER
I20260812 06:37:18.221772 24186 consensus_queue.cc:237] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } }
I20260812 06:37:18.223240 24187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 884e3c5e5de4458495dd8f10137050e3. Latest consensus state: current_term: 3 leader_uuid: "884e3c5e5de4458495dd8f10137050e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } } }
I20260812 06:37:18.223343 24187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:18.223639 24189 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:37:18.224694 24189 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:37:18.225085 24189 catalog_manager.cc:1295] Loaded cluster ID: 8834243fea2544cf8e7adad60e39b5bc
I20260812 06:37:18.225135 24189 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:37:18.225621 24189 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:37:18.225960 24189 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 884e3c5e5de4458495dd8f10137050e3: Loaded TSK: 0
I20260812 06:37:18.226516 24189 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:37:18.239727 23419 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.104.195:59467
I20260812 06:37:18.245137 19875 master_failover-itest.cc:507] Shutdown the leader master
I20260812 06:37:18.245221 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 23381
W20260812 06:37:18.374042 23847 heartbeater.cc:646] Failed to heartbeat to 127.19.104.254:40901 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.104.254:40901: connect: Connection refused (error 111)
W20260812 06:37:18.552567 24002 heartbeater.cc:646] Failed to heartbeat to 127.19.104.254:40901 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.104.254:40901: connect: Connection refused (error 111)
I20260812 06:37:18.741926 24170 heartbeater.cc:499] Master 127.19.104.254:40901 was elected leader, sending a full tablet report...
W20260812 06:37:18.742870 24170 heartbeater.cc:646] Failed to heartbeat to 127.19.104.254:40901 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.104.254:40901: connect: Connection refused (error 111)
I20260812 06:37:18.944509 24196 raft_consensus.cc:493] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:37:18.944741 24196 raft_consensus.cc:515] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } }
I20260812 06:37:18.945267 24196 leader_election.cc:290] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884e3c5e5de4458495dd8f10137050e3 (127.19.104.254:40901), d5f910a7236c45f3af3ce03936252ee8 (127.19.104.253:36501)
I20260812 06:37:18.945997 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75c24f176e7f4b22a21593548438edc7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d5f910a7236c45f3af3ce03936252ee8" is_pre_election: true
W20260812 06:37:18.946086 23571 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.104.254:40901: connect: Connection refused (error 111)
I20260812 06:37:18.946166 23518 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75c24f176e7f4b22a21593548438edc7 in term 3.
I20260812 06:37:18.946523 23570 leader_election.cc:304] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75c24f176e7f4b22a21593548438edc7, d5f910a7236c45f3af3ce03936252ee8; no voters: 
I20260812 06:37:18.946755 24196 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 06:37:18.946830 24196 raft_consensus.cc:493] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:37:18.946882 24196 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [term 3 FOLLOWER]: Advancing to term 4
W20260812 06:37:18.947039 23571 leader_election.cc:336] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 884e3c5e5de4458495dd8f10137050e3 (127.19.104.254:40901): Network error: Client connection negotiation failed: client connection to 127.19.104.254:40901: connect: Connection refused (error 111)
I20260812 06:37:18.947926 24196 raft_consensus.cc:515] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } }
I20260812 06:37:18.948351 24196 leader_election.cc:290] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [CANDIDATE]: Term 4 election: Requested vote from peers 884e3c5e5de4458495dd8f10137050e3 (127.19.104.254:40901), d5f910a7236c45f3af3ce03936252ee8 (127.19.104.253:36501)
I20260812 06:37:18.948825 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75c24f176e7f4b22a21593548438edc7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d5f910a7236c45f3af3ce03936252ee8"
I20260812 06:37:18.948985 23518 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [term 3 FOLLOWER]: Advancing to term 4
W20260812 06:37:18.949298 23571 leader_election.cc:336] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 884e3c5e5de4458495dd8f10137050e3 (127.19.104.254:40901): Network error: Client connection negotiation failed: client connection to 127.19.104.254:40901: connect: Connection refused (error 111)
I20260812 06:37:18.949931 23518 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d5f910a7236c45f3af3ce03936252ee8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75c24f176e7f4b22a21593548438edc7 in term 4.
I20260812 06:37:18.950239 23570 leader_election.cc:304] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75c24f176e7f4b22a21593548438edc7, d5f910a7236c45f3af3ce03936252ee8; no voters: 884e3c5e5de4458495dd8f10137050e3
I20260812 06:37:18.950429 24196 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 06:37:18.950613 24196 raft_consensus.cc:697] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [term 4 LEADER]: Becoming Leader. State: Replica: 75c24f176e7f4b22a21593548438edc7, State: Running, Role: LEADER
I20260812 06:37:18.950773 24196 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } }
I20260812 06:37:18.951409 24199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75c24f176e7f4b22a21593548438edc7. Latest consensus state: current_term: 4 leader_uuid: "75c24f176e7f4b22a21593548438edc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884e3c5e5de4458495dd8f10137050e3" member_type: VOTER last_known_addr { host: "127.19.104.254" port: 40901 } } peers { permanent_uuid: "d5f910a7236c45f3af3ce03936252ee8" member_type: VOTER last_known_addr { host: "127.19.104.253" port: 36501 } } peers { permanent_uuid: "75c24f176e7f4b22a21593548438edc7" member_type: VOTER last_known_addr { host: "127.19.104.252" port: 40219 } } }
I20260812 06:37:18.951526 24199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:18.951820 24201 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:37:18.952801 24201 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:37:18.953297 24201 catalog_manager.cc:1295] Loaded cluster ID: 8834243fea2544cf8e7adad60e39b5bc
I20260812 06:37:18.953347 24201 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:37:18.954061 24201 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:37:18.954483 24201 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 75c24f176e7f4b22a21593548438edc7: Loaded TSK: 0
I20260812 06:37:18.954921 24201 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:37:18.975077 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 23695
I20260812 06:37:18.986703 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 23855
I20260812 06:37:18.996142 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 24008
I20260812 06:37:19.007218 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 23464
I20260812 06:37:19.016882 19875 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskaCuUsW/build/debug/bin/kudu with pid 23550
2026-08-12T06:37:19Z chronyd exiting
[       OK ] HmsConfigurations/MasterFailoverTest.TestClusterIdOnFailover/0 (2671 ms)
[----------] 4 tests from HmsConfigurations/MasterFailoverTest (52869 ms total)

[----------] Global test environment tear-down
[==========] 4 tests from 1 test suite ran. (52869 ms total)
[  PASSED  ] 4 tests.
