[==========] Running 23 tests from 3 test suites.
[----------] Global test environment set-up.
[----------] 13 tests from RaftConsensusNonVoterITest
[ RUN      ] RaftConsensusNonVoterITest.GetTableAndTabletLocations
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260812 08:00:09.250530   745 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:00:09.250659   745 ts_itest-base.cc:116] --------------
I20260812 08:00:09.250670   745 ts_itest-base.cc:117] 4 tablet servers
I20260812 08:00:09.250681   745 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:00:09.250690   745 ts_itest-base.cc:119] --------------
2026-08-12T08:00:09Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:00:09Z Disabled control of system clock
I20260812 08:00:09.263741   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:44609
--webserver_interface=127.0.186.126
--webserver_port=0
--builtin_ntp_servers=127.0.186.84:33581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:44609
--catalog_manager_evict_excess_replicas=false with env {}
W20260812 08:00:09.375352   757 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:09.375589   757 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:09.375630   757 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:09.379094   757 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:00:09.379158   757 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:09.379177   757 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:00:09.379194   757 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:00:09.383159   757 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:33581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--catalog_manager_evict_excess_replicas=false
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:44609
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:44609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:09.384388   757 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:09.385730   757 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:09.392692   764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:09.392798   757 server_base.cc:1061] running on GCE node
W20260812 08:00:09.394162   765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:09.394236   767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:09.394903   757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:09.395998   757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:09.397156   757 hybrid_clock.cc:648] HybridClock initialized: now 1786521609397117 us; error 52 us; skew 500 ppm
I20260812 08:00:09.399032   757 webserver.cc:533] Webserver started at http://127.0.186.126:35157/ using document root <none> and password file <none>
I20260812 08:00:09.399680   757 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:09.399748   757 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:09.399948   757 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:09.401523   757 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "1aabbb95f44644ddb19692d603a2546c"
format_stamp: "Formatted at 2026-08-12 08:00:09 on dist-test-slave-49h3"
I20260812 08:00:09.401988   757 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "1aabbb95f44644ddb19692d603a2546c"
format_stamp: "Formatted at 2026-08-12 08:00:09 on dist-test-slave-49h3"
I20260812 08:00:09.405375   757 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.001s
I20260812 08:00:09.407541   773 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:09.408484   757 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:00:09.408604   757 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "1aabbb95f44644ddb19692d603a2546c"
format_stamp: "Formatted at 2026-08-12 08:00:09 on dist-test-slave-49h3"
I20260812 08:00:09.408689   757 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:09.422555   757 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:09.423257   757 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:09.423431   757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:09.432073   757 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:44609
I20260812 08:00:09.432102   841 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:44609 every 8 connection(s)
I20260812 08:00:09.433110   757 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:00:09.436301   842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:09.440189   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 757
I20260812 08:00:09.440344   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:00:09.441887   842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aabbb95f44644ddb19692d603a2546c: Bootstrap starting.
I20260812 08:00:09.444442   842 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1aabbb95f44644ddb19692d603a2546c: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:09.445302   842 log.cc:826] T 00000000000000000000000000000000 P 1aabbb95f44644ddb19692d603a2546c: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:09.447311   842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aabbb95f44644ddb19692d603a2546c: No bootstrap required, opened a new log
I20260812 08:00:09.450764   842 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1aabbb95f44644ddb19692d603a2546c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aabbb95f44644ddb19692d603a2546c" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44609 } }
I20260812 08:00:09.451129   842 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1aabbb95f44644ddb19692d603a2546c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:09.451195   842 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1aabbb95f44644ddb19692d603a2546c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aabbb95f44644ddb19692d603a2546c, State: Initialized, Role: FOLLOWER
I20260812 08:00:09.451731   842 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1aabbb95f44644ddb19692d603a2546c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aabbb95f44644ddb19692d603a2546c" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44609 } }
I20260812 08:00:09.451862   842 raft_consensus.cc:399] T 00000000000000000000000000000000 P 1aabbb95f44644ddb19692d603a2546c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:00:09.451922   842 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1aabbb95f44644ddb19692d603a2546c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:00:09.452036   842 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1aabbb95f44644ddb19692d603a2546c [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:09.452850   842 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1aabbb95f44644ddb19692d603a2546c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aabbb95f44644ddb19692d603a2546c" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44609 } }
I20260812 08:00:09.453213   842 leader_election.cc:304] T 00000000000000000000000000000000 P 1aabbb95f44644ddb19692d603a2546c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1aabbb95f44644ddb19692d603a2546c; no voters: 
I20260812 08:00:09.453540   842 leader_election.cc:290] T 00000000000000000000000000000000 P 1aabbb95f44644ddb19692d603a2546c [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:00:09.453652   848 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1aabbb95f44644ddb19692d603a2546c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:00:09.453876   848 raft_consensus.cc:697] T 00000000000000000000000000000000 P 1aabbb95f44644ddb19692d603a2546c [term 1 LEADER]: Becoming Leader. State: Replica: 1aabbb95f44644ddb19692d603a2546c, State: Running, Role: LEADER
I20260812 08:00:09.454213   848 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1aabbb95f44644ddb19692d603a2546c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aabbb95f44644ddb19692d603a2546c" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44609 } }
I20260812 08:00:09.454886   842 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1aabbb95f44644ddb19692d603a2546c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:00:09.456568   849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aabbb95f44644ddb19692d603a2546c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1aabbb95f44644ddb19692d603a2546c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aabbb95f44644ddb19692d603a2546c" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44609 } } }
I20260812 08:00:09.456696   849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aabbb95f44644ddb19692d603a2546c [sys.catalog]: This master's current role is: LEADER
I20260812 08:00:09.457697   859 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:00:09.457916   850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aabbb95f44644ddb19692d603a2546c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1aabbb95f44644ddb19692d603a2546c. Latest consensus state: current_term: 1 leader_uuid: "1aabbb95f44644ddb19692d603a2546c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aabbb95f44644ddb19692d603a2546c" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44609 } } }
I20260812 08:00:09.458003   850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aabbb95f44644ddb19692d603a2546c [sys.catalog]: This master's current role is: LEADER
I20260812 08:00:09.460259   859 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:00:09.465601   859 catalog_manager.cc:1383] Generated new cluster ID: eebef9b4f0384bf99349e0cf19350dd1
I20260812 08:00:09.465667   859 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:00:09.486761   859 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:00:09.487996   859 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:00:09.499234   859 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 1aabbb95f44644ddb19692d603a2546c: Generated new TSK 0
I20260812 08:00:09.499991   859 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:00:09.510004   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:0
--local_ip_for_outbound_sockets=127.0.186.65
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:44609
--builtin_ntp_servers=127.0.186.84:33581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=50 with env {}
W20260812 08:00:09.659047   874 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:09.659343   874 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:09.659402   874 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:09.664168   874 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:09.664331   874 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:00:09.670027   874 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:33581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=50
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:44609
--never_fsync=true
--heap_profile_path=/tmp/kudu.874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:09.671392   874 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:09.672762   874 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:09.680732   880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:09.680851   881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:09.681142   883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:09.681715   874 server_base.cc:1061] running on GCE node
I20260812 08:00:09.682186   874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:09.682957   874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:09.684149   874 hybrid_clock.cc:648] HybridClock initialized: now 1786521609684122 us; error 37 us; skew 500 ppm
I20260812 08:00:09.686484   874 webserver.cc:533] Webserver started at http://127.0.186.65:45215/ using document root <none> and password file <none>
I20260812 08:00:09.687278   874 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:09.687353   874 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:09.687570   874 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:09.689811   874 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "e3cc42012c014d17992d7442d18a0c18"
format_stamp: "Formatted at 2026-08-12 08:00:09 on dist-test-slave-49h3"
I20260812 08:00:09.690438   874 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "e3cc42012c014d17992d7442d18a0c18"
format_stamp: "Formatted at 2026-08-12 08:00:09 on dist-test-slave-49h3"
I20260812 08:00:09.695101   874 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260812 08:00:09.697991   891 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:09.699054   874 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:00:09.699172   874 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "e3cc42012c014d17992d7442d18a0c18"
format_stamp: "Formatted at 2026-08-12 08:00:09 on dist-test-slave-49h3"
I20260812 08:00:09.699280   874 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:09.723155   874 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:09.724035   874 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:09.724218   874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:09.724946   874 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:09.726138   874 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:09.726189   874 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:09.726246   874 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:09.726270   874 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:09.744575   874 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:33751
I20260812 08:00:09.744596  1026 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:33751 every 8 connection(s)
I20260812 08:00:09.745697   874 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:00:09.748036   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 874
I20260812 08:00:09.748144   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:00:09.750800   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.66:0
--local_ip_for_outbound_sockets=127.0.186.66
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:44609
--builtin_ntp_servers=127.0.186.84:33581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=50 with env {}
I20260812 08:00:09.755651  1027 heartbeater.cc:344] Connected to a master server at 127.0.186.126:44609
I20260812 08:00:09.755946  1027 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:09.756534  1027 heartbeater.cc:507] Master 127.0.186.126:44609 requested a full tablet report, sending...
I20260812 08:00:09.758278   796 ts_manager.cc:194] Registered new tserver with Master: e3cc42012c014d17992d7442d18a0c18 (127.0.186.65:33751)
I20260812 08:00:09.759503   796 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.65:46961
W20260812 08:00:09.883204  1030 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:09.883440  1030 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:09.883481  1030 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:09.886927  1030 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:09.887048  1030 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.66
I20260812 08:00:09.891026  1030 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:33581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=50
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:44609
--never_fsync=true
--heap_profile_path=/tmp/kudu.1030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:09.892175  1030 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:09.893348  1030 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:09.900079  1035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:09.900096  1039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:09.900097  1036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:09.900460  1030 server_base.cc:1061] running on GCE node
I20260812 08:00:09.900806  1030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:09.901357  1030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:09.902520  1030 hybrid_clock.cc:648] HybridClock initialized: now 1786521609902502 us; error 35 us; skew 500 ppm
I20260812 08:00:09.904413  1030 webserver.cc:533] Webserver started at http://127.0.186.66:42249/ using document root <none> and password file <none>
I20260812 08:00:09.904994  1030 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:09.905054  1030 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:09.905253  1030 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:09.906921  1030 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "b4d48a1d2ca64f65b400acd9b1fb8f1c"
format_stamp: "Formatted at 2026-08-12 08:00:09 on dist-test-slave-49h3"
I20260812 08:00:09.907409  1030 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "b4d48a1d2ca64f65b400acd9b1fb8f1c"
format_stamp: "Formatted at 2026-08-12 08:00:09 on dist-test-slave-49h3"
I20260812 08:00:09.910879  1030 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:00:09.913128  1047 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:09.914237  1030 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:00:09.914361  1030 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "b4d48a1d2ca64f65b400acd9b1fb8f1c"
format_stamp: "Formatted at 2026-08-12 08:00:09 on dist-test-slave-49h3"
I20260812 08:00:09.914456  1030 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:09.944335  1030 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:09.945083  1030 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:09.945277  1030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:09.946095  1030 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:09.947350  1030 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:09.947402  1030 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:09.947441  1030 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:09.947461  1030 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:09.962920  1030 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.66:45677
I20260812 08:00:09.962941  1184 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.66:45677 every 8 connection(s)
I20260812 08:00:09.963963  1030 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:00:09.970764   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 1030
I20260812 08:00:09.970876   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:00:09.972622  1185 heartbeater.cc:344] Connected to a master server at 127.0.186.126:44609
I20260812 08:00:09.972889  1185 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:09.973184   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.67:0
--local_ip_for_outbound_sockets=127.0.186.67
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:44609
--builtin_ntp_servers=127.0.186.84:33581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=50 with env {}
I20260812 08:00:09.973489  1185 heartbeater.cc:507] Master 127.0.186.126:44609 requested a full tablet report, sending...
I20260812 08:00:09.974637   796 ts_manager.cc:194] Registered new tserver with Master: b4d48a1d2ca64f65b400acd9b1fb8f1c (127.0.186.66:45677)
I20260812 08:00:09.975265   796 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.66:40315
W20260812 08:00:10.082939  1189 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:10.083160  1189 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:10.083218  1189 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:10.086514  1189 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:10.086637  1189 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.67
I20260812 08:00:10.090418  1189 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:33581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=50
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:44609
--never_fsync=true
--heap_profile_path=/tmp/kudu.1189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:10.091508  1189 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:10.092639  1189 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:10.099854  1197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:10.099859  1196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:10.099946  1199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:10.100399  1189 server_base.cc:1061] running on GCE node
I20260812 08:00:10.100776  1189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:10.101330  1189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:10.102474  1189 hybrid_clock.cc:648] HybridClock initialized: now 1786521610102462 us; error 26 us; skew 500 ppm
I20260812 08:00:10.104234  1189 webserver.cc:533] Webserver started at http://127.0.186.67:40747/ using document root <none> and password file <none>
I20260812 08:00:10.104827  1189 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:10.104879  1189 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:10.105072  1189 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:10.106700  1189 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "96ae1238e2634cb59f7cd931928f2b6c"
format_stamp: "Formatted at 2026-08-12 08:00:10 on dist-test-slave-49h3"
I20260812 08:00:10.107174  1189 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "96ae1238e2634cb59f7cd931928f2b6c"
format_stamp: "Formatted at 2026-08-12 08:00:10 on dist-test-slave-49h3"
I20260812 08:00:10.110648  1189 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:00:10.112839  1207 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:10.113971  1189 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:00:10.114075  1189 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "96ae1238e2634cb59f7cd931928f2b6c"
format_stamp: "Formatted at 2026-08-12 08:00:10 on dist-test-slave-49h3"
I20260812 08:00:10.114171  1189 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:10.137763  1189 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:10.138572  1189 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:10.138758  1189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:10.139398  1189 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:10.140482  1189 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:10.140530  1189 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:10.140575  1189 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:10.140604  1189 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:10.153761  1189 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.67:36381
I20260812 08:00:10.153785  1339 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.67:36381 every 8 connection(s)
I20260812 08:00:10.154836  1189 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:00:10.161635   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 1189
I20260812 08:00:10.161765   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:00:10.163160  1340 heartbeater.cc:344] Connected to a master server at 127.0.186.126:44609
I20260812 08:00:10.163425  1340 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:10.164063  1340 heartbeater.cc:507] Master 127.0.186.126:44609 requested a full tablet report, sending...
I20260812 08:00:10.164259   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.68:0
--local_ip_for_outbound_sockets=127.0.186.68
--webserver_interface=127.0.186.68
--webserver_port=0
--tserver_master_addrs=127.0.186.126:44609
--builtin_ntp_servers=127.0.186.84:33581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=50 with env {}
I20260812 08:00:10.165246   795 ts_manager.cc:194] Registered new tserver with Master: 96ae1238e2634cb59f7cd931928f2b6c (127.0.186.67:36381)
I20260812 08:00:10.165845   795 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.67:42975
W20260812 08:00:10.273957  1344 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:10.274181  1344 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:10.274242  1344 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:10.277570  1344 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:10.277693  1344 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.68
I20260812 08:00:10.281491  1344 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:33581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=50
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.0.186.68
--webserver_port=0
--tserver_master_addrs=127.0.186.126:44609
--never_fsync=true
--heap_profile_path=/tmp/kudu.1344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.68

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:10.282617  1344 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:10.283790  1344 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:10.290436  1354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:10.290474  1349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:10.290445  1350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:10.290609  1344 server_base.cc:1061] running on GCE node
I20260812 08:00:10.291107  1344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:10.291673  1344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:10.292831  1344 hybrid_clock.cc:648] HybridClock initialized: now 1786521610292815 us; error 30 us; skew 500 ppm
I20260812 08:00:10.294790  1344 webserver.cc:533] Webserver started at http://127.0.186.68:46011/ using document root <none> and password file <none>
I20260812 08:00:10.295397  1344 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:10.295461  1344 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:10.295655  1344 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:10.297329  1344 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "74372458b42743949acba52df7dc99da"
format_stamp: "Formatted at 2026-08-12 08:00:10 on dist-test-slave-49h3"
I20260812 08:00:10.297900  1344 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "74372458b42743949acba52df7dc99da"
format_stamp: "Formatted at 2026-08-12 08:00:10 on dist-test-slave-49h3"
I20260812 08:00:10.301411  1344 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:00:10.303629  1363 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:10.304622  1344 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:00:10.304728  1344 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "74372458b42743949acba52df7dc99da"
format_stamp: "Formatted at 2026-08-12 08:00:10 on dist-test-slave-49h3"
I20260812 08:00:10.304821  1344 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:10.332748  1344 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:10.333498  1344 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:10.333683  1344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:10.334367  1344 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:10.335559  1344 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:10.335613  1344 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:10.335667  1344 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:10.335693  1344 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:10.349423  1344 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.68:32853
I20260812 08:00:10.349471  1499 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.68:32853 every 8 connection(s)
I20260812 08:00:10.350778  1344 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20260812 08:00:10.352088   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 1344
I20260812 08:00:10.352216   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.GetTableAndTabletLocations.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20260812 08:00:10.360769  1500 heartbeater.cc:344] Connected to a master server at 127.0.186.126:44609
I20260812 08:00:10.361035  1500 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:10.361627  1500 heartbeater.cc:507] Master 127.0.186.126:44609 requested a full tablet report, sending...
I20260812 08:00:10.362764   795 ts_manager.cc:194] Registered new tserver with Master: 74372458b42743949acba52df7dc99da (127.0.186.68:32853)
I20260812 08:00:10.363302   795 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.68:40993
I20260812 08:00:10.367416   745 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20260812 08:00:10.380142   795 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35698:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
I20260812 08:00:10.400466  1262 tablet_service.cc:1511] Processing CreateTablet for tablet c0c48bd113d54ca4beadb425c79c109d (DEFAULT_TABLE table=TestTable [id=fba480e153de488aac2cfc6d9b537837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:10.400514   949 tablet_service.cc:1511] Processing CreateTablet for tablet c0c48bd113d54ca4beadb425c79c109d (DEFAULT_TABLE table=TestTable [id=fba480e153de488aac2cfc6d9b537837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:10.400471  1106 tablet_service.cc:1511] Processing CreateTablet for tablet c0c48bd113d54ca4beadb425c79c109d (DEFAULT_TABLE table=TestTable [id=fba480e153de488aac2cfc6d9b537837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:10.401438  1262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0c48bd113d54ca4beadb425c79c109d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:10.401480  1106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0c48bd113d54ca4beadb425c79c109d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:10.401485   949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0c48bd113d54ca4beadb425c79c109d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:10.406862  1522 tablet_bootstrap.cc:492] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c: Bootstrap starting.
I20260812 08:00:10.407557  1524 tablet_bootstrap.cc:492] T c0c48bd113d54ca4beadb425c79c109d P b4d48a1d2ca64f65b400acd9b1fb8f1c: Bootstrap starting.
I20260812 08:00:10.408107  1523 tablet_bootstrap.cc:492] T c0c48bd113d54ca4beadb425c79c109d P e3cc42012c014d17992d7442d18a0c18: Bootstrap starting.
I20260812 08:00:10.409737  1522 tablet_bootstrap.cc:654] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:10.409878  1524 tablet_bootstrap.cc:654] T c0c48bd113d54ca4beadb425c79c109d P b4d48a1d2ca64f65b400acd9b1fb8f1c: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:10.410159  1523 tablet_bootstrap.cc:654] T c0c48bd113d54ca4beadb425c79c109d P e3cc42012c014d17992d7442d18a0c18: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:10.410542  1522 log.cc:826] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:10.410696  1524 log.cc:826] T c0c48bd113d54ca4beadb425c79c109d P b4d48a1d2ca64f65b400acd9b1fb8f1c: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:10.410856  1523 log.cc:826] T c0c48bd113d54ca4beadb425c79c109d P e3cc42012c014d17992d7442d18a0c18: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:10.412432  1522 tablet_bootstrap.cc:492] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c: No bootstrap required, opened a new log
I20260812 08:00:10.412432  1524 tablet_bootstrap.cc:492] T c0c48bd113d54ca4beadb425c79c109d P b4d48a1d2ca64f65b400acd9b1fb8f1c: No bootstrap required, opened a new log
I20260812 08:00:10.412638  1522 ts_tablet_manager.cc:1403] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c: Time spent bootstrapping tablet: real 0.006s	user 0.004s	sys 0.000s
I20260812 08:00:10.412648  1524 ts_tablet_manager.cc:1403] T c0c48bd113d54ca4beadb425c79c109d P b4d48a1d2ca64f65b400acd9b1fb8f1c: Time spent bootstrapping tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 08:00:10.413779  1523 tablet_bootstrap.cc:492] T c0c48bd113d54ca4beadb425c79c109d P e3cc42012c014d17992d7442d18a0c18: No bootstrap required, opened a new log
I20260812 08:00:10.413959  1523 ts_tablet_manager.cc:1403] T c0c48bd113d54ca4beadb425c79c109d P e3cc42012c014d17992d7442d18a0c18: Time spent bootstrapping tablet: real 0.006s	user 0.004s	sys 0.000s
I20260812 08:00:10.415475  1522 raft_consensus.cc:359] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3cc42012c014d17992d7442d18a0c18" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 33751 } } peers { permanent_uuid: "b4d48a1d2ca64f65b400acd9b1fb8f1c" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 45677 } } peers { permanent_uuid: "96ae1238e2634cb59f7cd931928f2b6c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36381 } }
I20260812 08:00:10.415710  1522 raft_consensus.cc:385] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:10.415822  1522 raft_consensus.cc:740] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96ae1238e2634cb59f7cd931928f2b6c, State: Initialized, Role: FOLLOWER
I20260812 08:00:10.416281  1522 consensus_queue.cc:260] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3cc42012c014d17992d7442d18a0c18" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 33751 } } peers { permanent_uuid: "b4d48a1d2ca64f65b400acd9b1fb8f1c" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 45677 } } peers { permanent_uuid: "96ae1238e2634cb59f7cd931928f2b6c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36381 } }
I20260812 08:00:10.416898  1523 raft_consensus.cc:359] T c0c48bd113d54ca4beadb425c79c109d P e3cc42012c014d17992d7442d18a0c18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3cc42012c014d17992d7442d18a0c18" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 33751 } } peers { permanent_uuid: "b4d48a1d2ca64f65b400acd9b1fb8f1c" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 45677 } } peers { permanent_uuid: "96ae1238e2634cb59f7cd931928f2b6c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36381 } }
I20260812 08:00:10.417065  1340 heartbeater.cc:499] Master 127.0.186.126:44609 was elected leader, sending a full tablet report...
I20260812 08:00:10.417100  1523 raft_consensus.cc:385] T c0c48bd113d54ca4beadb425c79c109d P e3cc42012c014d17992d7442d18a0c18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:10.417194  1523 raft_consensus.cc:740] T c0c48bd113d54ca4beadb425c79c109d P e3cc42012c014d17992d7442d18a0c18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3cc42012c014d17992d7442d18a0c18, State: Initialized, Role: FOLLOWER
I20260812 08:00:10.417260  1522 ts_tablet_manager.cc:1434] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:00:10.417534  1524 raft_consensus.cc:359] T c0c48bd113d54ca4beadb425c79c109d P b4d48a1d2ca64f65b400acd9b1fb8f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3cc42012c014d17992d7442d18a0c18" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 33751 } } peers { permanent_uuid: "b4d48a1d2ca64f65b400acd9b1fb8f1c" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 45677 } } peers { permanent_uuid: "96ae1238e2634cb59f7cd931928f2b6c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36381 } }
I20260812 08:00:10.417620  1523 consensus_queue.cc:260] T c0c48bd113d54ca4beadb425c79c109d P e3cc42012c014d17992d7442d18a0c18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3cc42012c014d17992d7442d18a0c18" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 33751 } } peers { permanent_uuid: "b4d48a1d2ca64f65b400acd9b1fb8f1c" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 45677 } } peers { permanent_uuid: "96ae1238e2634cb59f7cd931928f2b6c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36381 } }
I20260812 08:00:10.417760  1524 raft_consensus.cc:385] T c0c48bd113d54ca4beadb425c79c109d P b4d48a1d2ca64f65b400acd9b1fb8f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:10.417834  1524 raft_consensus.cc:740] T c0c48bd113d54ca4beadb425c79c109d P b4d48a1d2ca64f65b400acd9b1fb8f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4d48a1d2ca64f65b400acd9b1fb8f1c, State: Initialized, Role: FOLLOWER
I20260812 08:00:10.418339  1524 consensus_queue.cc:260] T c0c48bd113d54ca4beadb425c79c109d P b4d48a1d2ca64f65b400acd9b1fb8f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3cc42012c014d17992d7442d18a0c18" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 33751 } } peers { permanent_uuid: "b4d48a1d2ca64f65b400acd9b1fb8f1c" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 45677 } } peers { permanent_uuid: "96ae1238e2634cb59f7cd931928f2b6c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36381 } }
I20260812 08:00:10.418534  1523 ts_tablet_manager.cc:1434] T c0c48bd113d54ca4beadb425c79c109d P e3cc42012c014d17992d7442d18a0c18: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:00:10.418900  1027 heartbeater.cc:499] Master 127.0.186.126:44609 was elected leader, sending a full tablet report...
I20260812 08:00:10.419064  1185 heartbeater.cc:499] Master 127.0.186.126:44609 was elected leader, sending a full tablet report...
I20260812 08:00:10.419819  1524 ts_tablet_manager.cc:1434] T c0c48bd113d54ca4beadb425c79c109d P b4d48a1d2ca64f65b400acd9b1fb8f1c: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260812 08:00:10.463856  1529 raft_consensus.cc:493] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:00:10.464079  1529 raft_consensus.cc:515] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3cc42012c014d17992d7442d18a0c18" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 33751 } } peers { permanent_uuid: "b4d48a1d2ca64f65b400acd9b1fb8f1c" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 45677 } } peers { permanent_uuid: "96ae1238e2634cb59f7cd931928f2b6c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36381 } }
I20260812 08:00:10.464896  1530 raft_consensus.cc:493] T c0c48bd113d54ca4beadb425c79c109d P e3cc42012c014d17992d7442d18a0c18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:00:10.465103  1530 raft_consensus.cc:515] T c0c48bd113d54ca4beadb425c79c109d P e3cc42012c014d17992d7442d18a0c18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3cc42012c014d17992d7442d18a0c18" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 33751 } } peers { permanent_uuid: "b4d48a1d2ca64f65b400acd9b1fb8f1c" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 45677 } } peers { permanent_uuid: "96ae1238e2634cb59f7cd931928f2b6c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36381 } }
W20260812 08:00:10.465157  1186 tablet.cc:2406] T c0c48bd113d54ca4beadb425c79c109d P b4d48a1d2ca64f65b400acd9b1fb8f1c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:00:10.465422  1529 leader_election.cc:290] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3cc42012c014d17992d7442d18a0c18 (127.0.186.65:33751), b4d48a1d2ca64f65b400acd9b1fb8f1c (127.0.186.66:45677)
I20260812 08:00:10.466365  1530 leader_election.cc:290] T c0c48bd113d54ca4beadb425c79c109d P e3cc42012c014d17992d7442d18a0c18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4d48a1d2ca64f65b400acd9b1fb8f1c (127.0.186.66:45677), 96ae1238e2634cb59f7cd931928f2b6c (127.0.186.67:36381)
I20260812 08:00:10.471104   974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c48bd113d54ca4beadb425c79c109d" candidate_uuid: "96ae1238e2634cb59f7cd931928f2b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3cc42012c014d17992d7442d18a0c18" is_pre_election: true
I20260812 08:00:10.471370   974 raft_consensus.cc:2468] T c0c48bd113d54ca4beadb425c79c109d P e3cc42012c014d17992d7442d18a0c18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96ae1238e2634cb59f7cd931928f2b6c in term 0.
I20260812 08:00:10.471642  1131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c48bd113d54ca4beadb425c79c109d" candidate_uuid: "96ae1238e2634cb59f7cd931928f2b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d48a1d2ca64f65b400acd9b1fb8f1c" is_pre_election: true
I20260812 08:00:10.471719  1129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c48bd113d54ca4beadb425c79c109d" candidate_uuid: "e3cc42012c014d17992d7442d18a0c18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d48a1d2ca64f65b400acd9b1fb8f1c" is_pre_election: true
I20260812 08:00:10.471928  1131 raft_consensus.cc:2468] T c0c48bd113d54ca4beadb425c79c109d P b4d48a1d2ca64f65b400acd9b1fb8f1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96ae1238e2634cb59f7cd931928f2b6c in term 0.
I20260812 08:00:10.471899  1208 leader_election.cc:304] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96ae1238e2634cb59f7cd931928f2b6c, e3cc42012c014d17992d7442d18a0c18; no voters: 
I20260812 08:00:10.472219  1529 raft_consensus.cc:2804] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:00:10.472293  1529 raft_consensus.cc:493] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:00:10.472338  1529 raft_consensus.cc:3060] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:10.472787  1285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c48bd113d54ca4beadb425c79c109d" candidate_uuid: "e3cc42012c014d17992d7442d18a0c18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96ae1238e2634cb59f7cd931928f2b6c" is_pre_election: true
I20260812 08:00:10.473482  1529 raft_consensus.cc:515] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3cc42012c014d17992d7442d18a0c18" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 33751 } } peers { permanent_uuid: "b4d48a1d2ca64f65b400acd9b1fb8f1c" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 45677 } } peers { permanent_uuid: "96ae1238e2634cb59f7cd931928f2b6c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36381 } }
I20260812 08:00:10.473744  1285 raft_consensus.cc:2393] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e3cc42012c014d17992d7442d18a0c18 in current term 1: Already voted for candidate 96ae1238e2634cb59f7cd931928f2b6c in this term.
I20260812 08:00:10.473875  1529 leader_election.cc:290] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c [CANDIDATE]: Term 1 election: Requested vote from peers e3cc42012c014d17992d7442d18a0c18 (127.0.186.65:33751), b4d48a1d2ca64f65b400acd9b1fb8f1c (127.0.186.66:45677)
I20260812 08:00:10.474210   974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c48bd113d54ca4beadb425c79c109d" candidate_uuid: "96ae1238e2634cb59f7cd931928f2b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3cc42012c014d17992d7442d18a0c18"
I20260812 08:00:10.474227  1129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c48bd113d54ca4beadb425c79c109d" candidate_uuid: "96ae1238e2634cb59f7cd931928f2b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d48a1d2ca64f65b400acd9b1fb8f1c"
I20260812 08:00:10.474222   894 leader_election.cc:304] T c0c48bd113d54ca4beadb425c79c109d P e3cc42012c014d17992d7442d18a0c18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e3cc42012c014d17992d7442d18a0c18; no voters: 96ae1238e2634cb59f7cd931928f2b6c, b4d48a1d2ca64f65b400acd9b1fb8f1c
I20260812 08:00:10.474349   974 raft_consensus.cc:3060] T c0c48bd113d54ca4beadb425c79c109d P e3cc42012c014d17992d7442d18a0c18 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:10.474354  1129 raft_consensus.cc:3060] T c0c48bd113d54ca4beadb425c79c109d P b4d48a1d2ca64f65b400acd9b1fb8f1c [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:10.475219  1129 raft_consensus.cc:2468] T c0c48bd113d54ca4beadb425c79c109d P b4d48a1d2ca64f65b400acd9b1fb8f1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96ae1238e2634cb59f7cd931928f2b6c in term 1.
I20260812 08:00:10.475350   974 raft_consensus.cc:2468] T c0c48bd113d54ca4beadb425c79c109d P e3cc42012c014d17992d7442d18a0c18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96ae1238e2634cb59f7cd931928f2b6c in term 1.
I20260812 08:00:10.475525  1210 leader_election.cc:304] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96ae1238e2634cb59f7cd931928f2b6c, b4d48a1d2ca64f65b400acd9b1fb8f1c; no voters: 
I20260812 08:00:10.475555  1530 raft_consensus.cc:2749] T c0c48bd113d54ca4beadb425c79c109d P e3cc42012c014d17992d7442d18a0c18 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 08:00:10.475759  1529 raft_consensus.cc:2804] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:00:10.476089  1529 raft_consensus.cc:697] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c [term 1 LEADER]: Becoming Leader. State: Replica: 96ae1238e2634cb59f7cd931928f2b6c, State: Running, Role: LEADER
I20260812 08:00:10.476390  1529 consensus_queue.cc:237] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3cc42012c014d17992d7442d18a0c18" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 33751 } } peers { permanent_uuid: "b4d48a1d2ca64f65b400acd9b1fb8f1c" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 45677 } } peers { permanent_uuid: "96ae1238e2634cb59f7cd931928f2b6c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36381 } }
I20260812 08:00:10.479763   794 catalog_manager.cc:5719] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 96ae1238e2634cb59f7cd931928f2b6c (127.0.186.67). New cstate: current_term: 1 leader_uuid: "96ae1238e2634cb59f7cd931928f2b6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3cc42012c014d17992d7442d18a0c18" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 33751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4d48a1d2ca64f65b400acd9b1fb8f1c" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96ae1238e2634cb59f7cd931928f2b6c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36381 } health_report { overall_health: HEALTHY } } }
I20260812 08:00:10.496812   745 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
W20260812 08:00:10.497351  1028 tablet.cc:2406] T c0c48bd113d54ca4beadb425c79c109d P e3cc42012c014d17992d7442d18a0c18: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:00:10.498608   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver e3cc42012c014d17992d7442d18a0c18 to finish bootstrapping
I20260812 08:00:10.504151   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver b4d48a1d2ca64f65b400acd9b1fb8f1c to finish bootstrapping
I20260812 08:00:10.508806   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 96ae1238e2634cb59f7cd931928f2b6c to finish bootstrapping
W20260812 08:00:10.656019  1342 tablet.cc:2406] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:00:10.883461  1529 consensus_queue.cc:1048] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3cc42012c014d17992d7442d18a0c18" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 33751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:00:10.889158  1547 consensus_queue.cc:1048] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d48a1d2ca64f65b400acd9b1fb8f1c" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:00:10.891723  1285 consensus_queue.cc:237] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3cc42012c014d17992d7442d18a0c18" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 33751 } } peers { permanent_uuid: "b4d48a1d2ca64f65b400acd9b1fb8f1c" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 45677 } } peers { permanent_uuid: "96ae1238e2634cb59f7cd931928f2b6c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36381 } } peers { permanent_uuid: "74372458b42743949acba52df7dc99da" member_type: NON_VOTER last_known_addr { host: "127.0.186.68" port: 32853 } attrs { } }
I20260812 08:00:10.893405  1129 raft_consensus.cc:1275] T c0c48bd113d54ca4beadb425c79c109d P b4d48a1d2ca64f65b400acd9b1fb8f1c [term 1 FOLLOWER]: Refusing update from remote peer 96ae1238e2634cb59f7cd931928f2b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:00:10.893563   974 raft_consensus.cc:1275] T c0c48bd113d54ca4beadb425c79c109d P e3cc42012c014d17992d7442d18a0c18 [term 1 FOLLOWER]: Refusing update from remote peer 96ae1238e2634cb59f7cd931928f2b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:00:10.893865  1547 consensus_queue.cc:1048] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d48a1d2ca64f65b400acd9b1fb8f1c" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:00:10.893996  1529 consensus_queue.cc:1048] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3cc42012c014d17992d7442d18a0c18" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 33751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:00:10.895629  1539 raft_consensus.cc:2955] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74372458b42743949acba52df7dc99da (127.0.186.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3cc42012c014d17992d7442d18a0c18" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 33751 } } peers { permanent_uuid: "b4d48a1d2ca64f65b400acd9b1fb8f1c" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 45677 } } peers { permanent_uuid: "96ae1238e2634cb59f7cd931928f2b6c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36381 } } peers { permanent_uuid: "74372458b42743949acba52df7dc99da" member_type: NON_VOTER last_known_addr { host: "127.0.186.68" port: 32853 } attrs { } } }
I20260812 08:00:10.896096   974 raft_consensus.cc:2955] T c0c48bd113d54ca4beadb425c79c109d P e3cc42012c014d17992d7442d18a0c18 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74372458b42743949acba52df7dc99da (127.0.186.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3cc42012c014d17992d7442d18a0c18" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 33751 } } peers { permanent_uuid: "b4d48a1d2ca64f65b400acd9b1fb8f1c" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 45677 } } peers { permanent_uuid: "96ae1238e2634cb59f7cd931928f2b6c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36381 } } peers { permanent_uuid: "74372458b42743949acba52df7dc99da" member_type: NON_VOTER last_known_addr { host: "127.0.186.68" port: 32853 } attrs { } } }
I20260812 08:00:10.896169  1129 raft_consensus.cc:2955] T c0c48bd113d54ca4beadb425c79c109d P b4d48a1d2ca64f65b400acd9b1fb8f1c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74372458b42743949acba52df7dc99da (127.0.186.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3cc42012c014d17992d7442d18a0c18" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 33751 } } peers { permanent_uuid: "b4d48a1d2ca64f65b400acd9b1fb8f1c" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 45677 } } peers { permanent_uuid: "96ae1238e2634cb59f7cd931928f2b6c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36381 } } peers { permanent_uuid: "74372458b42743949acba52df7dc99da" member_type: NON_VOTER last_known_addr { host: "127.0.186.68" port: 32853 } attrs { } } }
I20260812 08:00:10.899029   794 catalog_manager.cc:5719] T c0c48bd113d54ca4beadb425c79c109d P b4d48a1d2ca64f65b400acd9b1fb8f1c reported cstate change: config changed from index -1 to 2, NON_VOTER 74372458b42743949acba52df7dc99da (127.0.186.68) added. New cstate: current_term: 1 leader_uuid: "96ae1238e2634cb59f7cd931928f2b6c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3cc42012c014d17992d7442d18a0c18" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 33751 } } peers { permanent_uuid: "b4d48a1d2ca64f65b400acd9b1fb8f1c" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 45677 } } peers { permanent_uuid: "96ae1238e2634cb59f7cd931928f2b6c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36381 } } peers { permanent_uuid: "74372458b42743949acba52df7dc99da" member_type: NON_VOTER last_known_addr { host: "127.0.186.68" port: 32853 } attrs { } } }
W20260812 08:00:10.902598  1209 consensus_peers.cc:597] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c -> Peer 74372458b42743949acba52df7dc99da (127.0.186.68:32853): Couldn't send request to peer 74372458b42743949acba52df7dc99da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0c48bd113d54ca4beadb425c79c109d. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:00:11.338325  1557 ts_tablet_manager.cc:933] T c0c48bd113d54ca4beadb425c79c109d P 74372458b42743949acba52df7dc99da: Initiating tablet copy from peer 96ae1238e2634cb59f7cd931928f2b6c (127.0.186.67:36381)
I20260812 08:00:11.338898  1557 tablet_copy_client.cc:323] T c0c48bd113d54ca4beadb425c79c109d P 74372458b42743949acba52df7dc99da: tablet copy: Beginning tablet copy session from remote peer at address 127.0.186.67:36381
I20260812 08:00:11.343067  1309 tablet_copy_service.cc:140] P 96ae1238e2634cb59f7cd931928f2b6c: Received BeginTabletCopySession request for tablet c0c48bd113d54ca4beadb425c79c109d from peer 74372458b42743949acba52df7dc99da ({username='slave'} at 127.0.186.68:50637)
I20260812 08:00:11.343223  1309 tablet_copy_service.cc:161] P 96ae1238e2634cb59f7cd931928f2b6c: Beginning new tablet copy session on tablet c0c48bd113d54ca4beadb425c79c109d from peer 74372458b42743949acba52df7dc99da at {username='slave'} at 127.0.186.68:50637: session id = 74372458b42743949acba52df7dc99da-c0c48bd113d54ca4beadb425c79c109d
I20260812 08:00:11.344497  1309 tablet_copy_source_session.cc:215] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 08:00:11.347963  1557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0c48bd113d54ca4beadb425c79c109d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:11.353701  1557 tablet_copy_client.cc:806] T c0c48bd113d54ca4beadb425c79c109d P 74372458b42743949acba52df7dc99da: tablet copy: Starting download of 0 data blocks...
I20260812 08:00:11.353909  1557 tablet_copy_client.cc:670] T c0c48bd113d54ca4beadb425c79c109d P 74372458b42743949acba52df7dc99da: tablet copy: Starting download of 1 WAL segments...
I20260812 08:00:11.355747  1557 tablet_copy_client.cc:538] T c0c48bd113d54ca4beadb425c79c109d P 74372458b42743949acba52df7dc99da: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 08:00:11.357688  1557 tablet_bootstrap.cc:492] T c0c48bd113d54ca4beadb425c79c109d P 74372458b42743949acba52df7dc99da: Bootstrap starting.
I20260812 08:00:11.362150  1557 log.cc:826] T c0c48bd113d54ca4beadb425c79c109d P 74372458b42743949acba52df7dc99da: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:11.364410  1557 tablet_bootstrap.cc:492] T c0c48bd113d54ca4beadb425c79c109d P 74372458b42743949acba52df7dc99da: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:00:11.364642  1557 tablet_bootstrap.cc:492] T c0c48bd113d54ca4beadb425c79c109d P 74372458b42743949acba52df7dc99da: Bootstrap complete.
I20260812 08:00:11.364878  1557 ts_tablet_manager.cc:1403] T c0c48bd113d54ca4beadb425c79c109d P 74372458b42743949acba52df7dc99da: Time spent bootstrapping tablet: real 0.007s	user 0.005s	sys 0.000s
I20260812 08:00:11.365166  1500 heartbeater.cc:499] Master 127.0.186.126:44609 was elected leader, sending a full tablet report...
I20260812 08:00:11.367653  1557 raft_consensus.cc:359] T c0c48bd113d54ca4beadb425c79c109d P 74372458b42743949acba52df7dc99da [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3cc42012c014d17992d7442d18a0c18" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 33751 } } peers { permanent_uuid: "b4d48a1d2ca64f65b400acd9b1fb8f1c" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 45677 } } peers { permanent_uuid: "96ae1238e2634cb59f7cd931928f2b6c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36381 } } peers { permanent_uuid: "74372458b42743949acba52df7dc99da" member_type: NON_VOTER last_known_addr { host: "127.0.186.68" port: 32853 } attrs { } }
I20260812 08:00:11.367882  1557 raft_consensus.cc:740] T c0c48bd113d54ca4beadb425c79c109d P 74372458b42743949acba52df7dc99da [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74372458b42743949acba52df7dc99da, State: Initialized, Role: LEARNER
I20260812 08:00:11.368189  1557 consensus_queue.cc:260] T c0c48bd113d54ca4beadb425c79c109d P 74372458b42743949acba52df7dc99da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3cc42012c014d17992d7442d18a0c18" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 33751 } } peers { permanent_uuid: "b4d48a1d2ca64f65b400acd9b1fb8f1c" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 45677 } } peers { permanent_uuid: "96ae1238e2634cb59f7cd931928f2b6c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36381 } } peers { permanent_uuid: "74372458b42743949acba52df7dc99da" member_type: NON_VOTER last_known_addr { host: "127.0.186.68" port: 32853 } attrs { } }
I20260812 08:00:11.368947  1557 ts_tablet_manager.cc:1434] T c0c48bd113d54ca4beadb425c79c109d P 74372458b42743949acba52df7dc99da: Time spent starting tablet: real 0.004s	user 0.006s	sys 0.000s
I20260812 08:00:11.369663  1309 tablet_copy_service.cc:342] P 96ae1238e2634cb59f7cd931928f2b6c: Request end of tablet copy session 74372458b42743949acba52df7dc99da-c0c48bd113d54ca4beadb425c79c109d received from {username='slave'} at 127.0.186.68:50637
I20260812 08:00:11.369760  1309 tablet_copy_service.cc:434] P 96ae1238e2634cb59f7cd931928f2b6c: ending tablet copy session 74372458b42743949acba52df7dc99da-c0c48bd113d54ca4beadb425c79c109d on tablet c0c48bd113d54ca4beadb425c79c109d with peer 74372458b42743949acba52df7dc99da
I20260812 08:00:11.376245   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 874
W20260812 08:00:11.390424  1208 connection.cc:573] client connection to 127.0.186.65:33751 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 08:00:11.390621  1208 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 08:00:11.390709   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 1030
W20260812 08:00:11.393956  1208 consensus_peers.cc:597] T c0c48bd113d54ca4beadb425c79c109d P 96ae1238e2634cb59f7cd931928f2b6c -> Peer e3cc42012c014d17992d7442d18a0c18 (127.0.186.65:33751): Couldn't send request to peer e3cc42012c014d17992d7442d18a0c18. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:33751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 08:00:11.403067   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 1189
I20260812 08:00:11.412832   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 1344
I20260812 08:00:11.421363   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 757
2026-08-12T08:00:11Z chronyd exiting
[       OK ] RaftConsensusNonVoterITest.GetTableAndTabletLocations (2210 ms)
[ RUN      ] RaftConsensusNonVoterITest.ReplicaMatchPolicy
I20260812 08:00:11.449764   745 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:00:11.449813   745 ts_itest-base.cc:116] --------------
I20260812 08:00:11.449823   745 ts_itest-base.cc:117] 4 tablet servers
I20260812 08:00:11.449832   745 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:00:11.449841   745 ts_itest-base.cc:119] --------------
2026-08-12T08:00:11Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:00:11Z Disabled control of system clock
I20260812 08:00:11.458410   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:37737
--webserver_interface=127.0.186.126
--webserver_port=0
--builtin_ntp_servers=127.0.186.84:35115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:37737
--catalog_manager_evict_excess_replicas=false with env {}
W20260812 08:00:11.565805  1573 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:11.566088  1573 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:11.566154  1573 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:11.569584  1573 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:00:11.569654  1573 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:11.569674  1573 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:00:11.569692  1573 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:00:11.573711  1573 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:35115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--catalog_manager_evict_excess_replicas=false
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:37737
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:37737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.1573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:11.574846  1573 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:11.576085  1573 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:11.581506  1583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:11.581511  1580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:11.581506  1581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:11.581801  1573 server_base.cc:1061] running on GCE node
I20260812 08:00:11.582274  1573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:11.583130  1573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:11.584283  1573 hybrid_clock.cc:648] HybridClock initialized: now 1786521611584264 us; error 38 us; skew 500 ppm
I20260812 08:00:11.585976  1573 webserver.cc:533] Webserver started at http://127.0.186.126:38855/ using document root <none> and password file <none>
I20260812 08:00:11.586549  1573 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:11.586618  1573 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:11.586798  1573 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:11.588303  1573 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "b98b350332354370b732146997b8fb88"
format_stamp: "Formatted at 2026-08-12 08:00:11 on dist-test-slave-49h3"
I20260812 08:00:11.588770  1573 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "b98b350332354370b732146997b8fb88"
format_stamp: "Formatted at 2026-08-12 08:00:11 on dist-test-slave-49h3"
I20260812 08:00:11.592041  1573 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.004s
I20260812 08:00:11.594265  1589 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:11.595338  1573 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 08:00:11.595458  1573 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "b98b350332354370b732146997b8fb88"
format_stamp: "Formatted at 2026-08-12 08:00:11 on dist-test-slave-49h3"
I20260812 08:00:11.595551  1573 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:11.612847  1573 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:11.613847  1573 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:11.614073  1573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:11.622459  1649 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:37737 every 8 connection(s)
I20260812 08:00:11.622466  1573 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:37737
I20260812 08:00:11.623658  1573 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:00:11.624109   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 1573
I20260812 08:00:11.624214   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:00:11.626971  1650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:11.632660  1650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b98b350332354370b732146997b8fb88: Bootstrap starting.
I20260812 08:00:11.635275  1650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b98b350332354370b732146997b8fb88: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:11.636034  1650 log.cc:826] T 00000000000000000000000000000000 P b98b350332354370b732146997b8fb88: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:11.638011  1650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b98b350332354370b732146997b8fb88: No bootstrap required, opened a new log
I20260812 08:00:11.640525  1650 raft_consensus.cc:359] T 00000000000000000000000000000000 P b98b350332354370b732146997b8fb88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b98b350332354370b732146997b8fb88" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 37737 } }
I20260812 08:00:11.640734  1650 raft_consensus.cc:385] T 00000000000000000000000000000000 P b98b350332354370b732146997b8fb88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:11.640796  1650 raft_consensus.cc:740] T 00000000000000000000000000000000 P b98b350332354370b732146997b8fb88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b98b350332354370b732146997b8fb88, State: Initialized, Role: FOLLOWER
I20260812 08:00:11.641247  1650 consensus_queue.cc:260] T 00000000000000000000000000000000 P b98b350332354370b732146997b8fb88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b98b350332354370b732146997b8fb88" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 37737 } }
I20260812 08:00:11.641412  1650 raft_consensus.cc:399] T 00000000000000000000000000000000 P b98b350332354370b732146997b8fb88 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:00:11.641508  1650 raft_consensus.cc:493] T 00000000000000000000000000000000 P b98b350332354370b732146997b8fb88 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:00:11.641628  1650 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b98b350332354370b732146997b8fb88 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:11.642473  1650 raft_consensus.cc:515] T 00000000000000000000000000000000 P b98b350332354370b732146997b8fb88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b98b350332354370b732146997b8fb88" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 37737 } }
I20260812 08:00:11.642817  1650 leader_election.cc:304] T 00000000000000000000000000000000 P b98b350332354370b732146997b8fb88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b98b350332354370b732146997b8fb88; no voters: 
I20260812 08:00:11.643121  1650 leader_election.cc:290] T 00000000000000000000000000000000 P b98b350332354370b732146997b8fb88 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:00:11.643231  1657 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b98b350332354370b732146997b8fb88 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:00:11.643429  1657 raft_consensus.cc:697] T 00000000000000000000000000000000 P b98b350332354370b732146997b8fb88 [term 1 LEADER]: Becoming Leader. State: Replica: b98b350332354370b732146997b8fb88, State: Running, Role: LEADER
I20260812 08:00:11.643760  1657 consensus_queue.cc:237] T 00000000000000000000000000000000 P b98b350332354370b732146997b8fb88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b98b350332354370b732146997b8fb88" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 37737 } }
I20260812 08:00:11.644260  1650 sys_catalog.cc:565] T 00000000000000000000000000000000 P b98b350332354370b732146997b8fb88 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:00:11.645308  1658 sys_catalog.cc:455] T 00000000000000000000000000000000 P b98b350332354370b732146997b8fb88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b98b350332354370b732146997b8fb88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b98b350332354370b732146997b8fb88" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 37737 } } }
I20260812 08:00:11.645510  1658 sys_catalog.cc:458] T 00000000000000000000000000000000 P b98b350332354370b732146997b8fb88 [sys.catalog]: This master's current role is: LEADER
I20260812 08:00:11.646067  1659 sys_catalog.cc:455] T 00000000000000000000000000000000 P b98b350332354370b732146997b8fb88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b98b350332354370b732146997b8fb88. Latest consensus state: current_term: 1 leader_uuid: "b98b350332354370b732146997b8fb88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b98b350332354370b732146997b8fb88" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 37737 } } }
I20260812 08:00:11.646166  1659 sys_catalog.cc:458] T 00000000000000000000000000000000 P b98b350332354370b732146997b8fb88 [sys.catalog]: This master's current role is: LEADER
W20260812 08:00:11.649080  1678 catalog_manager.cc:1594] T 00000000000000000000000000000000 P b98b350332354370b732146997b8fb88: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 08:00:11.649153  1678 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 08:00:11.649245  1669 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:00:11.650080  1669 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:00:11.655296  1669 catalog_manager.cc:1383] Generated new cluster ID: 494aabe564b94b2b8fc74b2fb3c8c991
I20260812 08:00:11.655360  1669 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:00:11.671463  1669 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:00:11.672617  1669 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:00:11.690837  1669 catalog_manager.cc:6092] T 00000000000000000000000000000000 P b98b350332354370b732146997b8fb88: Generated new TSK 0
I20260812 08:00:11.691560  1669 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:00:11.703086   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:0
--local_ip_for_outbound_sockets=127.0.186.65
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:37737
--builtin_ntp_servers=127.0.186.84:35115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=50 with env {}
W20260812 08:00:11.809516  1682 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:11.809739  1682 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:11.809787  1682 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:11.813122  1682 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:11.813236  1682 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:00:11.817058  1682 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:35115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=50
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:37737
--never_fsync=true
--heap_profile_path=/tmp/kudu.1682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:11.818195  1682 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:11.819342  1682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:11.825783  1692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:11.825868  1689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:11.825865  1682 server_base.cc:1061] running on GCE node
W20260812 08:00:11.825785  1690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:11.826440  1682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:11.827004  1682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:11.828153  1682 hybrid_clock.cc:648] HybridClock initialized: now 1786521611828143 us; error 31 us; skew 500 ppm
I20260812 08:00:11.829978  1682 webserver.cc:533] Webserver started at http://127.0.186.65:39447/ using document root <none> and password file <none>
I20260812 08:00:11.830760  1682 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:11.830850  1682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:11.831049  1682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:11.832661  1682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "ecd3560ce77943b98efb1476a13e87ab"
format_stamp: "Formatted at 2026-08-12 08:00:11 on dist-test-slave-49h3"
I20260812 08:00:11.833163  1682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "ecd3560ce77943b98efb1476a13e87ab"
format_stamp: "Formatted at 2026-08-12 08:00:11 on dist-test-slave-49h3"
I20260812 08:00:11.836614  1682 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.006s	sys 0.000s
I20260812 08:00:11.838896  1699 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:11.839946  1682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:00:11.840058  1682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "ecd3560ce77943b98efb1476a13e87ab"
format_stamp: "Formatted at 2026-08-12 08:00:11 on dist-test-slave-49h3"
I20260812 08:00:11.840150  1682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:11.863828  1682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:11.864686  1682 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:11.864887  1682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:11.865579  1682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:11.866703  1682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:11.866756  1682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:11.866812  1682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:11.866837  1682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:11.875638  1682 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:39195
I20260812 08:00:11.875705  1846 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:39195 every 8 connection(s)
I20260812 08:00:11.876641  1682 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:00:11.879684   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 1682
I20260812 08:00:11.879792   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:00:11.882113   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.66:0
--local_ip_for_outbound_sockets=127.0.186.66
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:37737
--builtin_ntp_servers=127.0.186.84:35115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=50 with env {}
I20260812 08:00:11.885324  1847 heartbeater.cc:344] Connected to a master server at 127.0.186.126:37737
I20260812 08:00:11.885601  1847 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:11.886154  1847 heartbeater.cc:507] Master 127.0.186.126:37737 requested a full tablet report, sending...
I20260812 08:00:11.887696  1608 ts_manager.cc:194] Registered new tserver with Master: ecd3560ce77943b98efb1476a13e87ab (127.0.186.65:39195)
I20260812 08:00:11.888868  1608 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.65:60407
W20260812 08:00:11.991777  1850 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:11.992003  1850 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:11.992066  1850 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:11.995390  1850 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:11.995519  1850 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.66
I20260812 08:00:11.999384  1850 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:35115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=50
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:37737
--never_fsync=true
--heap_profile_path=/tmp/kudu.1850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:12.000557  1850 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:12.001722  1850 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:12.008005  1856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:12.008068  1860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:12.007993  1857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:12.008271  1850 server_base.cc:1061] running on GCE node
I20260812 08:00:12.008698  1850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:12.009243  1850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:12.010416  1850 hybrid_clock.cc:648] HybridClock initialized: now 1786521612010380 us; error 45 us; skew 500 ppm
I20260812 08:00:12.012154  1850 webserver.cc:533] Webserver started at http://127.0.186.66:34663/ using document root <none> and password file <none>
I20260812 08:00:12.012712  1850 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:12.012771  1850 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:12.012959  1850 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:12.014568  1850 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "accb7c6d259642ad9e94f6f90e0cd2b8"
format_stamp: "Formatted at 2026-08-12 08:00:12 on dist-test-slave-49h3"
I20260812 08:00:12.015065  1850 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "accb7c6d259642ad9e94f6f90e0cd2b8"
format_stamp: "Formatted at 2026-08-12 08:00:12 on dist-test-slave-49h3"
I20260812 08:00:12.018505  1850 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.001s	sys 0.003s
I20260812 08:00:12.020701  1868 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:12.021921  1850 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 08:00:12.022065  1850 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "accb7c6d259642ad9e94f6f90e0cd2b8"
format_stamp: "Formatted at 2026-08-12 08:00:12 on dist-test-slave-49h3"
I20260812 08:00:12.022166  1850 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:12.058634  1850 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:12.059378  1850 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:12.059558  1850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:12.060185  1850 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:12.061247  1850 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:12.061298  1850 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:12.061354  1850 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:12.061381  1850 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:12.070693  1850 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.66:44159
I20260812 08:00:12.070727  2013 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.66:44159 every 8 connection(s)
I20260812 08:00:12.071777  1850 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:00:12.079454   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 1850
I20260812 08:00:12.079581   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:00:12.080533  2014 heartbeater.cc:344] Connected to a master server at 127.0.186.126:37737
I20260812 08:00:12.080806  2014 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:12.081473  2014 heartbeater.cc:507] Master 127.0.186.126:37737 requested a full tablet report, sending...
I20260812 08:00:12.081811   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.67:0
--local_ip_for_outbound_sockets=127.0.186.67
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:37737
--builtin_ntp_servers=127.0.186.84:35115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=50 with env {}
I20260812 08:00:12.082638  1608 ts_manager.cc:194] Registered new tserver with Master: accb7c6d259642ad9e94f6f90e0cd2b8 (127.0.186.66:44159)
I20260812 08:00:12.083191  1608 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.66:56761
W20260812 08:00:12.193329  2017 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:12.193603  2017 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:12.193662  2017 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:12.197005  2017 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:12.197124  2017 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.67
I20260812 08:00:12.201036  2017 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:35115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=50
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:37737
--never_fsync=true
--heap_profile_path=/tmp/kudu.2017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:12.202174  2017 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:12.203308  2017 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:12.209614  2023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:12.209607  2025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:12.209615  2022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:12.209882  2017 server_base.cc:1061] running on GCE node
I20260812 08:00:12.210233  2017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:12.210765  2017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:12.211917  2017 hybrid_clock.cc:648] HybridClock initialized: now 1786521612211901 us; error 28 us; skew 500 ppm
I20260812 08:00:12.213541  2017 webserver.cc:533] Webserver started at http://127.0.186.67:33509/ using document root <none> and password file <none>
I20260812 08:00:12.214072  2017 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:12.214129  2017 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:12.214313  2017 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:12.215981  2017 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "c14da5f7f27e478f9b882d4a4160645b"
format_stamp: "Formatted at 2026-08-12 08:00:12 on dist-test-slave-49h3"
I20260812 08:00:12.216476  2017 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "c14da5f7f27e478f9b882d4a4160645b"
format_stamp: "Formatted at 2026-08-12 08:00:12 on dist-test-slave-49h3"
I20260812 08:00:12.219851  2017 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:00:12.222009  2033 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:12.222997  2017 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:00:12.223109  2017 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "c14da5f7f27e478f9b882d4a4160645b"
format_stamp: "Formatted at 2026-08-12 08:00:12 on dist-test-slave-49h3"
I20260812 08:00:12.223199  2017 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:12.240327  2017 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:12.241037  2017 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:12.241236  2017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:12.241906  2017 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:12.243002  2017 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:12.243052  2017 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:12.243106  2017 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:12.243134  2017 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:12.252661  2017 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.67:40155
I20260812 08:00:12.252686  2172 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.67:40155 every 8 connection(s)
I20260812 08:00:12.253811  2017 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:00:12.258924   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 2017
I20260812 08:00:12.259042   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:00:12.261677   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.68:0
--local_ip_for_outbound_sockets=127.0.186.68
--webserver_interface=127.0.186.68
--webserver_port=0
--tserver_master_addrs=127.0.186.126:37737
--builtin_ntp_servers=127.0.186.84:35115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=50 with env {}
I20260812 08:00:12.263619  2173 heartbeater.cc:344] Connected to a master server at 127.0.186.126:37737
I20260812 08:00:12.263942  2173 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:12.264617  2173 heartbeater.cc:507] Master 127.0.186.126:37737 requested a full tablet report, sending...
I20260812 08:00:12.266126  1608 ts_manager.cc:194] Registered new tserver with Master: c14da5f7f27e478f9b882d4a4160645b (127.0.186.67:40155)
I20260812 08:00:12.266808  1608 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.67:41093
W20260812 08:00:12.371220  2176 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:12.371440  2176 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:12.371482  2176 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:12.374778  2176 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:12.374892  2176 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.68
I20260812 08:00:12.378876  2176 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:35115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=50
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.0.186.68
--webserver_port=0
--tserver_master_addrs=127.0.186.126:37737
--never_fsync=true
--heap_profile_path=/tmp/kudu.2176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.68

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:12.379966  2176 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:12.381131  2176 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:12.387550  2182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:12.387514  2181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:12.387542  2184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:12.387754  2176 server_base.cc:1061] running on GCE node
I20260812 08:00:12.388188  2176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:12.388775  2176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:12.389919  2176 hybrid_clock.cc:648] HybridClock initialized: now 1786521612389901 us; error 33 us; skew 500 ppm
I20260812 08:00:12.391723  2176 webserver.cc:533] Webserver started at http://127.0.186.68:46269/ using document root <none> and password file <none>
I20260812 08:00:12.392308  2176 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:12.392369  2176 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:12.392560  2176 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:12.394210  2176 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "0329bf645bc54443b597923a0c0a7f1f"
format_stamp: "Formatted at 2026-08-12 08:00:12 on dist-test-slave-49h3"
I20260812 08:00:12.394685  2176 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "0329bf645bc54443b597923a0c0a7f1f"
format_stamp: "Formatted at 2026-08-12 08:00:12 on dist-test-slave-49h3"
I20260812 08:00:12.397961  2176 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:00:12.400056  2191 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:12.401072  2176 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:00:12.401185  2176 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "0329bf645bc54443b597923a0c0a7f1f"
format_stamp: "Formatted at 2026-08-12 08:00:12 on dist-test-slave-49h3"
I20260812 08:00:12.401268  2176 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:12.424834  2176 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:12.425585  2176 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:12.425756  2176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:12.426370  2176 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:12.427534  2176 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:12.427584  2176 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:12.427627  2176 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:12.427657  2176 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:12.437525  2176 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.68:40231
I20260812 08:00:12.437536  2319 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.68:40231 every 8 connection(s)
I20260812 08:00:12.438499  2176 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20260812 08:00:12.439262   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 2176
I20260812 08:00:12.439359   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.ReplicaMatchPolicy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20260812 08:00:12.446506  2320 heartbeater.cc:344] Connected to a master server at 127.0.186.126:37737
I20260812 08:00:12.446743  2320 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:12.447222  2320 heartbeater.cc:507] Master 127.0.186.126:37737 requested a full tablet report, sending...
I20260812 08:00:12.448205  1608 ts_manager.cc:194] Registered new tserver with Master: 0329bf645bc54443b597923a0c0a7f1f (127.0.186.68:40231)
I20260812 08:00:12.448736  1608 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.68:41395
I20260812 08:00:12.453554   745 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20260812 08:00:12.461834  1608 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40372:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
I20260812 08:00:12.481745  2245 tablet_service.cc:1511] Processing CreateTablet for tablet fa4852e6427341b08637527f8bc7cc58 (DEFAULT_TABLE table=TestTable [id=77b2deb3f7ff4b939fb78bae55ec5a2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:12.481746  1759 tablet_service.cc:1511] Processing CreateTablet for tablet fa4852e6427341b08637527f8bc7cc58 (DEFAULT_TABLE table=TestTable [id=77b2deb3f7ff4b939fb78bae55ec5a2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:12.481746  2093 tablet_service.cc:1511] Processing CreateTablet for tablet fa4852e6427341b08637527f8bc7cc58 (DEFAULT_TABLE table=TestTable [id=77b2deb3f7ff4b939fb78bae55ec5a2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:12.482726  2245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa4852e6427341b08637527f8bc7cc58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:12.482728  2093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa4852e6427341b08637527f8bc7cc58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:12.482728  1759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa4852e6427341b08637527f8bc7cc58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:12.488792  2341 tablet_bootstrap.cc:492] T fa4852e6427341b08637527f8bc7cc58 P ecd3560ce77943b98efb1476a13e87ab: Bootstrap starting.
I20260812 08:00:12.488855  2340 tablet_bootstrap.cc:492] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f: Bootstrap starting.
I20260812 08:00:12.490830  2340 tablet_bootstrap.cc:654] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:12.490833  2341 tablet_bootstrap.cc:654] T fa4852e6427341b08637527f8bc7cc58 P ecd3560ce77943b98efb1476a13e87ab: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:12.491494  2341 log.cc:826] T fa4852e6427341b08637527f8bc7cc58 P ecd3560ce77943b98efb1476a13e87ab: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:12.491494  2340 log.cc:826] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:12.492074  2342 tablet_bootstrap.cc:492] T fa4852e6427341b08637527f8bc7cc58 P c14da5f7f27e478f9b882d4a4160645b: Bootstrap starting.
I20260812 08:00:12.493018  2341 tablet_bootstrap.cc:492] T fa4852e6427341b08637527f8bc7cc58 P ecd3560ce77943b98efb1476a13e87ab: No bootstrap required, opened a new log
I20260812 08:00:12.493193  2340 tablet_bootstrap.cc:492] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f: No bootstrap required, opened a new log
I20260812 08:00:12.493209  2341 ts_tablet_manager.cc:1403] T fa4852e6427341b08637527f8bc7cc58 P ecd3560ce77943b98efb1476a13e87ab: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.000s
I20260812 08:00:12.493376  2340 ts_tablet_manager.cc:1403] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f: Time spent bootstrapping tablet: real 0.005s	user 0.000s	sys 0.003s
I20260812 08:00:12.494196  2342 tablet_bootstrap.cc:654] T fa4852e6427341b08637527f8bc7cc58 P c14da5f7f27e478f9b882d4a4160645b: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:12.494971  2342 log.cc:826] T fa4852e6427341b08637527f8bc7cc58 P c14da5f7f27e478f9b882d4a4160645b: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:12.495713  2341 raft_consensus.cc:359] T fa4852e6427341b08637527f8bc7cc58 P ecd3560ce77943b98efb1476a13e87ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0329bf645bc54443b597923a0c0a7f1f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40231 } } peers { permanent_uuid: "c14da5f7f27e478f9b882d4a4160645b" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40155 } } peers { permanent_uuid: "ecd3560ce77943b98efb1476a13e87ab" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 39195 } }
I20260812 08:00:12.495934  2341 raft_consensus.cc:385] T fa4852e6427341b08637527f8bc7cc58 P ecd3560ce77943b98efb1476a13e87ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:12.495990  2341 raft_consensus.cc:740] T fa4852e6427341b08637527f8bc7cc58 P ecd3560ce77943b98efb1476a13e87ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecd3560ce77943b98efb1476a13e87ab, State: Initialized, Role: FOLLOWER
I20260812 08:00:12.496402  2341 consensus_queue.cc:260] T fa4852e6427341b08637527f8bc7cc58 P ecd3560ce77943b98efb1476a13e87ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0329bf645bc54443b597923a0c0a7f1f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40231 } } peers { permanent_uuid: "c14da5f7f27e478f9b882d4a4160645b" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40155 } } peers { permanent_uuid: "ecd3560ce77943b98efb1476a13e87ab" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 39195 } }
I20260812 08:00:12.496349  2340 raft_consensus.cc:359] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0329bf645bc54443b597923a0c0a7f1f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40231 } } peers { permanent_uuid: "c14da5f7f27e478f9b882d4a4160645b" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40155 } } peers { permanent_uuid: "ecd3560ce77943b98efb1476a13e87ab" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 39195 } }
I20260812 08:00:12.496579  2340 raft_consensus.cc:385] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:12.496640  2342 tablet_bootstrap.cc:492] T fa4852e6427341b08637527f8bc7cc58 P c14da5f7f27e478f9b882d4a4160645b: No bootstrap required, opened a new log
I20260812 08:00:12.496663  2340 raft_consensus.cc:740] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0329bf645bc54443b597923a0c0a7f1f, State: Initialized, Role: FOLLOWER
I20260812 08:00:12.496825  2342 ts_tablet_manager.cc:1403] T fa4852e6427341b08637527f8bc7cc58 P c14da5f7f27e478f9b882d4a4160645b: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.000s
I20260812 08:00:12.497068  2340 consensus_queue.cc:260] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0329bf645bc54443b597923a0c0a7f1f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40231 } } peers { permanent_uuid: "c14da5f7f27e478f9b882d4a4160645b" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40155 } } peers { permanent_uuid: "ecd3560ce77943b98efb1476a13e87ab" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 39195 } }
I20260812 08:00:12.497189  2341 ts_tablet_manager.cc:1434] T fa4852e6427341b08637527f8bc7cc58 P ecd3560ce77943b98efb1476a13e87ab: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:00:12.497546  1847 heartbeater.cc:499] Master 127.0.186.126:37737 was elected leader, sending a full tablet report...
I20260812 08:00:12.497902  2340 ts_tablet_manager.cc:1434] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.001s
I20260812 08:00:12.498699  2320 heartbeater.cc:499] Master 127.0.186.126:37737 was elected leader, sending a full tablet report...
I20260812 08:00:12.499439  2342 raft_consensus.cc:359] T fa4852e6427341b08637527f8bc7cc58 P c14da5f7f27e478f9b882d4a4160645b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0329bf645bc54443b597923a0c0a7f1f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40231 } } peers { permanent_uuid: "c14da5f7f27e478f9b882d4a4160645b" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40155 } } peers { permanent_uuid: "ecd3560ce77943b98efb1476a13e87ab" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 39195 } }
I20260812 08:00:12.499637  2342 raft_consensus.cc:385] T fa4852e6427341b08637527f8bc7cc58 P c14da5f7f27e478f9b882d4a4160645b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:12.499763  2342 raft_consensus.cc:740] T fa4852e6427341b08637527f8bc7cc58 P c14da5f7f27e478f9b882d4a4160645b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c14da5f7f27e478f9b882d4a4160645b, State: Initialized, Role: FOLLOWER
I20260812 08:00:12.500197  2342 consensus_queue.cc:260] T fa4852e6427341b08637527f8bc7cc58 P c14da5f7f27e478f9b882d4a4160645b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0329bf645bc54443b597923a0c0a7f1f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40231 } } peers { permanent_uuid: "c14da5f7f27e478f9b882d4a4160645b" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40155 } } peers { permanent_uuid: "ecd3560ce77943b98efb1476a13e87ab" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 39195 } }
I20260812 08:00:12.501031  2342 ts_tablet_manager.cc:1434] T fa4852e6427341b08637527f8bc7cc58 P c14da5f7f27e478f9b882d4a4160645b: Time spent starting tablet: real 0.004s	user 0.001s	sys 0.004s
I20260812 08:00:12.501371  2173 heartbeater.cc:499] Master 127.0.186.126:37737 was elected leader, sending a full tablet report...
W20260812 08:00:12.504738  2174 tablet.cc:2406] T fa4852e6427341b08637527f8bc7cc58 P c14da5f7f27e478f9b882d4a4160645b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:00:12.627898  1848 tablet.cc:2406] T fa4852e6427341b08637527f8bc7cc58 P ecd3560ce77943b98efb1476a13e87ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:00:12.654637  2347 raft_consensus.cc:493] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:00:12.654876  2347 raft_consensus.cc:515] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0329bf645bc54443b597923a0c0a7f1f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40231 } } peers { permanent_uuid: "c14da5f7f27e478f9b882d4a4160645b" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40155 } } peers { permanent_uuid: "ecd3560ce77943b98efb1476a13e87ab" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 39195 } }
I20260812 08:00:12.656209  2347 leader_election.cc:290] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c14da5f7f27e478f9b882d4a4160645b (127.0.186.67:40155), ecd3560ce77943b98efb1476a13e87ab (127.0.186.65:39195)
I20260812 08:00:12.660449  1786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4852e6427341b08637527f8bc7cc58" candidate_uuid: "0329bf645bc54443b597923a0c0a7f1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecd3560ce77943b98efb1476a13e87ab" is_pre_election: true
I20260812 08:00:12.660811  1786 raft_consensus.cc:2468] T fa4852e6427341b08637527f8bc7cc58 P ecd3560ce77943b98efb1476a13e87ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0329bf645bc54443b597923a0c0a7f1f in term 0.
I20260812 08:00:12.660857  2115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4852e6427341b08637527f8bc7cc58" candidate_uuid: "0329bf645bc54443b597923a0c0a7f1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c14da5f7f27e478f9b882d4a4160645b" is_pre_election: true
I20260812 08:00:12.661113  2115 raft_consensus.cc:2468] T fa4852e6427341b08637527f8bc7cc58 P c14da5f7f27e478f9b882d4a4160645b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0329bf645bc54443b597923a0c0a7f1f in term 0.
I20260812 08:00:12.661505  2195 leader_election.cc:304] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0329bf645bc54443b597923a0c0a7f1f, ecd3560ce77943b98efb1476a13e87ab; no voters: 
I20260812 08:00:12.661985  2347 raft_consensus.cc:2804] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:00:12.662106  2347 raft_consensus.cc:493] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:00:12.662169  2347 raft_consensus.cc:3060] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:12.665846  2347 raft_consensus.cc:515] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0329bf645bc54443b597923a0c0a7f1f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40231 } } peers { permanent_uuid: "c14da5f7f27e478f9b882d4a4160645b" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40155 } } peers { permanent_uuid: "ecd3560ce77943b98efb1476a13e87ab" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 39195 } }
I20260812 08:00:12.666376  2347 leader_election.cc:290] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f [CANDIDATE]: Term 1 election: Requested vote from peers c14da5f7f27e478f9b882d4a4160645b (127.0.186.67:40155), ecd3560ce77943b98efb1476a13e87ab (127.0.186.65:39195)
I20260812 08:00:12.666893  1786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4852e6427341b08637527f8bc7cc58" candidate_uuid: "0329bf645bc54443b597923a0c0a7f1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecd3560ce77943b98efb1476a13e87ab"
I20260812 08:00:12.666891  2115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4852e6427341b08637527f8bc7cc58" candidate_uuid: "0329bf645bc54443b597923a0c0a7f1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c14da5f7f27e478f9b882d4a4160645b"
I20260812 08:00:12.667045  2115 raft_consensus.cc:3060] T fa4852e6427341b08637527f8bc7cc58 P c14da5f7f27e478f9b882d4a4160645b [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:12.667040  1786 raft_consensus.cc:3060] T fa4852e6427341b08637527f8bc7cc58 P ecd3560ce77943b98efb1476a13e87ab [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:12.668018  1786 raft_consensus.cc:2468] T fa4852e6427341b08637527f8bc7cc58 P ecd3560ce77943b98efb1476a13e87ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0329bf645bc54443b597923a0c0a7f1f in term 1.
I20260812 08:00:12.668390  2115 raft_consensus.cc:2468] T fa4852e6427341b08637527f8bc7cc58 P c14da5f7f27e478f9b882d4a4160645b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0329bf645bc54443b597923a0c0a7f1f in term 1.
I20260812 08:00:12.668465  2195 leader_election.cc:304] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0329bf645bc54443b597923a0c0a7f1f, ecd3560ce77943b98efb1476a13e87ab; no voters: 
I20260812 08:00:12.668675  2347 raft_consensus.cc:2804] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:00:12.668922  2347 raft_consensus.cc:697] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f [term 1 LEADER]: Becoming Leader. State: Replica: 0329bf645bc54443b597923a0c0a7f1f, State: Running, Role: LEADER
I20260812 08:00:12.669261  2347 consensus_queue.cc:237] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0329bf645bc54443b597923a0c0a7f1f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40231 } } peers { permanent_uuid: "c14da5f7f27e478f9b882d4a4160645b" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40155 } } peers { permanent_uuid: "ecd3560ce77943b98efb1476a13e87ab" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 39195 } }
I20260812 08:00:12.673408  1607 catalog_manager.cc:5719] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0329bf645bc54443b597923a0c0a7f1f (127.0.186.68). New cstate: current_term: 1 leader_uuid: "0329bf645bc54443b597923a0c0a7f1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0329bf645bc54443b597923a0c0a7f1f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c14da5f7f27e478f9b882d4a4160645b" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecd3560ce77943b98efb1476a13e87ab" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 39195 } health_report { overall_health: UNKNOWN } } }
W20260812 08:00:12.689599  2321 tablet.cc:2406] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:00:12.712072   745 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20260812 08:00:12.713532   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver ecd3560ce77943b98efb1476a13e87ab to finish bootstrapping
I20260812 08:00:12.719408   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver c14da5f7f27e478f9b882d4a4160645b to finish bootstrapping
I20260812 08:00:12.724472   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 0329bf645bc54443b597923a0c0a7f1f to finish bootstrapping
I20260812 08:00:13.085281  2014 heartbeater.cc:499] Master 127.0.186.126:37737 was elected leader, sending a full tablet report...
I20260812 08:00:13.094906  2347 consensus_queue.cc:1048] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecd3560ce77943b98efb1476a13e87ab" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 39195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:00:13.101907  2350 consensus_queue.cc:1048] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c14da5f7f27e478f9b882d4a4160645b" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:00:13.103065  2265 consensus_queue.cc:237] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0329bf645bc54443b597923a0c0a7f1f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40231 } } peers { permanent_uuid: "c14da5f7f27e478f9b882d4a4160645b" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40155 } } peers { permanent_uuid: "ecd3560ce77943b98efb1476a13e87ab" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 39195 } } peers { permanent_uuid: "accb7c6d259642ad9e94f6f90e0cd2b8" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 44159 } attrs { } }
I20260812 08:00:13.105007  1786 raft_consensus.cc:1275] T fa4852e6427341b08637527f8bc7cc58 P ecd3560ce77943b98efb1476a13e87ab [term 1 FOLLOWER]: Refusing update from remote peer 0329bf645bc54443b597923a0c0a7f1f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:00:13.105826  2350 consensus_queue.cc:1048] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecd3560ce77943b98efb1476a13e87ab" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 39195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:00:13.108134  2114 raft_consensus.cc:1275] T fa4852e6427341b08637527f8bc7cc58 P c14da5f7f27e478f9b882d4a4160645b [term 1 FOLLOWER]: Refusing update from remote peer 0329bf645bc54443b597923a0c0a7f1f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:00:13.107879  2350 raft_consensus.cc:2955] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER accb7c6d259642ad9e94f6f90e0cd2b8 (127.0.186.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0329bf645bc54443b597923a0c0a7f1f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40231 } } peers { permanent_uuid: "c14da5f7f27e478f9b882d4a4160645b" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40155 } } peers { permanent_uuid: "ecd3560ce77943b98efb1476a13e87ab" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 39195 } } peers { permanent_uuid: "accb7c6d259642ad9e94f6f90e0cd2b8" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 44159 } attrs { } } }
I20260812 08:00:13.108904  2347 consensus_queue.cc:1048] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c14da5f7f27e478f9b882d4a4160645b" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:00:13.110359  2114 raft_consensus.cc:2955] T fa4852e6427341b08637527f8bc7cc58 P c14da5f7f27e478f9b882d4a4160645b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER accb7c6d259642ad9e94f6f90e0cd2b8 (127.0.186.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0329bf645bc54443b597923a0c0a7f1f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40231 } } peers { permanent_uuid: "c14da5f7f27e478f9b882d4a4160645b" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40155 } } peers { permanent_uuid: "ecd3560ce77943b98efb1476a13e87ab" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 39195 } } peers { permanent_uuid: "accb7c6d259642ad9e94f6f90e0cd2b8" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 44159 } attrs { } } }
I20260812 08:00:13.110679  1607 catalog_manager.cc:5719] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f reported cstate change: config changed from index -1 to 2, NON_VOTER accb7c6d259642ad9e94f6f90e0cd2b8 (127.0.186.66) added. New cstate: current_term: 1 leader_uuid: "0329bf645bc54443b597923a0c0a7f1f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0329bf645bc54443b597923a0c0a7f1f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c14da5f7f27e478f9b882d4a4160645b" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecd3560ce77943b98efb1476a13e87ab" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 39195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "accb7c6d259642ad9e94f6f90e0cd2b8" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 44159 } attrs { } health_report { overall_health: UNKNOWN } } }
I20260812 08:00:13.111490  1786 raft_consensus.cc:2955] T fa4852e6427341b08637527f8bc7cc58 P ecd3560ce77943b98efb1476a13e87ab [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER accb7c6d259642ad9e94f6f90e0cd2b8 (127.0.186.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0329bf645bc54443b597923a0c0a7f1f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40231 } } peers { permanent_uuid: "c14da5f7f27e478f9b882d4a4160645b" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40155 } } peers { permanent_uuid: "ecd3560ce77943b98efb1476a13e87ab" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 39195 } } peers { permanent_uuid: "accb7c6d259642ad9e94f6f90e0cd2b8" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 44159 } attrs { } } }
W20260812 08:00:13.112794  2194 consensus_peers.cc:597] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f -> Peer accb7c6d259642ad9e94f6f90e0cd2b8 (127.0.186.66:44159): Couldn't send request to peer accb7c6d259642ad9e94f6f90e0cd2b8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fa4852e6427341b08637527f8bc7cc58. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:00:13.561685  2363 ts_tablet_manager.cc:933] T fa4852e6427341b08637527f8bc7cc58 P accb7c6d259642ad9e94f6f90e0cd2b8: Initiating tablet copy from peer 0329bf645bc54443b597923a0c0a7f1f (127.0.186.68:40231)
I20260812 08:00:13.562301  2363 tablet_copy_client.cc:323] T fa4852e6427341b08637527f8bc7cc58 P accb7c6d259642ad9e94f6f90e0cd2b8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.186.68:40231
I20260812 08:00:13.566440  2291 tablet_copy_service.cc:140] P 0329bf645bc54443b597923a0c0a7f1f: Received BeginTabletCopySession request for tablet fa4852e6427341b08637527f8bc7cc58 from peer accb7c6d259642ad9e94f6f90e0cd2b8 ({username='slave'} at 127.0.186.66:59713)
I20260812 08:00:13.566601  2291 tablet_copy_service.cc:161] P 0329bf645bc54443b597923a0c0a7f1f: Beginning new tablet copy session on tablet fa4852e6427341b08637527f8bc7cc58 from peer accb7c6d259642ad9e94f6f90e0cd2b8 at {username='slave'} at 127.0.186.66:59713: session id = accb7c6d259642ad9e94f6f90e0cd2b8-fa4852e6427341b08637527f8bc7cc58
I20260812 08:00:13.567842  2291 tablet_copy_source_session.cc:215] T fa4852e6427341b08637527f8bc7cc58 P 0329bf645bc54443b597923a0c0a7f1f: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 08:00:13.571339  2363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa4852e6427341b08637527f8bc7cc58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:13.576309  2363 tablet_copy_client.cc:806] T fa4852e6427341b08637527f8bc7cc58 P accb7c6d259642ad9e94f6f90e0cd2b8: tablet copy: Starting download of 0 data blocks...
I20260812 08:00:13.576522  2363 tablet_copy_client.cc:670] T fa4852e6427341b08637527f8bc7cc58 P accb7c6d259642ad9e94f6f90e0cd2b8: tablet copy: Starting download of 1 WAL segments...
I20260812 08:00:13.578279  2363 tablet_copy_client.cc:538] T fa4852e6427341b08637527f8bc7cc58 P accb7c6d259642ad9e94f6f90e0cd2b8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 08:00:13.579887  2363 tablet_bootstrap.cc:492] T fa4852e6427341b08637527f8bc7cc58 P accb7c6d259642ad9e94f6f90e0cd2b8: Bootstrap starting.
I20260812 08:00:13.584226  2363 log.cc:826] T fa4852e6427341b08637527f8bc7cc58 P accb7c6d259642ad9e94f6f90e0cd2b8: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:13.586764  2363 tablet_bootstrap.cc:492] T fa4852e6427341b08637527f8bc7cc58 P accb7c6d259642ad9e94f6f90e0cd2b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:00:13.587030  2363 tablet_bootstrap.cc:492] T fa4852e6427341b08637527f8bc7cc58 P accb7c6d259642ad9e94f6f90e0cd2b8: Bootstrap complete.
I20260812 08:00:13.587342  2363 ts_tablet_manager.cc:1403] T fa4852e6427341b08637527f8bc7cc58 P accb7c6d259642ad9e94f6f90e0cd2b8: Time spent bootstrapping tablet: real 0.008s	user 0.007s	sys 0.000s
I20260812 08:00:13.590072  2363 raft_consensus.cc:359] T fa4852e6427341b08637527f8bc7cc58 P accb7c6d259642ad9e94f6f90e0cd2b8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0329bf645bc54443b597923a0c0a7f1f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40231 } } peers { permanent_uuid: "c14da5f7f27e478f9b882d4a4160645b" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40155 } } peers { permanent_uuid: "ecd3560ce77943b98efb1476a13e87ab" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 39195 } } peers { permanent_uuid: "accb7c6d259642ad9e94f6f90e0cd2b8" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 44159 } attrs { } }
I20260812 08:00:13.590304  2363 raft_consensus.cc:740] T fa4852e6427341b08637527f8bc7cc58 P accb7c6d259642ad9e94f6f90e0cd2b8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: accb7c6d259642ad9e94f6f90e0cd2b8, State: Initialized, Role: LEARNER
I20260812 08:00:13.590627  2363 consensus_queue.cc:260] T fa4852e6427341b08637527f8bc7cc58 P accb7c6d259642ad9e94f6f90e0cd2b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0329bf645bc54443b597923a0c0a7f1f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40231 } } peers { permanent_uuid: "c14da5f7f27e478f9b882d4a4160645b" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40155 } } peers { permanent_uuid: "ecd3560ce77943b98efb1476a13e87ab" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 39195 } } peers { permanent_uuid: "accb7c6d259642ad9e94f6f90e0cd2b8" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 44159 } attrs { } }
I20260812 08:00:13.591495  2363 ts_tablet_manager.cc:1434] T fa4852e6427341b08637527f8bc7cc58 P accb7c6d259642ad9e94f6f90e0cd2b8: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:00:13.592402  2291 tablet_copy_service.cc:342] P 0329bf645bc54443b597923a0c0a7f1f: Request end of tablet copy session accb7c6d259642ad9e94f6f90e0cd2b8-fa4852e6427341b08637527f8bc7cc58 received from {username='slave'} at 127.0.186.66:59713
I20260812 08:00:13.592518  2291 tablet_copy_service.cc:434] P 0329bf645bc54443b597923a0c0a7f1f: ending tablet copy session accb7c6d259642ad9e94f6f90e0cd2b8-fa4852e6427341b08637527f8bc7cc58 on tablet fa4852e6427341b08637527f8bc7cc58 with peer accb7c6d259642ad9e94f6f90e0cd2b8
I20260812 08:00:13.618181   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 1682
I20260812 08:00:13.629019   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 1850
I20260812 08:00:13.638360   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 2017
I20260812 08:00:13.649706   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 2176
I20260812 08:00:13.660050   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 1573
2026-08-12T08:00:13Z chronyd exiting
[       OK ] RaftConsensusNonVoterITest.ReplicaMatchPolicy (2238 ms)
[ RUN      ] RaftConsensusNonVoterITest.AddNonVoterReplica
I20260812 08:00:13.688732   745 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:00:13.688786   745 ts_itest-base.cc:116] --------------
I20260812 08:00:13.688802   745 ts_itest-base.cc:117] 3 tablet servers
I20260812 08:00:13.688815   745 ts_itest-base.cc:118] 2 replicas per TS
I20260812 08:00:13.688828   745 ts_itest-base.cc:119] --------------
2026-08-12T08:00:13Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:00:13Z Disabled control of system clock
I20260812 08:00:13.698318   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:41067
--webserver_interface=127.0.186.126
--webserver_port=0
--builtin_ntp_servers=127.0.186.84:40993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:41067
--allow_unsafe_replication_factor=true
--catalog_manager_evict_excess_replicas=false with env {}
W20260812 08:00:13.805920  2384 flags.cc:432] Enabled unsafe flag: --allow_unsafe_replication_factor=true
W20260812 08:00:13.806182  2384 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:13.806221  2384 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:13.806279  2384 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:13.809628  2384 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:00:13.809695  2384 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:13.809721  2384 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:00:13.809743  2384 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:00:13.813640  2384 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:40993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--allow_unsafe_replication_factor=true
--catalog_manager_evict_excess_replicas=false
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:41067
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:41067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.2384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:13.814774  2384 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:13.815960  2384 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:13.821362  2392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:13.821367  2391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:13.821429  2396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:13.821732  2384 server_base.cc:1061] running on GCE node
I20260812 08:00:13.822309  2384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:13.823252  2384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:13.824409  2384 hybrid_clock.cc:648] HybridClock initialized: now 1786521613824397 us; error 30 us; skew 500 ppm
I20260812 08:00:13.826385  2384 webserver.cc:533] Webserver started at http://127.0.186.126:40099/ using document root <none> and password file <none>
I20260812 08:00:13.827025  2384 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:13.827092  2384 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:13.827293  2384 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:13.828977  2384 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "cdca8156c1154498b86604448ec6e27b"
format_stamp: "Formatted at 2026-08-12 08:00:13 on dist-test-slave-49h3"
I20260812 08:00:13.829609  2384 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "cdca8156c1154498b86604448ec6e27b"
format_stamp: "Formatted at 2026-08-12 08:00:13 on dist-test-slave-49h3"
I20260812 08:00:13.833199  2384 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:00:13.835676  2404 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:13.836776  2384 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:00:13.836915  2384 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "cdca8156c1154498b86604448ec6e27b"
format_stamp: "Formatted at 2026-08-12 08:00:13 on dist-test-slave-49h3"
I20260812 08:00:13.837016  2384 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:13.868225  2384 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:13.869055  2384 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:13.869256  2384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:13.877667  2464 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:41067 every 8 connection(s)
I20260812 08:00:13.877686  2384 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:41067
I20260812 08:00:13.878919  2384 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:00:13.882282  2466 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:13.884666   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 2384
I20260812 08:00:13.884797   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:00:13.887969  2466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdca8156c1154498b86604448ec6e27b: Bootstrap starting.
I20260812 08:00:13.890316  2466 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cdca8156c1154498b86604448ec6e27b: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:13.891130  2466 log.cc:826] T 00000000000000000000000000000000 P cdca8156c1154498b86604448ec6e27b: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:13.892978  2466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdca8156c1154498b86604448ec6e27b: No bootstrap required, opened a new log
I20260812 08:00:13.895920  2466 raft_consensus.cc:359] T 00000000000000000000000000000000 P cdca8156c1154498b86604448ec6e27b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdca8156c1154498b86604448ec6e27b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41067 } }
I20260812 08:00:13.896149  2466 raft_consensus.cc:385] T 00000000000000000000000000000000 P cdca8156c1154498b86604448ec6e27b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:13.896226  2466 raft_consensus.cc:740] T 00000000000000000000000000000000 P cdca8156c1154498b86604448ec6e27b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdca8156c1154498b86604448ec6e27b, State: Initialized, Role: FOLLOWER
I20260812 08:00:13.896624  2466 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdca8156c1154498b86604448ec6e27b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdca8156c1154498b86604448ec6e27b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41067 } }
I20260812 08:00:13.896747  2466 raft_consensus.cc:399] T 00000000000000000000000000000000 P cdca8156c1154498b86604448ec6e27b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:00:13.896809  2466 raft_consensus.cc:493] T 00000000000000000000000000000000 P cdca8156c1154498b86604448ec6e27b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:00:13.896914  2466 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cdca8156c1154498b86604448ec6e27b [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:13.897724  2466 raft_consensus.cc:515] T 00000000000000000000000000000000 P cdca8156c1154498b86604448ec6e27b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdca8156c1154498b86604448ec6e27b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41067 } }
I20260812 08:00:13.898048  2466 leader_election.cc:304] T 00000000000000000000000000000000 P cdca8156c1154498b86604448ec6e27b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cdca8156c1154498b86604448ec6e27b; no voters: 
I20260812 08:00:13.898322  2466 leader_election.cc:290] T 00000000000000000000000000000000 P cdca8156c1154498b86604448ec6e27b [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:00:13.898461  2471 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cdca8156c1154498b86604448ec6e27b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:00:13.898654  2471 raft_consensus.cc:697] T 00000000000000000000000000000000 P cdca8156c1154498b86604448ec6e27b [term 1 LEADER]: Becoming Leader. State: Replica: cdca8156c1154498b86604448ec6e27b, State: Running, Role: LEADER
I20260812 08:00:13.898952  2471 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdca8156c1154498b86604448ec6e27b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdca8156c1154498b86604448ec6e27b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41067 } }
I20260812 08:00:13.899416  2466 sys_catalog.cc:565] T 00000000000000000000000000000000 P cdca8156c1154498b86604448ec6e27b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:00:13.900517  2474 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdca8156c1154498b86604448ec6e27b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdca8156c1154498b86604448ec6e27b. Latest consensus state: current_term: 1 leader_uuid: "cdca8156c1154498b86604448ec6e27b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdca8156c1154498b86604448ec6e27b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41067 } } }
I20260812 08:00:13.900655  2474 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdca8156c1154498b86604448ec6e27b [sys.catalog]: This master's current role is: LEADER
I20260812 08:00:13.901067  2482 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:00:13.901569  2472 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdca8156c1154498b86604448ec6e27b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cdca8156c1154498b86604448ec6e27b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdca8156c1154498b86604448ec6e27b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41067 } } }
I20260812 08:00:13.901672  2472 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdca8156c1154498b86604448ec6e27b [sys.catalog]: This master's current role is: LEADER
I20260812 08:00:13.904649  2482 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:00:13.910480  2482 catalog_manager.cc:1383] Generated new cluster ID: 5aaa2b268eff4badb22c96c36f1485ad
I20260812 08:00:13.910604  2482 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:00:13.924862  2482 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:00:13.925925  2482 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:00:13.938500  2482 catalog_manager.cc:6092] T 00000000000000000000000000000000 P cdca8156c1154498b86604448ec6e27b: Generated new TSK 0
I20260812 08:00:13.939432  2482 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:00:13.951383   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:0
--local_ip_for_outbound_sockets=127.0.186.65
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:41067
--builtin_ntp_servers=127.0.186.84:40993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tablet_copy_download_file_inject_latency_ms=1000 with env {}
W20260812 08:00:14.065847  2492 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:14.066094  2492 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:14.066170  2492 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:14.069568  2492 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:14.069700  2492 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:00:14.073570  2492 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:40993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1000
--tserver_master_addrs=127.0.186.126:41067
--never_fsync=true
--heap_profile_path=/tmp/kudu.2492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:14.074725  2492 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:14.075891  2492 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:14.083248  2497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:14.083248  2501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:14.083248  2498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:14.083604  2492 server_base.cc:1061] running on GCE node
I20260812 08:00:14.084074  2492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:14.084813  2492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:14.085994  2492 hybrid_clock.cc:648] HybridClock initialized: now 1786521614085974 us; error 37 us; skew 500 ppm
I20260812 08:00:14.088209  2492 webserver.cc:533] Webserver started at http://127.0.186.65:43639/ using document root <none> and password file <none>
I20260812 08:00:14.088927  2492 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:14.088991  2492 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:14.089215  2492 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:14.091014  2492 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "08ddd6b135c8405bb15c2fe306b04cee"
format_stamp: "Formatted at 2026-08-12 08:00:14 on dist-test-slave-49h3"
I20260812 08:00:14.091511  2492 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "08ddd6b135c8405bb15c2fe306b04cee"
format_stamp: "Formatted at 2026-08-12 08:00:14 on dist-test-slave-49h3"
I20260812 08:00:14.095485  2492 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:00:14.097908  2508 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:14.098991  2492 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:00:14.099102  2492 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "08ddd6b135c8405bb15c2fe306b04cee"
format_stamp: "Formatted at 2026-08-12 08:00:14 on dist-test-slave-49h3"
I20260812 08:00:14.099211  2492 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:14.115769  2492 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:14.116540  2492 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:14.116734  2492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:14.117386  2492 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:14.118541  2492 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:14.118594  2492 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:14.118639  2492 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:14.118669  2492 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:14.128193  2492 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:40081
I20260812 08:00:14.128216  2638 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:40081 every 8 connection(s)
I20260812 08:00:14.129261  2492 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:00:14.137995  2639 heartbeater.cc:344] Connected to a master server at 127.0.186.126:41067
I20260812 08:00:14.138271  2639 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:14.138832  2639 heartbeater.cc:507] Master 127.0.186.126:41067 requested a full tablet report, sending...
I20260812 08:00:14.139179   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 2492
I20260812 08:00:14.139284   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:00:14.140566  2422 ts_manager.cc:194] Registered new tserver with Master: 08ddd6b135c8405bb15c2fe306b04cee (127.0.186.65:40081)
I20260812 08:00:14.141724   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.66:0
--local_ip_for_outbound_sockets=127.0.186.66
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:41067
--builtin_ntp_servers=127.0.186.84:40993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tablet_copy_download_file_inject_latency_ms=1000 with env {}
I20260812 08:00:14.142138  2422 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.65:45615
W20260812 08:00:14.251327  2642 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:14.251551  2642 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:14.251596  2642 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:14.254911  2642 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:14.255028  2642 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.66
I20260812 08:00:14.258953  2642 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:40993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.186.66
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1000
--tserver_master_addrs=127.0.186.126:41067
--never_fsync=true
--heap_profile_path=/tmp/kudu.2642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:14.260098  2642 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:14.261345  2642 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:14.268152  2649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:14.268155  2652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:14.268244  2650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:14.268335  2642 server_base.cc:1061] running on GCE node
I20260812 08:00:14.268841  2642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:14.269418  2642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:14.270589  2642 hybrid_clock.cc:648] HybridClock initialized: now 1786521614270577 us; error 30 us; skew 500 ppm
I20260812 08:00:14.272377  2642 webserver.cc:533] Webserver started at http://127.0.186.66:35713/ using document root <none> and password file <none>
I20260812 08:00:14.272979  2642 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:14.273036  2642 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:14.273236  2642 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:14.274873  2642 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "021778186abb464b913200770a437fc5"
format_stamp: "Formatted at 2026-08-12 08:00:14 on dist-test-slave-49h3"
I20260812 08:00:14.275331  2642 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "021778186abb464b913200770a437fc5"
format_stamp: "Formatted at 2026-08-12 08:00:14 on dist-test-slave-49h3"
I20260812 08:00:14.278896  2642 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.001s
I20260812 08:00:14.281226  2658 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:14.282442  2642 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 08:00:14.282573  2642 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "021778186abb464b913200770a437fc5"
format_stamp: "Formatted at 2026-08-12 08:00:14 on dist-test-slave-49h3"
I20260812 08:00:14.282691  2642 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:14.300952  2642 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:14.301890  2642 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:14.302095  2642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:14.302734  2642 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:14.303896  2642 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:14.303947  2642 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:14.303988  2642 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:14.304016  2642 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:14.312844  2642 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.66:43691
I20260812 08:00:14.312902  2793 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.66:43691 every 8 connection(s)
I20260812 08:00:14.313855  2642 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:00:14.318737   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 2642
I20260812 08:00:14.318862   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:00:14.321089   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.67:0
--local_ip_for_outbound_sockets=127.0.186.67
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:41067
--builtin_ntp_servers=127.0.186.84:40993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tablet_copy_download_file_inject_latency_ms=1000 with env {}
I20260812 08:00:14.322211  2794 heartbeater.cc:344] Connected to a master server at 127.0.186.126:41067
I20260812 08:00:14.322479  2794 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:14.323096  2794 heartbeater.cc:507] Master 127.0.186.126:41067 requested a full tablet report, sending...
I20260812 08:00:14.324234  2422 ts_manager.cc:194] Registered new tserver with Master: 021778186abb464b913200770a437fc5 (127.0.186.66:43691)
I20260812 08:00:14.324781  2422 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.66:37261
W20260812 08:00:14.430209  2798 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:14.430411  2798 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:14.430464  2798 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:14.433729  2798 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:14.433848  2798 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.67
I20260812 08:00:14.437723  2798 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:40993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.186.67
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1000
--tserver_master_addrs=127.0.186.126:41067
--never_fsync=true
--heap_profile_path=/tmp/kudu.2798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:14.438988  2798 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:14.440217  2798 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:14.446497  2803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:14.446537  2807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:14.446552  2805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:14.446823  2798 server_base.cc:1061] running on GCE node
I20260812 08:00:14.447207  2798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:14.447726  2798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:14.448872  2798 hybrid_clock.cc:648] HybridClock initialized: now 1786521614448857 us; error 32 us; skew 500 ppm
I20260812 08:00:14.450567  2798 webserver.cc:533] Webserver started at http://127.0.186.67:43727/ using document root <none> and password file <none>
I20260812 08:00:14.451126  2798 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:14.451185  2798 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:14.451365  2798 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:14.452920  2798 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259"
format_stamp: "Formatted at 2026-08-12 08:00:14 on dist-test-slave-49h3"
I20260812 08:00:14.453411  2798 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259"
format_stamp: "Formatted at 2026-08-12 08:00:14 on dist-test-slave-49h3"
I20260812 08:00:14.456835  2798 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.001s
I20260812 08:00:14.459240  2815 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:14.460289  2798 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.001s
I20260812 08:00:14.460404  2798 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259"
format_stamp: "Formatted at 2026-08-12 08:00:14 on dist-test-slave-49h3"
I20260812 08:00:14.460498  2798 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:14.478909  2798 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:14.479662  2798 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:14.479852  2798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:14.480526  2798 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:14.481738  2798 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:14.481804  2798 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:14.481866  2798 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:14.481896  2798 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:14.491019  2798 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.67:38951
I20260812 08:00:14.491060  2943 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.67:38951 every 8 connection(s)
I20260812 08:00:14.492075  2798 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:00:14.498131   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 2798
I20260812 08:00:14.498260   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:00:14.500366  2944 heartbeater.cc:344] Connected to a master server at 127.0.186.126:41067
I20260812 08:00:14.500639  2944 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:14.501217  2944 heartbeater.cc:507] Master 127.0.186.126:41067 requested a full tablet report, sending...
I20260812 08:00:14.502259  2422 ts_manager.cc:194] Registered new tserver with Master: c1b9ac9b6bc4426b97bdb7ce6a5b8259 (127.0.186.67:38951)
I20260812 08:00:14.502846  2422 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.67:54547
I20260812 08:00:14.513334   745 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:00:14.522017  2422 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46296:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
I20260812 08:00:14.540431  2561 tablet_service.cc:1511] Processing CreateTablet for tablet 97525c8aad564d008d3a0f6654fc5c4b (DEFAULT_TABLE table=TestTable [id=e24f335a7e2e41a4b4e949c5a82bbbff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:14.540431  2867 tablet_service.cc:1511] Processing CreateTablet for tablet 97525c8aad564d008d3a0f6654fc5c4b (DEFAULT_TABLE table=TestTable [id=e24f335a7e2e41a4b4e949c5a82bbbff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:14.541651  2867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97525c8aad564d008d3a0f6654fc5c4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:14.541651  2561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97525c8aad564d008d3a0f6654fc5c4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:14.547921  2961 tablet_bootstrap.cc:492] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259: Bootstrap starting.
I20260812 08:00:14.548591  2962 tablet_bootstrap.cc:492] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee: Bootstrap starting.
I20260812 08:00:14.550977  2962 tablet_bootstrap.cc:654] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:14.550977  2961 tablet_bootstrap.cc:654] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:14.551913  2962 log.cc:826] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:14.551913  2961 log.cc:826] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:14.553705  2961 tablet_bootstrap.cc:492] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259: No bootstrap required, opened a new log
I20260812 08:00:14.553712  2962 tablet_bootstrap.cc:492] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee: No bootstrap required, opened a new log
I20260812 08:00:14.553900  2961 ts_tablet_manager.cc:1403] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259: Time spent bootstrapping tablet: real 0.006s	user 0.001s	sys 0.004s
I20260812 08:00:14.553903  2962 ts_tablet_manager.cc:1403] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee: Time spent bootstrapping tablet: real 0.006s	user 0.005s	sys 0.000s
I20260812 08:00:14.556496  2962 raft_consensus.cc:359] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ddd6b135c8405bb15c2fe306b04cee" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40081 } } peers { permanent_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 38951 } }
I20260812 08:00:14.556504  2961 raft_consensus.cc:359] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ddd6b135c8405bb15c2fe306b04cee" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40081 } } peers { permanent_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 38951 } }
I20260812 08:00:14.556692  2961 raft_consensus.cc:385] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:14.556696  2962 raft_consensus.cc:385] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:14.556726  2961 raft_consensus.cc:740] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b9ac9b6bc4426b97bdb7ce6a5b8259, State: Initialized, Role: FOLLOWER
I20260812 08:00:14.556757  2962 raft_consensus.cc:740] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08ddd6b135c8405bb15c2fe306b04cee, State: Initialized, Role: FOLLOWER
I20260812 08:00:14.557152  2962 consensus_queue.cc:260] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ddd6b135c8405bb15c2fe306b04cee" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40081 } } peers { permanent_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 38951 } }
I20260812 08:00:14.557154  2961 consensus_queue.cc:260] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ddd6b135c8405bb15c2fe306b04cee" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40081 } } peers { permanent_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 38951 } }
I20260812 08:00:14.557881  2944 heartbeater.cc:499] Master 127.0.186.126:41067 was elected leader, sending a full tablet report...
I20260812 08:00:14.558041  2962 ts_tablet_manager.cc:1434] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260812 08:00:14.558046  2961 ts_tablet_manager.cc:1434] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260812 08:00:14.558135  2639 heartbeater.cc:499] Master 127.0.186.126:41067 was elected leader, sending a full tablet report...
I20260812 08:00:14.558470  2966 raft_consensus.cc:493] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:00:14.558616  2966 raft_consensus.cc:515] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ddd6b135c8405bb15c2fe306b04cee" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40081 } } peers { permanent_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 38951 } }
I20260812 08:00:14.559568  2966 leader_election.cc:290] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08ddd6b135c8405bb15c2fe306b04cee (127.0.186.65:40081)
I20260812 08:00:14.563956  2584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97525c8aad564d008d3a0f6654fc5c4b" candidate_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08ddd6b135c8405bb15c2fe306b04cee" is_pre_election: true
I20260812 08:00:14.564199  2584 raft_consensus.cc:2468] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b9ac9b6bc4426b97bdb7ce6a5b8259 in term 0.
I20260812 08:00:14.565160  2816 leader_election.cc:304] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 08ddd6b135c8405bb15c2fe306b04cee, c1b9ac9b6bc4426b97bdb7ce6a5b8259; no voters: 
I20260812 08:00:14.565382  2966 raft_consensus.cc:2804] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:00:14.565470  2966 raft_consensus.cc:493] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:00:14.565526  2966 raft_consensus.cc:3060] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:14.566372  2966 raft_consensus.cc:515] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ddd6b135c8405bb15c2fe306b04cee" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40081 } } peers { permanent_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 38951 } }
I20260812 08:00:14.566653  2966 leader_election.cc:290] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [CANDIDATE]: Term 1 election: Requested vote from peers 08ddd6b135c8405bb15c2fe306b04cee (127.0.186.65:40081)
I20260812 08:00:14.566987  2584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97525c8aad564d008d3a0f6654fc5c4b" candidate_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08ddd6b135c8405bb15c2fe306b04cee"
I20260812 08:00:14.567092  2584 raft_consensus.cc:3060] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:14.567890  2584 raft_consensus.cc:2468] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b9ac9b6bc4426b97bdb7ce6a5b8259 in term 1.
I20260812 08:00:14.568162  2816 leader_election.cc:304] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 08ddd6b135c8405bb15c2fe306b04cee, c1b9ac9b6bc4426b97bdb7ce6a5b8259; no voters: 
I20260812 08:00:14.568329  2966 raft_consensus.cc:2804] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:00:14.568535  2966 raft_consensus.cc:697] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [term 1 LEADER]: Becoming Leader. State: Replica: c1b9ac9b6bc4426b97bdb7ce6a5b8259, State: Running, Role: LEADER
I20260812 08:00:14.568806  2966 consensus_queue.cc:237] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ddd6b135c8405bb15c2fe306b04cee" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40081 } } peers { permanent_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 38951 } }
I20260812 08:00:14.571700  2421 catalog_manager.cc:5719] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1b9ac9b6bc4426b97bdb7ce6a5b8259 (127.0.186.67). New cstate: current_term: 1 leader_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ddd6b135c8405bb15c2fe306b04cee" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 38951 } health_report { overall_health: HEALTHY } } }
I20260812 08:00:14.584725   745 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:00:14.586290   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 08ddd6b135c8405bb15c2fe306b04cee to finish bootstrapping
I20260812 08:00:14.591732   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver c1b9ac9b6bc4426b97bdb7ce6a5b8259 to finish bootstrapping
I20260812 08:00:14.605798   745 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
W20260812 08:00:14.630527  2640 tablet.cc:2406] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:00:14.638105  2584 raft_consensus.cc:1275] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee [term 1 FOLLOWER]: Refusing update from remote peer c1b9ac9b6bc4426b97bdb7ce6a5b8259: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:00:14.638860  2966 consensus_queue.cc:1048] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08ddd6b135c8405bb15c2fe306b04cee" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:00:14.647632  2983 mvcc.cc:204] Tried to move back new op lower bound from 7317592533557780480 to 7317592533276073984. Current Snapshot: MvccSnapshot[applied={T|T < 7317592533550059520}]
I20260812 08:00:14.646077  2982 mvcc.cc:204] Tried to move back new op lower bound from 7317592533557780480 to 7317592533276073984. Current Snapshot: MvccSnapshot[applied={T|T < 7317592533550059520}]
I20260812 08:00:14.680197  2889 consensus_queue.cc:237] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "08ddd6b135c8405bb15c2fe306b04cee" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40081 } } peers { permanent_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 38951 } } peers { permanent_uuid: "021778186abb464b913200770a437fc5" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 43691 } attrs { } }
I20260812 08:00:14.681903  2584 raft_consensus.cc:1275] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee [term 1 FOLLOWER]: Refusing update from remote peer c1b9ac9b6bc4426b97bdb7ce6a5b8259: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260812 08:00:14.682399  2966 consensus_queue.cc:1048] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08ddd6b135c8405bb15c2fe306b04cee" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260812 08:00:14.684281  2967 raft_consensus.cc:2955] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index -1 to 10, NON_VOTER 021778186abb464b913200770a437fc5 (127.0.186.66) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "08ddd6b135c8405bb15c2fe306b04cee" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40081 } } peers { permanent_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 38951 } } peers { permanent_uuid: "021778186abb464b913200770a437fc5" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 43691 } attrs { } } }
I20260812 08:00:14.684757  2584 raft_consensus.cc:2955] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index -1 to 10, NON_VOTER 021778186abb464b913200770a437fc5 (127.0.186.66) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "08ddd6b135c8405bb15c2fe306b04cee" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40081 } } peers { permanent_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 38951 } } peers { permanent_uuid: "021778186abb464b913200770a437fc5" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 43691 } attrs { } } }
I20260812 08:00:14.687474  2418 catalog_manager.cc:5719] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 reported cstate change: config changed from index -1 to 10, NON_VOTER 021778186abb464b913200770a437fc5 (127.0.186.66) added. New cstate: current_term: 1 leader_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "08ddd6b135c8405bb15c2fe306b04cee" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40081 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 38951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "021778186abb464b913200770a437fc5" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 43691 } attrs { } health_report { overall_health: UNKNOWN } } }
W20260812 08:00:14.688491  2817 consensus_peers.cc:597] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 -> Peer 021778186abb464b913200770a437fc5 (127.0.186.66:43691): Couldn't send request to peer 021778186abb464b913200770a437fc5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 97525c8aad564d008d3a0f6654fc5c4b. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:00:15.062618  2994 ts_tablet_manager.cc:933] T 97525c8aad564d008d3a0f6654fc5c4b P 021778186abb464b913200770a437fc5: Initiating tablet copy from peer c1b9ac9b6bc4426b97bdb7ce6a5b8259 (127.0.186.67:38951)
I20260812 08:00:15.063170  2994 tablet_copy_client.cc:323] T 97525c8aad564d008d3a0f6654fc5c4b P 021778186abb464b913200770a437fc5: tablet copy: Beginning tablet copy session from remote peer at address 127.0.186.67:38951
I20260812 08:00:15.067134  2912 tablet_copy_service.cc:140] P c1b9ac9b6bc4426b97bdb7ce6a5b8259: Received BeginTabletCopySession request for tablet 97525c8aad564d008d3a0f6654fc5c4b from peer 021778186abb464b913200770a437fc5 ({username='slave'} at 127.0.186.66:46153)
I20260812 08:00:15.067313  2912 tablet_copy_service.cc:161] P c1b9ac9b6bc4426b97bdb7ce6a5b8259: Beginning new tablet copy session on tablet 97525c8aad564d008d3a0f6654fc5c4b from peer 021778186abb464b913200770a437fc5 at {username='slave'} at 127.0.186.66:46153: session id = 021778186abb464b913200770a437fc5-97525c8aad564d008d3a0f6654fc5c4b
I20260812 08:00:15.068437  2912 tablet_copy_source_session.cc:215] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 08:00:15.071961  2994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97525c8aad564d008d3a0f6654fc5c4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:15.077632  2994 tablet_copy_client.cc:806] T 97525c8aad564d008d3a0f6654fc5c4b P 021778186abb464b913200770a437fc5: tablet copy: Starting download of 0 data blocks...
I20260812 08:00:15.077880  2994 tablet_copy_client.cc:670] T 97525c8aad564d008d3a0f6654fc5c4b P 021778186abb464b913200770a437fc5: tablet copy: Starting download of 1 WAL segments...
I20260812 08:00:15.079815  2995 tablet_copy_client.cc:1025] T 97525c8aad564d008d3a0f6654fc5c4b P 021778186abb464b913200770a437fc5: tablet copy: Injecting latency into file download: 1000
I20260812 08:00:15.327087  2794 heartbeater.cc:499] Master 127.0.186.126:41067 was elected leader, sending a full tablet report...
I20260812 08:00:16.080256  2994 tablet_copy_client.cc:538] T 97525c8aad564d008d3a0f6654fc5c4b P 021778186abb464b913200770a437fc5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 08:00:16.082288  2994 tablet_bootstrap.cc:492] T 97525c8aad564d008d3a0f6654fc5c4b P 021778186abb464b913200770a437fc5: Bootstrap starting.
I20260812 08:00:16.088117  2994 log.cc:826] T 97525c8aad564d008d3a0f6654fc5c4b P 021778186abb464b913200770a437fc5: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:16.105867  2994 tablet_bootstrap.cc:492] T 97525c8aad564d008d3a0f6654fc5c4b P 021778186abb464b913200770a437fc5: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=400 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:00:16.106240  2994 tablet_bootstrap.cc:492] T 97525c8aad564d008d3a0f6654fc5c4b P 021778186abb464b913200770a437fc5: Bootstrap complete.
I20260812 08:00:16.106583  2994 ts_tablet_manager.cc:1403] T 97525c8aad564d008d3a0f6654fc5c4b P 021778186abb464b913200770a437fc5: Time spent bootstrapping tablet: real 0.024s	user 0.024s	sys 0.000s
I20260812 08:00:16.108932  2994 raft_consensus.cc:359] T 97525c8aad564d008d3a0f6654fc5c4b P 021778186abb464b913200770a437fc5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "08ddd6b135c8405bb15c2fe306b04cee" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40081 } } peers { permanent_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 38951 } } peers { permanent_uuid: "021778186abb464b913200770a437fc5" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 43691 } attrs { } }
I20260812 08:00:16.109195  2994 raft_consensus.cc:740] T 97525c8aad564d008d3a0f6654fc5c4b P 021778186abb464b913200770a437fc5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 021778186abb464b913200770a437fc5, State: Initialized, Role: LEARNER
I20260812 08:00:16.109565  2994 consensus_queue.cc:260] T 97525c8aad564d008d3a0f6654fc5c4b P 021778186abb464b913200770a437fc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 1.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "08ddd6b135c8405bb15c2fe306b04cee" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40081 } } peers { permanent_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 38951 } } peers { permanent_uuid: "021778186abb464b913200770a437fc5" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 43691 } attrs { } }
I20260812 08:00:16.110378  2994 ts_tablet_manager.cc:1434] T 97525c8aad564d008d3a0f6654fc5c4b P 021778186abb464b913200770a437fc5: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260812 08:00:16.111294  2912 tablet_copy_service.cc:342] P c1b9ac9b6bc4426b97bdb7ce6a5b8259: Request end of tablet copy session 021778186abb464b913200770a437fc5-97525c8aad564d008d3a0f6654fc5c4b received from {username='slave'} at 127.0.186.66:46153
I20260812 08:00:16.111428  2912 tablet_copy_service.cc:434] P c1b9ac9b6bc4426b97bdb7ce6a5b8259: ending tablet copy session 021778186abb464b913200770a437fc5-97525c8aad564d008d3a0f6654fc5c4b on tablet 97525c8aad564d008d3a0f6654fc5c4b with peer 021778186abb464b913200770a437fc5
I20260812 08:00:16.122852  2889 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97525c8aad564d008d3a0f6654fc5c4b"
dest_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259"
 from {username='slave'} at 127.0.0.1:33340
I20260812 08:00:16.123077  2889 raft_consensus.cc:591] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [term 1 LEADER]: Received request to step down
I20260812 08:00:16.123128  2889 raft_consensus.cc:3055] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 08:00:16.123148  2889 raft_consensus.cc:740] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c1b9ac9b6bc4426b97bdb7ce6a5b8259, State: Running, Role: LEADER
I20260812 08:00:16.123332  2889 consensus_queue.cc:260] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 10, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "08ddd6b135c8405bb15c2fe306b04cee" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40081 } } peers { permanent_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 38951 } } peers { permanent_uuid: "021778186abb464b913200770a437fc5" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 43691 } attrs { } }
I20260812 08:00:16.123520  2889 raft_consensus.cc:3060] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [term 1 FOLLOWER]: Advancing to term 2
W20260812 08:00:16.130658  2845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.130681  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.130682  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.138258  2539 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.138260  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.138258  2538 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.138264  2537 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.144445  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.144481  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.146842  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.146903  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.152725  2538 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.152747  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.154316  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.154316  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.159212  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.161967  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.161972  2845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.162009  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.168551  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.168557  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.168557  2538 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.169617  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.179246  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.180701  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.181483  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.182510  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.190300  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.192358  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.193250  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.195874  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.204365  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.204619  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.204733  2845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.208058  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.216063  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.217914  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.218735  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.222481  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.232932  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.234280  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.234293  2845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.237838  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.248495  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.248519  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.250851  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.251909  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.264931  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.266446  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.267264  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.267531  2845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.282373  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.283692  2845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.283814  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.284787  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.299641  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.302487  2845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.302487  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.303390  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.318358  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.323076  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.323077  2537 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.323077  2538 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.337831  2538 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.343044  2845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.343044  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.343153  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.361191  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.362483  2538 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.363581  2538 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.365829  2538 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.384696  2538 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.385818  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.386857  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.389022  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.409770  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.411082  2538 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.411187  2537 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.411187  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.433012  2537 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.434415  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.434415  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.436940  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.459606  2537 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.459614  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.459614  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.460642  2537 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.482689  2537 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.488235  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.488247  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.489282  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.509193  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.514230  2537 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.515136  2537 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.516232  2537 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.538288  2537 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.544605  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.544605  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.545244  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.569707  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.571118  2537 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.571120  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.574370  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.601153  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.601715  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.603363  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.603569  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.632413  2538 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.632432  2537 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.632436  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.636420  2538 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.666559  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.667608  2842 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.667755  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.668089  2845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.698772  2845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.701273  2537 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.701269  2538 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.701273  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.733002  2538 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.734441  2845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.734498  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.735323  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.766790  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.768337  2538 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.770344  2538 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.770401  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.803213  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.807397  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.807401  2845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.808210  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.842676  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.842974  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.845378  2538 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.845378  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.878173  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.880964  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.883206  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.885336  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.919418  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.919669  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.922013  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.923087  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.959784  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:16.960549  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.966512  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:16.966699  2845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:17.000615  2845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:17.003106  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:17.006451  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:17.008630  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:17.044221  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:17.047968  2845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:17.048765  2845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:17.049903  2845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:17.089327  2845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:17.089327  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:17.091809  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:17.093914  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:17.133739  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:17.135336  2845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:17.136191  2845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:17.136256  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:17.179711  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:17.179742  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:17.182241  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:17.183248  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:17.226004  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:17.227425  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:17.229010  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:17.231124  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:17.274130  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
W20260812 08:00:17.274446  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:17.277379  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
W20260812 08:00:17.278523  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
I20260812 08:00:17.314486  3013 raft_consensus.cc:493] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b9ac9b6bc4426b97bdb7ce6a5b8259)
I20260812 08:00:17.314644  3013 raft_consensus.cc:515] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "08ddd6b135c8405bb15c2fe306b04cee" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40081 } } peers { permanent_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 38951 } } peers { permanent_uuid: "021778186abb464b913200770a437fc5" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 43691 } attrs { } }
I20260812 08:00:17.316243  3013 leader_election.cc:290] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b9ac9b6bc4426b97bdb7ce6a5b8259 (127.0.186.67:38951)
I20260812 08:00:17.319717  2889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97525c8aad564d008d3a0f6654fc5c4b" candidate_uuid: "08ddd6b135c8405bb15c2fe306b04cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: false dest_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259" is_pre_election: true
I20260812 08:00:17.319937  2889 raft_consensus.cc:2468] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08ddd6b135c8405bb15c2fe306b04cee in term 2.
I20260812 08:00:17.320366  2512 leader_election.cc:304] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 08ddd6b135c8405bb15c2fe306b04cee, c1b9ac9b6bc4426b97bdb7ce6a5b8259; no voters: 
I20260812 08:00:17.320587  3013 raft_consensus.cc:2804] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 08:00:17.320698  3013 raft_consensus.cc:493] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee [term 1 FOLLOWER]: Starting leader election (detected failure of leader c1b9ac9b6bc4426b97bdb7ce6a5b8259)
I20260812 08:00:17.320753  3013 raft_consensus.cc:3060] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee [term 1 FOLLOWER]: Advancing to term 2
W20260812 08:00:17.321350  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
I20260812 08:00:17.321746  3013 raft_consensus.cc:515] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee [term 2 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "08ddd6b135c8405bb15c2fe306b04cee" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40081 } } peers { permanent_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 38951 } } peers { permanent_uuid: "021778186abb464b913200770a437fc5" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 43691 } attrs { } }
I20260812 08:00:17.322126  3013 leader_election.cc:290] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee [CANDIDATE]: Term 2 election: Requested vote from peers c1b9ac9b6bc4426b97bdb7ce6a5b8259 (127.0.186.67:38951)
I20260812 08:00:17.322530  2889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97525c8aad564d008d3a0f6654fc5c4b" candidate_uuid: "08ddd6b135c8405bb15c2fe306b04cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: false dest_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259"
W20260812 08:00:17.322583  2536 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58096: Illegal state: replica 08ddd6b135c8405bb15c2fe306b04cee is not leader of this config: current role FOLLOWER
I20260812 08:00:17.323841  2889 raft_consensus.cc:2468] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08ddd6b135c8405bb15c2fe306b04cee in term 2.
W20260812 08:00:17.324056  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
I20260812 08:00:17.324128  2512 leader_election.cc:304] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 08ddd6b135c8405bb15c2fe306b04cee, c1b9ac9b6bc4426b97bdb7ce6a5b8259; no voters: 
I20260812 08:00:17.324313  3013 raft_consensus.cc:2804] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:00:17.324528  3013 raft_consensus.cc:697] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee [term 2 LEADER]: Becoming Leader. State: Replica: 08ddd6b135c8405bb15c2fe306b04cee, State: Running, Role: LEADER
I20260812 08:00:17.324875  3013 consensus_queue.cc:237] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 10, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "08ddd6b135c8405bb15c2fe306b04cee" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40081 } } peers { permanent_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 38951 } } peers { permanent_uuid: "021778186abb464b913200770a437fc5" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 43691 } attrs { } }
W20260812 08:00:17.326728  2844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33336: Illegal state: replica c1b9ac9b6bc4426b97bdb7ce6a5b8259 is not leader of this config: current role FOLLOWER
I20260812 08:00:17.327862  2417 catalog_manager.cc:5719] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee reported cstate change: term changed from 1 to 2, leader changed from c1b9ac9b6bc4426b97bdb7ce6a5b8259 (127.0.186.67) to 08ddd6b135c8405bb15c2fe306b04cee (127.0.186.65). New cstate: current_term: 2 leader_uuid: "08ddd6b135c8405bb15c2fe306b04cee" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "08ddd6b135c8405bb15c2fe306b04cee" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40081 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 38951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "021778186abb464b913200770a437fc5" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 43691 } attrs { } health_report { overall_health: UNKNOWN } } }
I20260812 08:00:17.375133  2889 raft_consensus.cc:1275] T 97525c8aad564d008d3a0f6654fc5c4b P c1b9ac9b6bc4426b97bdb7ce6a5b8259 [term 2 FOLLOWER]: Refusing update from remote peer 08ddd6b135c8405bb15c2fe306b04cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 2 index: 12. (index mismatch)
I20260812 08:00:17.375855  3017 consensus_queue.cc:1048] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b9ac9b6bc4426b97bdb7ce6a5b8259" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 38951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260812 08:00:17.379072  2737 raft_consensus.cc:3060] T 97525c8aad564d008d3a0f6654fc5c4b P 021778186abb464b913200770a437fc5 [term 1 LEARNER]: Advancing to term 2
I20260812 08:00:17.380409  2737 raft_consensus.cc:1275] T 97525c8aad564d008d3a0f6654fc5c4b P 021778186abb464b913200770a437fc5 [term 2 LEARNER]: Refusing update from remote peer 08ddd6b135c8405bb15c2fe306b04cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 2 index: 12. (index mismatch)
I20260812 08:00:17.381989  3017 consensus_queue.cc:1048] T 97525c8aad564d008d3a0f6654fc5c4b P 08ddd6b135c8405bb15c2fe306b04cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "021778186abb464b913200770a437fc5" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 43691 } attrs { }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260812 08:00:17.392407  3039 mvcc.cc:204] Tried to move back new op lower bound from 7317592544768921600 to 7317592544566263808. Current Snapshot: MvccSnapshot[applied={T|T < 7317592544761724928}]
I20260812 08:00:17.485435  2715 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:00:17.485961  2867 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:00:17.487632  2561 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
Master Summary
               UUID               |       Address       | Status
----------------------------------+---------------------+---------
 cdca8156c1154498b86604448ec6e27b | 127.0.186.126:41067 | HEALTHY

Unusual flags for Master:
                 Flag                  |                                                                                           Value                                                                                            |      Tags       |         Master
---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 allow_unsafe_replication_factor       | true                                                                                                                                                                                       | runtime,unsafe  | all 1 server(s) checked
 catalog_manager_evict_excess_replicas | false                                                                                                                                                                                      | runtime,hidden  | all 1 server(s) checked
 ipki_ca_key_size                      | 768                                                                                                                                                                                        | experimental    | all 1 server(s) checked
 ipki_server_key_size                  | 768                                                                                                                                                                                        | experimental    | all 1 server(s) checked
 never_fsync                           | true                                                                                                                                                                                       | unsafe,advanced | all 1 server(s) checked
 openssl_security_level_override       | 0                                                                                                                                                                                          | unsafe,hidden   | all 1 server(s) checked
 rpc_reuseport                         | true                                                                                                                                                                                       | experimental    | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports      | true                                                                                                                                                                                       | unsafe          | all 1 server(s) checked
 server_dump_info_format               | pb                                                                                                                                                                                         | hidden          | all 1 server(s) checked
 server_dump_info_path                 | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden          | all 1 server(s) checked
 tsk_num_rsa_bits                      | 512                                                                                                                                                                                        | experimental    | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |       Value        |         Master
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.0.186.84:40993 | all 1 server(s) checked
 time_source         | builtin            | all 1 server(s) checked

Tablet Server Summary
               UUID               |      Address       | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
 021778186abb464b913200770a437fc5 | 127.0.186.66:43691 | HEALTHY | <none>   |       0        |       0
 08ddd6b135c8405bb15c2fe306b04cee | 127.0.186.65:40081 | HEALTHY | <none>   |       1        |       0
 c1b9ac9b6bc4426b97bdb7ce6a5b8259 | 127.0.186.67:38951 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
                    Flag                     |                                                                                         Value                                                                                          |      Tags       |      Tablet Server
---------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 ipki_server_key_size                        | 768                                                                                                                                                                                    | experimental    | all 3 server(s) checked
 local_ip_for_outbound_sockets               | 127.0.186.65                                                                                                                                                                           | experimental    | 127.0.186.65:40081
 local_ip_for_outbound_sockets               | 127.0.186.66                                                                                                                                                                           | experimental    | 127.0.186.66:43691
 local_ip_for_outbound_sockets               | 127.0.186.67                                                                                                                                                                           | experimental    | 127.0.186.67:38951
 never_fsync                                 | true                                                                                                                                                                                   | unsafe,advanced | all 3 server(s) checked
 openssl_security_level_override             | 0                                                                                                                                                                                      | unsafe,hidden   | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports            | true                                                                                                                                                                                   | unsafe          | all 3 server(s) checked
 server_dump_info_format                     | pb                                                                                                                                                                                     | hidden          | all 3 server(s) checked
 server_dump_info_path                       | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden          | 127.0.186.65:40081
 server_dump_info_path                       | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden          | 127.0.186.66:43691
 server_dump_info_path                       | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden          | 127.0.186.67:38951
 tablet_copy_download_file_inject_latency_ms | 1000                                                                                                                                                                                   | hidden          | all 3 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |       Value        |      Tablet Server
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.0.186.84:40993 | all 3 server(s) checked
 time_source         | builtin            | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
   Name    | RF | Status  | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-----------+----+---------+---------------+---------+------------+------------------+-------------
 TestTable | 2  | HEALTHY | 1             | 1       | 0          | 0                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

OK
I20260812 08:00:17.520211   745 log_verifier.cc:126] Checking tablet 97525c8aad564d008d3a0f6654fc5c4b
I20260812 08:00:17.551789   745 log_verifier.cc:177] Verified matching terms for 25 ops in tablet 97525c8aad564d008d3a0f6654fc5c4b
I20260812 08:00:17.573608   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 2492
I20260812 08:00:17.584563   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 2642
I20260812 08:00:17.596987   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 2798
I20260812 08:00:17.607174   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 2384
2026-08-12T08:00:17Z chronyd exiting
[       OK ] RaftConsensusNonVoterITest.AddNonVoterReplica (3946 ms)
[ RUN      ] RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica
I20260812 08:00:17.635371   745 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:00:17.635423   745 ts_itest-base.cc:116] --------------
I20260812 08:00:17.635434   745 ts_itest-base.cc:117] 3 tablet servers
I20260812 08:00:17.635444   745 ts_itest-base.cc:118] 2 replicas per TS
I20260812 08:00:17.635452   745 ts_itest-base.cc:119] --------------
2026-08-12T08:00:17Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:00:17Z Disabled control of system clock
I20260812 08:00:17.644616   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:39931
--webserver_interface=127.0.186.126
--webserver_port=0
--builtin_ntp_servers=127.0.186.84:41539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:39931
--allow_unsafe_replication_factor=true
--catalog_manager_evict_excess_replicas=false with env {}
W20260812 08:00:17.756857  3092 flags.cc:432] Enabled unsafe flag: --allow_unsafe_replication_factor=true
W20260812 08:00:17.757124  3092 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:17.757164  3092 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:17.757225  3092 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:17.760862  3092 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:00:17.760954  3092 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:17.760984  3092 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:00:17.761008  3092 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:00:17.765278  3092 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:41539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--allow_unsafe_replication_factor=true
--catalog_manager_evict_excess_replicas=false
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:39931
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:39931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.3092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:17.766803  3092 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:17.768190  3092 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:17.774220  3100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:17.774220  3097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:17.774326  3098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:17.774482  3092 server_base.cc:1061] running on GCE node
I20260812 08:00:17.775077  3092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:17.776235  3092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:17.777421  3092 hybrid_clock.cc:648] HybridClock initialized: now 1786521617777419 us; error 45 us; skew 500 ppm
I20260812 08:00:17.779503  3092 webserver.cc:533] Webserver started at http://127.0.186.126:33641/ using document root <none> and password file <none>
I20260812 08:00:17.780160  3092 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:17.780226  3092 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:17.780433  3092 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:17.782217  3092 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "dc34b63bd6aa412ba0220dd1a4067b4f"
format_stamp: "Formatted at 2026-08-12 08:00:17 on dist-test-slave-49h3"
I20260812 08:00:17.782698  3092 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "dc34b63bd6aa412ba0220dd1a4067b4f"
format_stamp: "Formatted at 2026-08-12 08:00:17 on dist-test-slave-49h3"
I20260812 08:00:17.786588  3092 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260812 08:00:17.789136  3109 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:17.790269  3092 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 08:00:17.790402  3092 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "dc34b63bd6aa412ba0220dd1a4067b4f"
format_stamp: "Formatted at 2026-08-12 08:00:17 on dist-test-slave-49h3"
I20260812 08:00:17.790510  3092 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:17.808331  3092 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:17.809171  3092 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:17.809382  3092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:17.818351  3092 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:39931
I20260812 08:00:17.818375  3168 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:39931 every 8 connection(s)
I20260812 08:00:17.819679  3092 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:00:17.820626   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 3092
I20260812 08:00:17.820747   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:00:17.823832  3169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:17.830000  3169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc34b63bd6aa412ba0220dd1a4067b4f: Bootstrap starting.
I20260812 08:00:17.832768  3169 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc34b63bd6aa412ba0220dd1a4067b4f: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:17.833741  3169 log.cc:826] T 00000000000000000000000000000000 P dc34b63bd6aa412ba0220dd1a4067b4f: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:17.835755  3169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc34b63bd6aa412ba0220dd1a4067b4f: No bootstrap required, opened a new log
I20260812 08:00:17.838737  3169 raft_consensus.cc:359] T 00000000000000000000000000000000 P dc34b63bd6aa412ba0220dd1a4067b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc34b63bd6aa412ba0220dd1a4067b4f" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 39931 } }
I20260812 08:00:17.839046  3169 raft_consensus.cc:385] T 00000000000000000000000000000000 P dc34b63bd6aa412ba0220dd1a4067b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:17.839108  3169 raft_consensus.cc:740] T 00000000000000000000000000000000 P dc34b63bd6aa412ba0220dd1a4067b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc34b63bd6aa412ba0220dd1a4067b4f, State: Initialized, Role: FOLLOWER
I20260812 08:00:17.839685  3169 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc34b63bd6aa412ba0220dd1a4067b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc34b63bd6aa412ba0220dd1a4067b4f" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 39931 } }
I20260812 08:00:17.839911  3169 raft_consensus.cc:399] T 00000000000000000000000000000000 P dc34b63bd6aa412ba0220dd1a4067b4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:00:17.839991  3169 raft_consensus.cc:493] T 00000000000000000000000000000000 P dc34b63bd6aa412ba0220dd1a4067b4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:00:17.840118  3169 raft_consensus.cc:3060] T 00000000000000000000000000000000 P dc34b63bd6aa412ba0220dd1a4067b4f [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:17.841107  3169 raft_consensus.cc:515] T 00000000000000000000000000000000 P dc34b63bd6aa412ba0220dd1a4067b4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc34b63bd6aa412ba0220dd1a4067b4f" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 39931 } }
I20260812 08:00:17.841611  3169 leader_election.cc:304] T 00000000000000000000000000000000 P dc34b63bd6aa412ba0220dd1a4067b4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc34b63bd6aa412ba0220dd1a4067b4f; no voters: 
I20260812 08:00:17.841995  3169 leader_election.cc:290] T 00000000000000000000000000000000 P dc34b63bd6aa412ba0220dd1a4067b4f [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:00:17.842180  3175 raft_consensus.cc:2804] T 00000000000000000000000000000000 P dc34b63bd6aa412ba0220dd1a4067b4f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:00:17.842423  3175 raft_consensus.cc:697] T 00000000000000000000000000000000 P dc34b63bd6aa412ba0220dd1a4067b4f [term 1 LEADER]: Becoming Leader. State: Replica: dc34b63bd6aa412ba0220dd1a4067b4f, State: Running, Role: LEADER
I20260812 08:00:17.842780  3175 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc34b63bd6aa412ba0220dd1a4067b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc34b63bd6aa412ba0220dd1a4067b4f" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 39931 } }
I20260812 08:00:17.843456  3169 sys_catalog.cc:565] T 00000000000000000000000000000000 P dc34b63bd6aa412ba0220dd1a4067b4f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:00:17.844774  3176 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc34b63bd6aa412ba0220dd1a4067b4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc34b63bd6aa412ba0220dd1a4067b4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc34b63bd6aa412ba0220dd1a4067b4f" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 39931 } } }
I20260812 08:00:17.844817  3177 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc34b63bd6aa412ba0220dd1a4067b4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc34b63bd6aa412ba0220dd1a4067b4f. Latest consensus state: current_term: 1 leader_uuid: "dc34b63bd6aa412ba0220dd1a4067b4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc34b63bd6aa412ba0220dd1a4067b4f" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 39931 } } }
I20260812 08:00:17.844975  3176 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc34b63bd6aa412ba0220dd1a4067b4f [sys.catalog]: This master's current role is: LEADER
I20260812 08:00:17.845005  3177 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc34b63bd6aa412ba0220dd1a4067b4f [sys.catalog]: This master's current role is: LEADER
I20260812 08:00:17.845402  3184 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:00:17.849299  3184 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:00:17.856591  3184 catalog_manager.cc:1383] Generated new cluster ID: 74f81c4cf41743658a11627f77e67c30
I20260812 08:00:17.856726  3184 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:00:17.873908  3184 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:00:17.874891  3184 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:00:17.885743  3184 catalog_manager.cc:6092] T 00000000000000000000000000000000 P dc34b63bd6aa412ba0220dd1a4067b4f: Generated new TSK 0
I20260812 08:00:17.886430  3184 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:00:17.889487   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:0
--local_ip_for_outbound_sockets=127.0.186.65
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:39931
--builtin_ntp_servers=127.0.186.84:41539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=50 with env {}
W20260812 08:00:18.024333  3195 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:18.024569  3195 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:18.024612  3195 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:18.028244  3195 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:18.028371  3195 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:00:18.032459  3195 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:41539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=50
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:39931
--never_fsync=true
--heap_profile_path=/tmp/kudu.3195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:18.033672  3195 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:18.034931  3195 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:18.041837  3204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:18.041847  3201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:18.041841  3202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:18.042414  3195 server_base.cc:1061] running on GCE node
I20260812 08:00:18.042850  3195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:18.043475  3195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:18.044644  3195 hybrid_clock.cc:648] HybridClock initialized: now 1786521618044625 us; error 28 us; skew 500 ppm
I20260812 08:00:18.046936  3195 webserver.cc:533] Webserver started at http://127.0.186.65:42047/ using document root <none> and password file <none>
I20260812 08:00:18.047632  3195 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:18.047725  3195 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:18.047950  3195 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:18.049821  3195 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "ebce7e75217a4cf0b57083bbb9b17ec2"
format_stamp: "Formatted at 2026-08-12 08:00:18 on dist-test-slave-49h3"
I20260812 08:00:18.050349  3195 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "ebce7e75217a4cf0b57083bbb9b17ec2"
format_stamp: "Formatted at 2026-08-12 08:00:18 on dist-test-slave-49h3"
I20260812 08:00:18.054416  3195 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260812 08:00:18.057361  3213 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:18.058833  3195 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260812 08:00:18.059023  3195 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "ebce7e75217a4cf0b57083bbb9b17ec2"
format_stamp: "Formatted at 2026-08-12 08:00:18 on dist-test-slave-49h3"
I20260812 08:00:18.059154  3195 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:18.080338  3195 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:18.081105  3195 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:18.081301  3195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:18.081974  3195 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:18.083129  3195 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:18.083181  3195 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:18.083242  3195 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:18.083269  3195 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:18.093060  3195 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:41921
I20260812 08:00:18.093086  3340 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:41921 every 8 connection(s)
I20260812 08:00:18.094172  3195 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:00:18.097249   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 3195
I20260812 08:00:18.097359   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:00:18.099967   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.66:0
--local_ip_for_outbound_sockets=127.0.186.66
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:39931
--builtin_ntp_servers=127.0.186.84:41539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=50 with env {}
I20260812 08:00:18.103991  3342 heartbeater.cc:344] Connected to a master server at 127.0.186.126:39931
I20260812 08:00:18.104298  3342 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:18.104916  3342 heartbeater.cc:507] Master 127.0.186.126:39931 requested a full tablet report, sending...
I20260812 08:00:18.106637  3128 ts_manager.cc:194] Registered new tserver with Master: ebce7e75217a4cf0b57083bbb9b17ec2 (127.0.186.65:41921)
I20260812 08:00:18.107897  3128 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.65:38067
W20260812 08:00:18.213292  3348 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:18.213587  3348 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:18.213649  3348 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:18.217237  3348 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:18.217394  3348 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.66
I20260812 08:00:18.221630  3348 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:41539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=50
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:39931
--never_fsync=true
--heap_profile_path=/tmp/kudu.3348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:18.222985  3348 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:18.224332  3348 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:18.231063  3357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:18.231133  3348 server_base.cc:1061] running on GCE node
W20260812 08:00:18.231137  3354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:18.231065  3359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:18.231617  3348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:18.232147  3348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:18.233340  3348 hybrid_clock.cc:648] HybridClock initialized: now 1786521618233315 us; error 32 us; skew 500 ppm
I20260812 08:00:18.235258  3348 webserver.cc:533] Webserver started at http://127.0.186.66:44191/ using document root <none> and password file <none>
I20260812 08:00:18.235900  3348 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:18.235963  3348 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:18.236166  3348 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:18.237859  3348 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "0076661a9b564772bcbe1a9c8e009f05"
format_stamp: "Formatted at 2026-08-12 08:00:18 on dist-test-slave-49h3"
I20260812 08:00:18.238317  3348 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "0076661a9b564772bcbe1a9c8e009f05"
format_stamp: "Formatted at 2026-08-12 08:00:18 on dist-test-slave-49h3"
I20260812 08:00:18.242120  3348 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.000s
I20260812 08:00:18.244563  3365 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:18.245743  3348 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:00:18.245869  3348 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "0076661a9b564772bcbe1a9c8e009f05"
format_stamp: "Formatted at 2026-08-12 08:00:18 on dist-test-slave-49h3"
I20260812 08:00:18.245972  3348 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:18.272482  3348 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:18.273291  3348 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:18.273509  3348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:18.274140  3348 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:18.275215  3348 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:18.275266  3348 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:18.275323  3348 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:18.275352  3348 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:18.284754  3348 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.66:40197
I20260812 08:00:18.284791  3493 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.66:40197 every 8 connection(s)
I20260812 08:00:18.285872  3348 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:00:18.287720   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 3348
I20260812 08:00:18.287832   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:00:18.290576   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.67:0
--local_ip_for_outbound_sockets=127.0.186.67
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:39931
--builtin_ntp_servers=127.0.186.84:41539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=50 with env {}
I20260812 08:00:18.295348  3494 heartbeater.cc:344] Connected to a master server at 127.0.186.126:39931
I20260812 08:00:18.295683  3494 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:18.296566  3494 heartbeater.cc:507] Master 127.0.186.126:39931 requested a full tablet report, sending...
I20260812 08:00:18.298010  3128 ts_manager.cc:194] Registered new tserver with Master: 0076661a9b564772bcbe1a9c8e009f05 (127.0.186.66:40197)
I20260812 08:00:18.298657  3128 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.66:39637
W20260812 08:00:18.409600  3497 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:18.409919  3497 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:18.410043  3497 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:18.414709  3497 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:18.414868  3497 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.67
I20260812 08:00:18.420496  3497 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:41539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=50
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:39931
--never_fsync=true
--heap_profile_path=/tmp/kudu.3497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:18.422060  3497 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:18.423796  3497 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:18.432550  3505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:18.432504  3504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:18.432756  3497 server_base.cc:1061] running on GCE node
W20260812 08:00:18.432924  3508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:18.433394  3497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:18.434199  3497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:18.435416  3497 hybrid_clock.cc:648] HybridClock initialized: now 1786521618435383 us; error 56 us; skew 500 ppm
I20260812 08:00:18.438228  3497 webserver.cc:533] Webserver started at http://127.0.186.67:44507/ using document root <none> and password file <none>
I20260812 08:00:18.439029  3497 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:18.439108  3497 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:18.439352  3497 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:18.442114  3497 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "07dec9146dce416ebf095855fc6ff00a"
format_stamp: "Formatted at 2026-08-12 08:00:18 on dist-test-slave-49h3"
I20260812 08:00:18.442775  3497 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "07dec9146dce416ebf095855fc6ff00a"
format_stamp: "Formatted at 2026-08-12 08:00:18 on dist-test-slave-49h3"
I20260812 08:00:18.447577  3497 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260812 08:00:18.450141  3516 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:18.451380  3497 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:00:18.451502  3497 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "07dec9146dce416ebf095855fc6ff00a"
format_stamp: "Formatted at 2026-08-12 08:00:18 on dist-test-slave-49h3"
I20260812 08:00:18.451624  3497 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:18.470299  3497 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:18.471055  3497 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:18.471243  3497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:18.471868  3497 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:18.472968  3497 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:18.473016  3497 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:18.473063  3497 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:18.473093  3497 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:18.482784  3497 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.67:33861
I20260812 08:00:18.482824  3645 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.67:33861 every 8 connection(s)
I20260812 08:00:18.483918  3497 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:00:18.488296   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 3497
I20260812 08:00:18.488420   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:00:18.493108  3646 heartbeater.cc:344] Connected to a master server at 127.0.186.126:39931
I20260812 08:00:18.493402  3646 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:18.494035  3646 heartbeater.cc:507] Master 127.0.186.126:39931 requested a full tablet report, sending...
I20260812 08:00:18.495203  3128 ts_manager.cc:194] Registered new tserver with Master: 07dec9146dce416ebf095855fc6ff00a (127.0.186.67:33861)
I20260812 08:00:18.495801  3128 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.67:56917
I20260812 08:00:18.502885   745 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:00:18.511616  3128 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49634:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
I20260812 08:00:18.530331  3265 tablet_service.cc:1511] Processing CreateTablet for tablet ee8b4348af2c454bbc6666f3f101bb59 (DEFAULT_TABLE table=TestTable [id=3315796743994665b910d92c8465e383]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:18.530323  3417 tablet_service.cc:1511] Processing CreateTablet for tablet ee8b4348af2c454bbc6666f3f101bb59 (DEFAULT_TABLE table=TestTable [id=3315796743994665b910d92c8465e383]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:18.531342  3417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee8b4348af2c454bbc6666f3f101bb59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:18.531339  3265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee8b4348af2c454bbc6666f3f101bb59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:18.536391  3664 tablet_bootstrap.cc:492] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05: Bootstrap starting.
I20260812 08:00:18.536391  3663 tablet_bootstrap.cc:492] T ee8b4348af2c454bbc6666f3f101bb59 P ebce7e75217a4cf0b57083bbb9b17ec2: Bootstrap starting.
I20260812 08:00:18.538594  3663 tablet_bootstrap.cc:654] T ee8b4348af2c454bbc6666f3f101bb59 P ebce7e75217a4cf0b57083bbb9b17ec2: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:18.538589  3664 tablet_bootstrap.cc:654] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:18.539338  3664 log.cc:826] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:18.539340  3663 log.cc:826] T ee8b4348af2c454bbc6666f3f101bb59 P ebce7e75217a4cf0b57083bbb9b17ec2: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:18.541021  3664 tablet_bootstrap.cc:492] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05: No bootstrap required, opened a new log
I20260812 08:00:18.541021  3663 tablet_bootstrap.cc:492] T ee8b4348af2c454bbc6666f3f101bb59 P ebce7e75217a4cf0b57083bbb9b17ec2: No bootstrap required, opened a new log
I20260812 08:00:18.541213  3663 ts_tablet_manager.cc:1403] T ee8b4348af2c454bbc6666f3f101bb59 P ebce7e75217a4cf0b57083bbb9b17ec2: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:00:18.541213  3664 ts_tablet_manager.cc:1403] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05: Time spent bootstrapping tablet: real 0.005s	user 0.000s	sys 0.004s
I20260812 08:00:18.543664  3664 raft_consensus.cc:359] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebce7e75217a4cf0b57083bbb9b17ec2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41921 } } peers { permanent_uuid: "0076661a9b564772bcbe1a9c8e009f05" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40197 } }
I20260812 08:00:18.543658  3663 raft_consensus.cc:359] T ee8b4348af2c454bbc6666f3f101bb59 P ebce7e75217a4cf0b57083bbb9b17ec2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebce7e75217a4cf0b57083bbb9b17ec2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41921 } } peers { permanent_uuid: "0076661a9b564772bcbe1a9c8e009f05" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40197 } }
I20260812 08:00:18.543851  3663 raft_consensus.cc:385] T ee8b4348af2c454bbc6666f3f101bb59 P ebce7e75217a4cf0b57083bbb9b17ec2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:18.543853  3664 raft_consensus.cc:385] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:18.543902  3664 raft_consensus.cc:740] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0076661a9b564772bcbe1a9c8e009f05, State: Initialized, Role: FOLLOWER
I20260812 08:00:18.543901  3663 raft_consensus.cc:740] T ee8b4348af2c454bbc6666f3f101bb59 P ebce7e75217a4cf0b57083bbb9b17ec2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebce7e75217a4cf0b57083bbb9b17ec2, State: Initialized, Role: FOLLOWER
I20260812 08:00:18.544327  3663 consensus_queue.cc:260] T ee8b4348af2c454bbc6666f3f101bb59 P ebce7e75217a4cf0b57083bbb9b17ec2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebce7e75217a4cf0b57083bbb9b17ec2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41921 } } peers { permanent_uuid: "0076661a9b564772bcbe1a9c8e009f05" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40197 } }
I20260812 08:00:18.544327  3664 consensus_queue.cc:260] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebce7e75217a4cf0b57083bbb9b17ec2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41921 } } peers { permanent_uuid: "0076661a9b564772bcbe1a9c8e009f05" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40197 } }
I20260812 08:00:18.545056  3494 heartbeater.cc:499] Master 127.0.186.126:39931 was elected leader, sending a full tablet report...
I20260812 08:00:18.545070  3342 heartbeater.cc:499] Master 127.0.186.126:39931 was elected leader, sending a full tablet report...
I20260812 08:00:18.545226  3664 ts_tablet_manager.cc:1434] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.001s
I20260812 08:00:18.545228  3663 ts_tablet_manager.cc:1434] T ee8b4348af2c454bbc6666f3f101bb59 P ebce7e75217a4cf0b57083bbb9b17ec2: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
W20260812 08:00:18.595361  3343 tablet.cc:2406] T ee8b4348af2c454bbc6666f3f101bb59 P ebce7e75217a4cf0b57083bbb9b17ec2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:00:18.622716  3667 raft_consensus.cc:493] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:00:18.622939  3667 raft_consensus.cc:515] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebce7e75217a4cf0b57083bbb9b17ec2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41921 } } peers { permanent_uuid: "0076661a9b564772bcbe1a9c8e009f05" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40197 } }
I20260812 08:00:18.624215  3667 leader_election.cc:290] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ebce7e75217a4cf0b57083bbb9b17ec2 (127.0.186.65:41921)
I20260812 08:00:18.627966  3287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee8b4348af2c454bbc6666f3f101bb59" candidate_uuid: "0076661a9b564772bcbe1a9c8e009f05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebce7e75217a4cf0b57083bbb9b17ec2" is_pre_election: true
I20260812 08:00:18.628319  3287 raft_consensus.cc:2468] T ee8b4348af2c454bbc6666f3f101bb59 P ebce7e75217a4cf0b57083bbb9b17ec2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0076661a9b564772bcbe1a9c8e009f05 in term 0.
I20260812 08:00:18.629285  3368 leader_election.cc:304] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 0076661a9b564772bcbe1a9c8e009f05, ebce7e75217a4cf0b57083bbb9b17ec2; no voters: 
I20260812 08:00:18.629632  3667 raft_consensus.cc:2804] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:00:18.629714  3667 raft_consensus.cc:493] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:00:18.629765  3667 raft_consensus.cc:3060] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:18.630716  3667 raft_consensus.cc:515] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebce7e75217a4cf0b57083bbb9b17ec2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41921 } } peers { permanent_uuid: "0076661a9b564772bcbe1a9c8e009f05" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40197 } }
I20260812 08:00:18.631088  3667 leader_election.cc:290] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 [CANDIDATE]: Term 1 election: Requested vote from peers ebce7e75217a4cf0b57083bbb9b17ec2 (127.0.186.65:41921)
I20260812 08:00:18.631682  3287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee8b4348af2c454bbc6666f3f101bb59" candidate_uuid: "0076661a9b564772bcbe1a9c8e009f05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebce7e75217a4cf0b57083bbb9b17ec2"
I20260812 08:00:18.631847  3287 raft_consensus.cc:3060] T ee8b4348af2c454bbc6666f3f101bb59 P ebce7e75217a4cf0b57083bbb9b17ec2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:18.633330  3287 raft_consensus.cc:2468] T ee8b4348af2c454bbc6666f3f101bb59 P ebce7e75217a4cf0b57083bbb9b17ec2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0076661a9b564772bcbe1a9c8e009f05 in term 1.
I20260812 08:00:18.633806  3368 leader_election.cc:304] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 0076661a9b564772bcbe1a9c8e009f05, ebce7e75217a4cf0b57083bbb9b17ec2; no voters: 
I20260812 08:00:18.634045  3667 raft_consensus.cc:2804] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:00:18.634271  3667 raft_consensus.cc:697] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 [term 1 LEADER]: Becoming Leader. State: Replica: 0076661a9b564772bcbe1a9c8e009f05, State: Running, Role: LEADER
I20260812 08:00:18.634596  3667 consensus_queue.cc:237] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebce7e75217a4cf0b57083bbb9b17ec2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41921 } } peers { permanent_uuid: "0076661a9b564772bcbe1a9c8e009f05" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40197 } }
I20260812 08:00:18.638151  3126 catalog_manager.cc:5719] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0076661a9b564772bcbe1a9c8e009f05 (127.0.186.66). New cstate: current_term: 1 leader_uuid: "0076661a9b564772bcbe1a9c8e009f05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebce7e75217a4cf0b57083bbb9b17ec2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0076661a9b564772bcbe1a9c8e009f05" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40197 } health_report { overall_health: HEALTHY } } }
I20260812 08:00:18.650048   745 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:00:18.651553   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver ebce7e75217a4cf0b57083bbb9b17ec2 to finish bootstrapping
I20260812 08:00:18.660209   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 0076661a9b564772bcbe1a9c8e009f05 to finish bootstrapping
W20260812 08:00:18.787011  3495 tablet.cc:2406] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:00:19.022969  3667 consensus_queue.cc:1048] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebce7e75217a4cf0b57083bbb9b17ec2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:00:19.029731  3441 consensus_queue.cc:237] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ebce7e75217a4cf0b57083bbb9b17ec2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41921 } } peers { permanent_uuid: "0076661a9b564772bcbe1a9c8e009f05" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40197 } } peers { permanent_uuid: "07dec9146dce416ebf095855fc6ff00a" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 33861 } attrs { } }
I20260812 08:00:19.031495  3287 raft_consensus.cc:1275] T ee8b4348af2c454bbc6666f3f101bb59 P ebce7e75217a4cf0b57083bbb9b17ec2 [term 1 FOLLOWER]: Refusing update from remote peer 0076661a9b564772bcbe1a9c8e009f05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:00:19.031884  3667 consensus_queue.cc:1048] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebce7e75217a4cf0b57083bbb9b17ec2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:00:19.033833  3669 raft_consensus.cc:2955] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 07dec9146dce416ebf095855fc6ff00a (127.0.186.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ebce7e75217a4cf0b57083bbb9b17ec2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41921 } } peers { permanent_uuid: "0076661a9b564772bcbe1a9c8e009f05" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40197 } } peers { permanent_uuid: "07dec9146dce416ebf095855fc6ff00a" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 33861 } attrs { } } }
I20260812 08:00:19.034265  3287 raft_consensus.cc:2955] T ee8b4348af2c454bbc6666f3f101bb59 P ebce7e75217a4cf0b57083bbb9b17ec2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 07dec9146dce416ebf095855fc6ff00a (127.0.186.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ebce7e75217a4cf0b57083bbb9b17ec2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41921 } } peers { permanent_uuid: "0076661a9b564772bcbe1a9c8e009f05" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40197 } } peers { permanent_uuid: "07dec9146dce416ebf095855fc6ff00a" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 33861 } attrs { } } }
I20260812 08:00:19.037039  3127 catalog_manager.cc:5719] T ee8b4348af2c454bbc6666f3f101bb59 P ebce7e75217a4cf0b57083bbb9b17ec2 reported cstate change: config changed from index -1 to 2, NON_VOTER 07dec9146dce416ebf095855fc6ff00a (127.0.186.67) added. New cstate: current_term: 1 leader_uuid: "0076661a9b564772bcbe1a9c8e009f05" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ebce7e75217a4cf0b57083bbb9b17ec2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41921 } } peers { permanent_uuid: "0076661a9b564772bcbe1a9c8e009f05" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40197 } } peers { permanent_uuid: "07dec9146dce416ebf095855fc6ff00a" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 33861 } attrs { } } }
W20260812 08:00:19.037632  3368 consensus_peers.cc:597] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 -> Peer 07dec9146dce416ebf095855fc6ff00a (127.0.186.67:33861): Couldn't send request to peer 07dec9146dce416ebf095855fc6ff00a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ee8b4348af2c454bbc6666f3f101bb59. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:00:19.046063   745 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
I20260812 08:00:19.436259  3702 ts_tablet_manager.cc:933] T ee8b4348af2c454bbc6666f3f101bb59 P 07dec9146dce416ebf095855fc6ff00a: Initiating tablet copy from peer 0076661a9b564772bcbe1a9c8e009f05 (127.0.186.66:40197)
I20260812 08:00:19.436911  3702 tablet_copy_client.cc:323] T ee8b4348af2c454bbc6666f3f101bb59 P 07dec9146dce416ebf095855fc6ff00a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.186.66:40197
I20260812 08:00:19.440943  3463 tablet_copy_service.cc:140] P 0076661a9b564772bcbe1a9c8e009f05: Received BeginTabletCopySession request for tablet ee8b4348af2c454bbc6666f3f101bb59 from peer 07dec9146dce416ebf095855fc6ff00a ({username='slave'} at 127.0.186.67:34815)
I20260812 08:00:19.441126  3463 tablet_copy_service.cc:161] P 0076661a9b564772bcbe1a9c8e009f05: Beginning new tablet copy session on tablet ee8b4348af2c454bbc6666f3f101bb59 from peer 07dec9146dce416ebf095855fc6ff00a at {username='slave'} at 127.0.186.67:34815: session id = 07dec9146dce416ebf095855fc6ff00a-ee8b4348af2c454bbc6666f3f101bb59
I20260812 08:00:19.442389  3463 tablet_copy_source_session.cc:215] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 08:00:19.446219  3702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee8b4348af2c454bbc6666f3f101bb59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:19.451563  3702 tablet_copy_client.cc:806] T ee8b4348af2c454bbc6666f3f101bb59 P 07dec9146dce416ebf095855fc6ff00a: tablet copy: Starting download of 0 data blocks...
I20260812 08:00:19.451803  3702 tablet_copy_client.cc:670] T ee8b4348af2c454bbc6666f3f101bb59 P 07dec9146dce416ebf095855fc6ff00a: tablet copy: Starting download of 1 WAL segments...
I20260812 08:00:19.453835  3702 tablet_copy_client.cc:538] T ee8b4348af2c454bbc6666f3f101bb59 P 07dec9146dce416ebf095855fc6ff00a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 08:00:19.455961  3702 tablet_bootstrap.cc:492] T ee8b4348af2c454bbc6666f3f101bb59 P 07dec9146dce416ebf095855fc6ff00a: Bootstrap starting.
I20260812 08:00:19.463587  3702 log.cc:826] T ee8b4348af2c454bbc6666f3f101bb59 P 07dec9146dce416ebf095855fc6ff00a: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:19.483870  3702 tablet_bootstrap.cc:492] T ee8b4348af2c454bbc6666f3f101bb59 P 07dec9146dce416ebf095855fc6ff00a: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=400 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:00:19.484256  3702 tablet_bootstrap.cc:492] T ee8b4348af2c454bbc6666f3f101bb59 P 07dec9146dce416ebf095855fc6ff00a: Bootstrap complete.
I20260812 08:00:19.484624  3702 ts_tablet_manager.cc:1403] T ee8b4348af2c454bbc6666f3f101bb59 P 07dec9146dce416ebf095855fc6ff00a: Time spent bootstrapping tablet: real 0.029s	user 0.019s	sys 0.008s
I20260812 08:00:19.487522  3702 raft_consensus.cc:359] T ee8b4348af2c454bbc6666f3f101bb59 P 07dec9146dce416ebf095855fc6ff00a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ebce7e75217a4cf0b57083bbb9b17ec2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41921 } } peers { permanent_uuid: "0076661a9b564772bcbe1a9c8e009f05" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40197 } } peers { permanent_uuid: "07dec9146dce416ebf095855fc6ff00a" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 33861 } attrs { } }
I20260812 08:00:19.487783  3702 raft_consensus.cc:740] T ee8b4348af2c454bbc6666f3f101bb59 P 07dec9146dce416ebf095855fc6ff00a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 07dec9146dce416ebf095855fc6ff00a, State: Initialized, Role: LEARNER
I20260812 08:00:19.488153  3702 consensus_queue.cc:260] T ee8b4348af2c454bbc6666f3f101bb59 P 07dec9146dce416ebf095855fc6ff00a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 1.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ebce7e75217a4cf0b57083bbb9b17ec2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41921 } } peers { permanent_uuid: "0076661a9b564772bcbe1a9c8e009f05" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40197 } } peers { permanent_uuid: "07dec9146dce416ebf095855fc6ff00a" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 33861 } attrs { } }
I20260812 08:00:19.488929  3646 heartbeater.cc:499] Master 127.0.186.126:39931 was elected leader, sending a full tablet report...
I20260812 08:00:19.489132  3702 ts_tablet_manager.cc:1434] T ee8b4348af2c454bbc6666f3f101bb59 P 07dec9146dce416ebf095855fc6ff00a: Time spent starting tablet: real 0.004s	user 0.006s	sys 0.001s
I20260812 08:00:19.490127  3463 tablet_copy_service.cc:342] P 0076661a9b564772bcbe1a9c8e009f05: Request end of tablet copy session 07dec9146dce416ebf095855fc6ff00a-ee8b4348af2c454bbc6666f3f101bb59 received from {username='slave'} at 127.0.186.67:34815
I20260812 08:00:19.491115  3463 tablet_copy_service.cc:434] P 0076661a9b564772bcbe1a9c8e009f05: ending tablet copy session 07dec9146dce416ebf095855fc6ff00a-ee8b4348af2c454bbc6666f3f101bb59 on tablet ee8b4348af2c454bbc6666f3f101bb59 with peer 07dec9146dce416ebf095855fc6ff00a
I20260812 08:00:19.551777  3417 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260812 08:00:19.552134  3569 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:00:19.552726  3265 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |       Address       | Status
----------------------------------+---------------------+---------
 dc34b63bd6aa412ba0220dd1a4067b4f | 127.0.186.126:39931 | HEALTHY

Unusual flags for Master:
                 Flag                  |                                                                                                Value                                                                                                 |      Tags       |         Master
---------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 allow_unsafe_replication_factor       | true                                                                                                                                                                                                 | runtime,unsafe  | all 1 server(s) checked
 catalog_manager_evict_excess_replicas | false                                                                                                                                                                                                | runtime,hidden  | all 1 server(s) checked
 ipki_ca_key_size                      | 768                                                                                                                                                                                                  | experimental    | all 1 server(s) checked
 ipki_server_key_size                  | 768                                                                                                                                                                                                  | experimental    | all 1 server(s) checked
 never_fsync                           | true                                                                                                                                                                                                 | unsafe,advanced | all 1 server(s) checked
 openssl_security_level_override       | 0                                                                                                                                                                                                    | unsafe,hidden   | all 1 server(s) checked
 rpc_reuseport                         | true                                                                                                                                                                                                 | experimental    | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports      | true                                                                                                                                                                                                 | unsafe          | all 1 server(s) checked
 server_dump_info_format               | pb                                                                                                                                                                                                   | hidden          | all 1 server(s) checked
 server_dump_info_path                 | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden          | all 1 server(s) checked
 tsk_num_rsa_bits                      | 512                                                                                                                                                                                                  | experimental    | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |       Value        |         Master
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.0.186.84:41539 | all 1 server(s) checked
 time_source         | builtin            | all 1 server(s) checked

Tablet Server Summary
               UUID               |      Address       | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
 0076661a9b564772bcbe1a9c8e009f05 | 127.0.186.66:40197 | HEALTHY | <none>   |       1        |       0
 07dec9146dce416ebf095855fc6ff00a | 127.0.186.67:33861 | HEALTHY | <none>   |       0        |       0
 ebce7e75217a4cf0b57083bbb9b17ec2 | 127.0.186.65:41921 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
               Flag               |                                                                                              Value                                                                                               |      Tags       |      Tablet Server
----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 ipki_server_key_size             | 768                                                                                                                                                                                              | experimental    | all 3 server(s) checked
 local_ip_for_outbound_sockets    | 127.0.186.65                                                                                                                                                                                     | experimental    | 127.0.186.65:41921
 local_ip_for_outbound_sockets    | 127.0.186.66                                                                                                                                                                                     | experimental    | 127.0.186.66:40197
 local_ip_for_outbound_sockets    | 127.0.186.67                                                                                                                                                                                     | experimental    | 127.0.186.67:33861
 never_fsync                      | true                                                                                                                                                                                             | unsafe,advanced | all 3 server(s) checked
 openssl_security_level_override  | 0                                                                                                                                                                                                | unsafe,hidden   | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports | true                                                                                                                                                                                             | unsafe          | all 3 server(s) checked
 server_dump_info_format          | pb                                                                                                                                                                                               | hidden          | all 3 server(s) checked
 server_dump_info_path            | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden          | 127.0.186.65:41921
 server_dump_info_path            | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden          | 127.0.186.66:40197
 server_dump_info_path            | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden          | 127.0.186.67:33861

Flags of checked categories for Tablet Server:
        Flag         |       Value        |      Tablet Server
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.0.186.84:41539 | all 3 server(s) checked
 time_source         | builtin            | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
Tablet ee8b4348af2c454bbc6666f3f101bb59 of table 'TestTable' is conflicted: 1 replicas' active configs disagree with the leader master's
  ebce7e75217a4cf0b57083bbb9b17ec2 (127.0.186.65:41921): RUNNING
  0076661a9b564772bcbe1a9c8e009f05 (127.0.186.66:40197): RUNNING [LEADER]
  07dec9146dce416ebf095855fc6ff00a (127.0.186.67:33861): RUNNING [NONVOTER]
All reported replicas are:
  A = ebce7e75217a4cf0b57083bbb9b17ec2
  B = 0076661a9b564772bcbe1a9c8e009f05
  C = 07dec9146dce416ebf095855fc6ff00a
The consensus matrix is:
 Config source |   Replicas   | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
 master        | A   B*  C~   |              |              | Yes
 A             | A   B*  C~   | 1            | 2            | Yes
 B             | A   B*  C~   | 1            | 2            | Yes
 C             | A   B   C~   | 1            | 2            | Yes

The cluster doesn't have any matching system tables
Summary by table
   Name    | RF |       Status       | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-----------+----+--------------------+---------------+---------+------------+------------------+-------------
 TestTable | 2  | CONSENSUS_MISMATCH | 1             | 0       | 0          | 0                | 1

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20260812 08:00:19.578514   745 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260812 08:00:19.779714  3265 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:00:19.779734  3417 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260812 08:00:19.779784  3569 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |       Address       | Status
----------------------------------+---------------------+---------
 dc34b63bd6aa412ba0220dd1a4067b4f | 127.0.186.126:39931 | HEALTHY

Unusual flags for Master:
                 Flag                  |                                                                                                Value                                                                                                 |      Tags       |         Master
---------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 allow_unsafe_replication_factor       | true                                                                                                                                                                                                 | runtime,unsafe  | all 1 server(s) checked
 catalog_manager_evict_excess_replicas | false                                                                                                                                                                                                | runtime,hidden  | all 1 server(s) checked
 ipki_ca_key_size                      | 768                                                                                                                                                                                                  | experimental    | all 1 server(s) checked
 ipki_server_key_size                  | 768                                                                                                                                                                                                  | experimental    | all 1 server(s) checked
 never_fsync                           | true                                                                                                                                                                                                 | unsafe,advanced | all 1 server(s) checked
 openssl_security_level_override       | 0                                                                                                                                                                                                    | unsafe,hidden   | all 1 server(s) checked
 rpc_reuseport                         | true                                                                                                                                                                                                 | experimental    | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports      | true                                                                                                                                                                                                 | unsafe          | all 1 server(s) checked
 server_dump_info_format               | pb                                                                                                                                                                                                   | hidden          | all 1 server(s) checked
 server_dump_info_path                 | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden          | all 1 server(s) checked
 tsk_num_rsa_bits                      | 512                                                                                                                                                                                                  | experimental    | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |       Value        |         Master
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.0.186.84:41539 | all 1 server(s) checked
 time_source         | builtin            | all 1 server(s) checked

Tablet Server Summary
               UUID               |      Address       | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
 0076661a9b564772bcbe1a9c8e009f05 | 127.0.186.66:40197 | HEALTHY | <none>   |       1        |       0
 07dec9146dce416ebf095855fc6ff00a | 127.0.186.67:33861 | HEALTHY | <none>   |       0        |       0
 ebce7e75217a4cf0b57083bbb9b17ec2 | 127.0.186.65:41921 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
               Flag               |                                                                                              Value                                                                                               |      Tags       |      Tablet Server
----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 ipki_server_key_size             | 768                                                                                                                                                                                              | experimental    | all 3 server(s) checked
 local_ip_for_outbound_sockets    | 127.0.186.65                                                                                                                                                                                     | experimental    | 127.0.186.65:41921
 local_ip_for_outbound_sockets    | 127.0.186.66                                                                                                                                                                                     | experimental    | 127.0.186.66:40197
 local_ip_for_outbound_sockets    | 127.0.186.67                                                                                                                                                                                     | experimental    | 127.0.186.67:33861
 never_fsync                      | true                                                                                                                                                                                             | unsafe,advanced | all 3 server(s) checked
 openssl_security_level_override  | 0                                                                                                                                                                                                | unsafe,hidden   | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports | true                                                                                                                                                                                             | unsafe          | all 3 server(s) checked
 server_dump_info_format          | pb                                                                                                                                                                                               | hidden          | all 3 server(s) checked
 server_dump_info_path            | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden          | 127.0.186.65:41921
 server_dump_info_path            | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden          | 127.0.186.66:40197
 server_dump_info_path            | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden          | 127.0.186.67:33861

Flags of checked categories for Tablet Server:
        Flag         |       Value        |      Tablet Server
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.0.186.84:41539 | all 3 server(s) checked
 time_source         | builtin            | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
Tablet ee8b4348af2c454bbc6666f3f101bb59 of table 'TestTable' is conflicted: 1 replicas' active configs disagree with the leader master's
  ebce7e75217a4cf0b57083bbb9b17ec2 (127.0.186.65:41921): RUNNING
  0076661a9b564772bcbe1a9c8e009f05 (127.0.186.66:40197): RUNNING [LEADER]
  07dec9146dce416ebf095855fc6ff00a (127.0.186.67:33861): RUNNING [NONVOTER]
All reported replicas are:
  A = ebce7e75217a4cf0b57083bbb9b17ec2
  B = 0076661a9b564772bcbe1a9c8e009f05
  C = 07dec9146dce416ebf095855fc6ff00a
The consensus matrix is:
 Config source |   Replicas   | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
 master        | A   B*  C~   |              |              | Yes
 A             | A   B*  C~   | 1            | 2            | Yes
 B             | A   B*  C~   | 1            | 2            | Yes
 C             | A   B   C~   | 1            | 2            | Yes

The cluster doesn't have any matching system tables
Summary by table
   Name    | RF |       Status       | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-----------+----+--------------------+---------------+---------+------------+------------------+-------------
 TestTable | 2  | CONSENSUS_MISMATCH | 1             | 0       | 0          | 0                | 1

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20260812 08:00:19.809615   745 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260812 08:00:19.868232  3594 raft_consensus.cc:1217] T ee8b4348af2c454bbc6666f3f101bb59 P 07dec9146dce416ebf095855fc6ff00a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.10]   Dedup: 1.10->[]
I20260812 08:00:20.086776  3417 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260812 08:00:20.087113  3265 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:00:20.088128  3569 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |       Address       | Status
----------------------------------+---------------------+---------
 dc34b63bd6aa412ba0220dd1a4067b4f | 127.0.186.126:39931 | HEALTHY

Unusual flags for Master:
                 Flag                  |                                                                                                Value                                                                                                 |      Tags       |         Master
---------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 allow_unsafe_replication_factor       | true                                                                                                                                                                                                 | runtime,unsafe  | all 1 server(s) checked
 catalog_manager_evict_excess_replicas | false                                                                                                                                                                                                | runtime,hidden  | all 1 server(s) checked
 ipki_ca_key_size                      | 768                                                                                                                                                                                                  | experimental    | all 1 server(s) checked
 ipki_server_key_size                  | 768                                                                                                                                                                                                  | experimental    | all 1 server(s) checked
 never_fsync                           | true                                                                                                                                                                                                 | unsafe,advanced | all 1 server(s) checked
 openssl_security_level_override       | 0                                                                                                                                                                                                    | unsafe,hidden   | all 1 server(s) checked
 rpc_reuseport                         | true                                                                                                                                                                                                 | experimental    | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports      | true                                                                                                                                                                                                 | unsafe          | all 1 server(s) checked
 server_dump_info_format               | pb                                                                                                                                                                                                   | hidden          | all 1 server(s) checked
 server_dump_info_path                 | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden          | all 1 server(s) checked
 tsk_num_rsa_bits                      | 512                                                                                                                                                                                                  | experimental    | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |       Value        |         Master
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.0.186.84:41539 | all 1 server(s) checked
 time_source         | builtin            | all 1 server(s) checked

Tablet Server Summary
               UUID               |      Address       | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
 0076661a9b564772bcbe1a9c8e009f05 | 127.0.186.66:40197 | HEALTHY | <none>   |       1        |       0
 07dec9146dce416ebf095855fc6ff00a | 127.0.186.67:33861 | HEALTHY | <none>   |       0        |       0
 ebce7e75217a4cf0b57083bbb9b17ec2 | 127.0.186.65:41921 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
               Flag               |                                                                                              Value                                                                                               |      Tags       |      Tablet Server
----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 ipki_server_key_size             | 768                                                                                                                                                                                              | experimental    | all 3 server(s) checked
 local_ip_for_outbound_sockets    | 127.0.186.65                                                                                                                                                                                     | experimental    | 127.0.186.65:41921
 local_ip_for_outbound_sockets    | 127.0.186.66                                                                                                                                                                                     | experimental    | 127.0.186.66:40197
 local_ip_for_outbound_sockets    | 127.0.186.67                                                                                                                                                                                     | experimental    | 127.0.186.67:33861
 never_fsync                      | true                                                                                                                                                                                             | unsafe,advanced | all 3 server(s) checked
 openssl_security_level_override  | 0                                                                                                                                                                                                | unsafe,hidden   | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports | true                                                                                                                                                                                             | unsafe          | all 3 server(s) checked
 server_dump_info_format          | pb                                                                                                                                                                                               | hidden          | all 3 server(s) checked
 server_dump_info_path            | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden          | 127.0.186.65:41921
 server_dump_info_path            | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden          | 127.0.186.66:40197
 server_dump_info_path            | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden          | 127.0.186.67:33861

Flags of checked categories for Tablet Server:
        Flag         |       Value        |      Tablet Server
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.0.186.84:41539 | all 3 server(s) checked
 time_source         | builtin            | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
   Name    | RF | Status  | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-----------+----+---------+---------------+---------+------------+------------------+-------------
 TestTable | 2  | HEALTHY | 1             | 1       | 0          | 0                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

OK
I20260812 08:00:20.119316   745 log_verifier.cc:126] Checking tablet ee8b4348af2c454bbc6666f3f101bb59
I20260812 08:00:20.147652   745 log_verifier.cc:177] Verified matching terms for 10 ops in tablet ee8b4348af2c454bbc6666f3f101bb59
I20260812 08:00:20.637859  3441 consensus_queue.cc:237] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ebce7e75217a4cf0b57083bbb9b17ec2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41921 } } peers { permanent_uuid: "0076661a9b564772bcbe1a9c8e009f05" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40197 } }
I20260812 08:00:20.639535  3286 raft_consensus.cc:1275] T ee8b4348af2c454bbc6666f3f101bb59 P ebce7e75217a4cf0b57083bbb9b17ec2 [term 1 FOLLOWER]: Refusing update from remote peer 0076661a9b564772bcbe1a9c8e009f05: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260812 08:00:20.640096  3761 consensus_queue.cc:1048] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebce7e75217a4cf0b57083bbb9b17ec2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260812 08:00:20.642551  3797 raft_consensus.cc:2955] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 2 to 11, NON_VOTER 07dec9146dce416ebf095855fc6ff00a (127.0.186.67) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ebce7e75217a4cf0b57083bbb9b17ec2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41921 } } peers { permanent_uuid: "0076661a9b564772bcbe1a9c8e009f05" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40197 } } }
I20260812 08:00:20.643179  3286 raft_consensus.cc:2955] T ee8b4348af2c454bbc6666f3f101bb59 P ebce7e75217a4cf0b57083bbb9b17ec2 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 2 to 11, NON_VOTER 07dec9146dce416ebf095855fc6ff00a (127.0.186.67) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ebce7e75217a4cf0b57083bbb9b17ec2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41921 } } peers { permanent_uuid: "0076661a9b564772bcbe1a9c8e009f05" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40197 } } }
I20260812 08:00:20.646116  3125 catalog_manager.cc:5719] T ee8b4348af2c454bbc6666f3f101bb59 P 0076661a9b564772bcbe1a9c8e009f05 reported cstate change: config changed from index 2 to 11, NON_VOTER 07dec9146dce416ebf095855fc6ff00a (127.0.186.67) evicted. New cstate: current_term: 1 leader_uuid: "0076661a9b564772bcbe1a9c8e009f05" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ebce7e75217a4cf0b57083bbb9b17ec2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41921 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0076661a9b564772bcbe1a9c8e009f05" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40197 } health_report { overall_health: HEALTHY } } }
I20260812 08:00:20.656868  3569 tablet_service.cc:1558] Processing DeleteTablet for tablet ee8b4348af2c454bbc6666f3f101bb59 with delete_type TABLET_DATA_TOMBSTONED (TS 07dec9146dce416ebf095855fc6ff00a not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:52206
I20260812 08:00:20.657608  3806 tablet_replica.cc:333] T ee8b4348af2c454bbc6666f3f101bb59 P 07dec9146dce416ebf095855fc6ff00a: stopping tablet replica
I20260812 08:00:20.657956  3806 raft_consensus.cc:2243] T ee8b4348af2c454bbc6666f3f101bb59 P 07dec9146dce416ebf095855fc6ff00a [term 1 LEARNER]: Raft consensus shutting down.
I20260812 08:00:20.658288  3806 raft_consensus.cc:2272] T ee8b4348af2c454bbc6666f3f101bb59 P 07dec9146dce416ebf095855fc6ff00a [term 1 LEARNER]: Raft consensus is shut down!
I20260812 08:00:20.659900  3806 ts_tablet_manager.cc:1916] T ee8b4348af2c454bbc6666f3f101bb59 P 07dec9146dce416ebf095855fc6ff00a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 08:00:20.662062  3806 ts_tablet_manager.cc:1929] T ee8b4348af2c454bbc6666f3f101bb59 P 07dec9146dce416ebf095855fc6ff00a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.10
I20260812 08:00:20.662186  3806 log.cc:1199] T ee8b4348af2c454bbc6666f3f101bb59 P 07dec9146dce416ebf095855fc6ff00a: Deleting WAL directory at /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/ee8b4348af2c454bbc6666f3f101bb59
I20260812 08:00:20.662923  3112 catalog_manager.cc:5050] TS 07dec9146dce416ebf095855fc6ff00a (127.0.186.67:33861): tablet ee8b4348af2c454bbc6666f3f101bb59 (table TestTable [id=3315796743994665b910d92c8465e383]) successfully deleted
I20260812 08:00:20.774770  3417 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260812 08:00:20.774868  3265 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:00:20.774853  3569 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |       Address       | Status
----------------------------------+---------------------+---------
 dc34b63bd6aa412ba0220dd1a4067b4f | 127.0.186.126:39931 | HEALTHY

Unusual flags for Master:
                 Flag                  |                                                                                                Value                                                                                                 |      Tags       |         Master
---------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 allow_unsafe_replication_factor       | true                                                                                                                                                                                                 | runtime,unsafe  | all 1 server(s) checked
 catalog_manager_evict_excess_replicas | false                                                                                                                                                                                                | runtime,hidden  | all 1 server(s) checked
 ipki_ca_key_size                      | 768                                                                                                                                                                                                  | experimental    | all 1 server(s) checked
 ipki_server_key_size                  | 768                                                                                                                                                                                                  | experimental    | all 1 server(s) checked
 never_fsync                           | true                                                                                                                                                                                                 | unsafe,advanced | all 1 server(s) checked
 openssl_security_level_override       | 0                                                                                                                                                                                                    | unsafe,hidden   | all 1 server(s) checked
 rpc_reuseport                         | true                                                                                                                                                                                                 | experimental    | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports      | true                                                                                                                                                                                                 | unsafe          | all 1 server(s) checked
 server_dump_info_format               | pb                                                                                                                                                                                                   | hidden          | all 1 server(s) checked
 server_dump_info_path                 | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden          | all 1 server(s) checked
 tsk_num_rsa_bits                      | 512                                                                                                                                                                                                  | experimental    | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |       Value        |         Master
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.0.186.84:41539 | all 1 server(s) checked
 time_source         | builtin            | all 1 server(s) checked

Tablet Server Summary
               UUID               |      Address       | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
 0076661a9b564772bcbe1a9c8e009f05 | 127.0.186.66:40197 | HEALTHY | <none>   |       1        |       0
 07dec9146dce416ebf095855fc6ff00a | 127.0.186.67:33861 | HEALTHY | <none>   |       0        |       0
 ebce7e75217a4cf0b57083bbb9b17ec2 | 127.0.186.65:41921 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
               Flag               |                                                                                              Value                                                                                               |      Tags       |      Tablet Server
----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 ipki_server_key_size             | 768                                                                                                                                                                                              | experimental    | all 3 server(s) checked
 local_ip_for_outbound_sockets    | 127.0.186.65                                                                                                                                                                                     | experimental    | 127.0.186.65:41921
 local_ip_for_outbound_sockets    | 127.0.186.66                                                                                                                                                                                     | experimental    | 127.0.186.66:40197
 local_ip_for_outbound_sockets    | 127.0.186.67                                                                                                                                                                                     | experimental    | 127.0.186.67:33861
 never_fsync                      | true                                                                                                                                                                                             | unsafe,advanced | all 3 server(s) checked
 openssl_security_level_override  | 0                                                                                                                                                                                                | unsafe,hidden   | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports | true                                                                                                                                                                                             | unsafe          | all 3 server(s) checked
 server_dump_info_format          | pb                                                                                                                                                                                               | hidden          | all 3 server(s) checked
 server_dump_info_path            | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden          | 127.0.186.65:41921
 server_dump_info_path            | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden          | 127.0.186.66:40197
 server_dump_info_path            | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden          | 127.0.186.67:33861

Flags of checked categories for Tablet Server:
        Flag         |       Value        |      Tablet Server
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.0.186.84:41539 | all 3 server(s) checked
 time_source         | builtin            | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
   Name    | RF | Status  | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-----------+----+---------+---------------+---------+------------+------------------+-------------
 TestTable | 2  | HEALTHY | 1             | 1       | 0          | 0                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 0
 First Quartile | 0
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 2

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

OK
I20260812 08:00:20.806273   745 log_verifier.cc:126] Checking tablet ee8b4348af2c454bbc6666f3f101bb59
I20260812 08:00:20.822362   745 log_verifier.cc:177] Verified matching terms for 23 ops in tablet ee8b4348af2c454bbc6666f3f101bb59
I20260812 08:00:21.170324   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 3195
I20260812 08:00:21.183904   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 3348
I20260812 08:00:21.197134   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 3497
I20260812 08:00:21.212146   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 3092
2026-08-12T08:00:21Z chronyd exiting
[       OK ] RaftConsensusNonVoterITest.AddThenRemoveNonVoterReplica (3606 ms)
[ RUN      ] RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote
I20260812 08:00:21.242442   745 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:00:21.242501   745 ts_itest-base.cc:116] --------------
I20260812 08:00:21.242512   745 ts_itest-base.cc:117] 3 tablet servers
I20260812 08:00:21.242522   745 ts_itest-base.cc:118] 2 replicas per TS
I20260812 08:00:21.242530   745 ts_itest-base.cc:119] --------------
2026-08-12T08:00:21Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:00:21Z Disabled control of system clock
I20260812 08:00:21.252219   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:40423
--webserver_interface=127.0.186.126
--webserver_port=0
--builtin_ntp_servers=127.0.186.84:38065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:40423
--allow_unsafe_replication_factor=true
--catalog_manager_evict_excess_replicas=false with env {}
W20260812 08:00:21.370375  3865 flags.cc:432] Enabled unsafe flag: --allow_unsafe_replication_factor=true
W20260812 08:00:21.370661  3865 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:21.370703  3865 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:21.370759  3865 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:21.374289  3865 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:00:21.374370  3865 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:21.374397  3865 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:00:21.374419  3865 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:00:21.378453  3865 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:38065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--allow_unsafe_replication_factor=true
--catalog_manager_evict_excess_replicas=false
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:40423
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:40423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.3865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:21.379640  3865 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:21.380846  3865 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:21.386766  3871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:21.386755  3870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:21.386953  3865 server_base.cc:1061] running on GCE node
W20260812 08:00:21.386755  3874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:21.387601  3865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:21.388712  3865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:21.389874  3865 hybrid_clock.cc:648] HybridClock initialized: now 1786521621389856 us; error 35 us; skew 500 ppm
I20260812 08:00:21.391978  3865 webserver.cc:533] Webserver started at http://127.0.186.126:39229/ using document root <none> and password file <none>
I20260812 08:00:21.392664  3865 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:21.392742  3865 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:21.392977  3865 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:21.395165  3865 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "a947592fc4e14574b628a4830822d57f"
format_stamp: "Formatted at 2026-08-12 08:00:21 on dist-test-slave-49h3"
I20260812 08:00:21.395828  3865 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "a947592fc4e14574b628a4830822d57f"
format_stamp: "Formatted at 2026-08-12 08:00:21 on dist-test-slave-49h3"
I20260812 08:00:21.399968  3865 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.002s
I20260812 08:00:21.402755  3880 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:21.403971  3865 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:00:21.404120  3865 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "a947592fc4e14574b628a4830822d57f"
format_stamp: "Formatted at 2026-08-12 08:00:21 on dist-test-slave-49h3"
I20260812 08:00:21.404245  3865 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:21.425977  3865 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:21.426779  3865 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:21.426967  3865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:21.434981  3941 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:40423 every 8 connection(s)
I20260812 08:00:21.434978  3865 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:40423
I20260812 08:00:21.436195  3865 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:00:21.439230   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 3865
I20260812 08:00:21.439365   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:00:21.439500  3942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:21.445902  3942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a947592fc4e14574b628a4830822d57f: Bootstrap starting.
I20260812 08:00:21.448436  3942 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a947592fc4e14574b628a4830822d57f: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:21.449354  3942 log.cc:826] T 00000000000000000000000000000000 P a947592fc4e14574b628a4830822d57f: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:21.451399  3942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a947592fc4e14574b628a4830822d57f: No bootstrap required, opened a new log
I20260812 08:00:21.454175  3942 raft_consensus.cc:359] T 00000000000000000000000000000000 P a947592fc4e14574b628a4830822d57f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a947592fc4e14574b628a4830822d57f" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 40423 } }
I20260812 08:00:21.454442  3942 raft_consensus.cc:385] T 00000000000000000000000000000000 P a947592fc4e14574b628a4830822d57f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:21.454524  3942 raft_consensus.cc:740] T 00000000000000000000000000000000 P a947592fc4e14574b628a4830822d57f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a947592fc4e14574b628a4830822d57f, State: Initialized, Role: FOLLOWER
I20260812 08:00:21.455024  3942 consensus_queue.cc:260] T 00000000000000000000000000000000 P a947592fc4e14574b628a4830822d57f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a947592fc4e14574b628a4830822d57f" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 40423 } }
I20260812 08:00:21.455178  3942 raft_consensus.cc:399] T 00000000000000000000000000000000 P a947592fc4e14574b628a4830822d57f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:00:21.455252  3942 raft_consensus.cc:493] T 00000000000000000000000000000000 P a947592fc4e14574b628a4830822d57f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:00:21.455365  3942 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a947592fc4e14574b628a4830822d57f [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:21.456280  3942 raft_consensus.cc:515] T 00000000000000000000000000000000 P a947592fc4e14574b628a4830822d57f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a947592fc4e14574b628a4830822d57f" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 40423 } }
I20260812 08:00:21.456714  3942 leader_election.cc:304] T 00000000000000000000000000000000 P a947592fc4e14574b628a4830822d57f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a947592fc4e14574b628a4830822d57f; no voters: 
I20260812 08:00:21.457098  3942 leader_election.cc:290] T 00000000000000000000000000000000 P a947592fc4e14574b628a4830822d57f [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:00:21.457238  3947 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a947592fc4e14574b628a4830822d57f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:00:21.457484  3947 raft_consensus.cc:697] T 00000000000000000000000000000000 P a947592fc4e14574b628a4830822d57f [term 1 LEADER]: Becoming Leader. State: Replica: a947592fc4e14574b628a4830822d57f, State: Running, Role: LEADER
I20260812 08:00:21.457844  3947 consensus_queue.cc:237] T 00000000000000000000000000000000 P a947592fc4e14574b628a4830822d57f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a947592fc4e14574b628a4830822d57f" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 40423 } }
I20260812 08:00:21.458451  3942 sys_catalog.cc:565] T 00000000000000000000000000000000 P a947592fc4e14574b628a4830822d57f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:00:21.459661  3948 sys_catalog.cc:455] T 00000000000000000000000000000000 P a947592fc4e14574b628a4830822d57f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a947592fc4e14574b628a4830822d57f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a947592fc4e14574b628a4830822d57f" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 40423 } } }
I20260812 08:00:21.459684  3949 sys_catalog.cc:455] T 00000000000000000000000000000000 P a947592fc4e14574b628a4830822d57f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a947592fc4e14574b628a4830822d57f. Latest consensus state: current_term: 1 leader_uuid: "a947592fc4e14574b628a4830822d57f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a947592fc4e14574b628a4830822d57f" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 40423 } } }
I20260812 08:00:21.459897  3949 sys_catalog.cc:458] T 00000000000000000000000000000000 P a947592fc4e14574b628a4830822d57f [sys.catalog]: This master's current role is: LEADER
I20260812 08:00:21.459925  3948 sys_catalog.cc:458] T 00000000000000000000000000000000 P a947592fc4e14574b628a4830822d57f [sys.catalog]: This master's current role is: LEADER
W20260812 08:00:21.464182  3963 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a947592fc4e14574b628a4830822d57f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 08:00:21.464278  3963 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 08:00:21.464344  3956 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:00:21.465166  3956 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:00:21.472193  3956 catalog_manager.cc:1383] Generated new cluster ID: 6fd92fa9c8b84d2c9244f41d0de60699
I20260812 08:00:21.472282  3956 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:00:21.484217  3956 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:00:21.485270  3956 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:00:21.491927  3956 catalog_manager.cc:6092] T 00000000000000000000000000000000 P a947592fc4e14574b628a4830822d57f: Generated new TSK 0
I20260812 08:00:21.492794  3956 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:00:21.497650   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:0
--local_ip_for_outbound_sockets=127.0.186.65
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:40423
--builtin_ntp_servers=127.0.186.84:38065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=64
--leader_failure_max_missed_heartbeat_periods=1 with env {}
W20260812 08:00:21.622114  3966 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:21.622372  3966 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:21.622418  3966 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:21.626085  3966 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:21.626221  3966 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:00:21.630470  3966 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:38065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_max_missed_heartbeat_periods=1
--raft_heartbeat_interval_ms=64
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:40423
--never_fsync=true
--heap_profile_path=/tmp/kudu.3966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:21.631664  3966 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:21.632848  3966 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:00:21.635757  3966 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 32
W20260812 08:00:21.639355  3971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:21.639328  3972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:21.639333  3974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:21.639411  3966 server_base.cc:1061] running on GCE node
I20260812 08:00:21.639861  3966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:21.640477  3966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:21.641623  3966 hybrid_clock.cc:648] HybridClock initialized: now 1786521621641613 us; error 39 us; skew 500 ppm
I20260812 08:00:21.643355  3966 webserver.cc:533] Webserver started at http://127.0.186.65:41269/ using document root <none> and password file <none>
I20260812 08:00:21.643922  3966 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:21.643981  3966 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:21.644177  3966 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:21.645833  3966 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "3f95519b8ee347baa084e494a1933f43"
format_stamp: "Formatted at 2026-08-12 08:00:21 on dist-test-slave-49h3"
I20260812 08:00:21.646356  3966 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "3f95519b8ee347baa084e494a1933f43"
format_stamp: "Formatted at 2026-08-12 08:00:21 on dist-test-slave-49h3"
I20260812 08:00:21.650027  3966 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.001s
I20260812 08:00:21.652415  3981 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:21.653577  3966 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:00:21.653709  3966 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "3f95519b8ee347baa084e494a1933f43"
format_stamp: "Formatted at 2026-08-12 08:00:21 on dist-test-slave-49h3"
I20260812 08:00:21.653810  3966 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:21.681949  3966 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:21.682793  3966 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:21.683007  3966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:21.683709  3966 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:21.684989  3966 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:21.685048  3966 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:21.685109  3966 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:21.685133  3966 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:21.695819  3966 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:36339
I20260812 08:00:21.695861  4105 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:36339 every 8 connection(s)
I20260812 08:00:21.697134  3966 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:00:21.705868  4106 heartbeater.cc:344] Connected to a master server at 127.0.186.126:40423
I20260812 08:00:21.705947   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 3966
I20260812 08:00:21.706079   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:00:21.706158  4106 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:21.706717  4106 heartbeater.cc:507] Master 127.0.186.126:40423 requested a full tablet report, sending...
I20260812 08:00:21.708261   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.66:0
--local_ip_for_outbound_sockets=127.0.186.66
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:40423
--builtin_ntp_servers=127.0.186.84:38065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=64
--leader_failure_max_missed_heartbeat_periods=1 with env {}
I20260812 08:00:21.708284  3900 ts_manager.cc:194] Registered new tserver with Master: 3f95519b8ee347baa084e494a1933f43 (127.0.186.65:36339)
I20260812 08:00:21.709687  3900 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.65:39121
W20260812 08:00:21.829263  4109 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:21.829593  4109 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:21.829666  4109 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:21.833339  4109 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:21.833575  4109 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.66
I20260812 08:00:21.837975  4109 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:38065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_max_missed_heartbeat_periods=1
--raft_heartbeat_interval_ms=64
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:40423
--never_fsync=true
--heap_profile_path=/tmp/kudu.4109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:21.839515  4109 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:21.841162  4109 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:00:21.844914  4109 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 32
W20260812 08:00:21.849327  4114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:21.849351  4115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:21.849351  4117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:21.849702  4109 server_base.cc:1061] running on GCE node
I20260812 08:00:21.850119  4109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:21.850723  4109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:21.851890  4109 hybrid_clock.cc:648] HybridClock initialized: now 1786521621851874 us; error 26 us; skew 500 ppm
I20260812 08:00:21.854038  4109 webserver.cc:533] Webserver started at http://127.0.186.66:45279/ using document root <none> and password file <none>
I20260812 08:00:21.854735  4109 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:21.854817  4109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:21.855048  4109 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:21.857033  4109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "6e7b0e672d5e40008295c761b59dc045"
format_stamp: "Formatted at 2026-08-12 08:00:21 on dist-test-slave-49h3"
I20260812 08:00:21.857686  4109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "6e7b0e672d5e40008295c761b59dc045"
format_stamp: "Formatted at 2026-08-12 08:00:21 on dist-test-slave-49h3"
I20260812 08:00:21.862645  4109 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.001s
I20260812 08:00:21.865756  4123 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:21.867132  4109 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260812 08:00:21.867295  4109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "6e7b0e672d5e40008295c761b59dc045"
format_stamp: "Formatted at 2026-08-12 08:00:21 on dist-test-slave-49h3"
I20260812 08:00:21.867413  4109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:21.892295  4109 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:21.893132  4109 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:21.893344  4109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:21.894091  4109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:21.895486  4109 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:21.895555  4109 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:21.895628  4109 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:21.895653  4109 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:21.905787  4109 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.66:39541
I20260812 08:00:21.905805  4248 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.66:39541 every 8 connection(s)
I20260812 08:00:21.906968  4109 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:00:21.916303  4249 heartbeater.cc:344] Connected to a master server at 127.0.186.126:40423
I20260812 08:00:21.916579   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 4109
I20260812 08:00:21.916654  4249 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:21.916709   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:00:21.917362  4249 heartbeater.cc:507] Master 127.0.186.126:40423 requested a full tablet report, sending...
I20260812 08:00:21.918669  3900 ts_manager.cc:194] Registered new tserver with Master: 6e7b0e672d5e40008295c761b59dc045 (127.0.186.66:39541)
I20260812 08:00:21.919230   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.67:0
--local_ip_for_outbound_sockets=127.0.186.67
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:40423
--builtin_ntp_servers=127.0.186.84:38065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=64
--leader_failure_max_missed_heartbeat_periods=1 with env {}
I20260812 08:00:21.919271  3900 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.66:42287
W20260812 08:00:22.045233  4253 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:22.045653  4253 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:22.045737  4253 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:22.049804  4253 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:22.049965  4253 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.67
I20260812 08:00:22.054526  4253 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:38065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_max_missed_heartbeat_periods=1
--raft_heartbeat_interval_ms=64
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:40423
--never_fsync=true
--heap_profile_path=/tmp/kudu.4253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:22.056028  4253 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:22.057647  4253 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:00:22.061233  4253 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 32
W20260812 08:00:22.065469  4259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:22.065515  4262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:22.065515  4260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:22.065833  4253 server_base.cc:1061] running on GCE node
I20260812 08:00:22.066315  4253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:22.067037  4253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:22.068207  4253 hybrid_clock.cc:648] HybridClock initialized: now 1786521622068190 us; error 38 us; skew 500 ppm
I20260812 08:00:22.070377  4253 webserver.cc:533] Webserver started at http://127.0.186.67:44063/ using document root <none> and password file <none>
I20260812 08:00:22.071086  4253 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:22.071157  4253 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:22.071437  4253 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:22.073432  4253 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "dd4c13e70f154f6c8541bc5cb66af6ce"
format_stamp: "Formatted at 2026-08-12 08:00:22 on dist-test-slave-49h3"
I20260812 08:00:22.074004  4253 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "dd4c13e70f154f6c8541bc5cb66af6ce"
format_stamp: "Formatted at 2026-08-12 08:00:22 on dist-test-slave-49h3"
I20260812 08:00:22.077926  4253 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260812 08:00:22.080591  4269 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:22.081964  4253 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260812 08:00:22.082115  4253 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "dd4c13e70f154f6c8541bc5cb66af6ce"
format_stamp: "Formatted at 2026-08-12 08:00:22 on dist-test-slave-49h3"
I20260812 08:00:22.082242  4253 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:22.095176  4253 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:22.096180  4253 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:22.096407  4253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:22.097141  4253 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:22.098552  4253 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:22.098615  4253 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:22.098675  4253 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:22.098706  4253 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:22.108770  4253 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.67:34263
I20260812 08:00:22.108793  4402 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.67:34263 every 8 connection(s)
I20260812 08:00:22.110054  4253 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:00:22.117650   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 4253
I20260812 08:00:22.117780   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:00:22.120427  4403 heartbeater.cc:344] Connected to a master server at 127.0.186.126:40423
I20260812 08:00:22.120795  4403 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:22.121436  4403 heartbeater.cc:507] Master 127.0.186.126:40423 requested a full tablet report, sending...
I20260812 08:00:22.122818  3900 ts_manager.cc:194] Registered new tserver with Master: dd4c13e70f154f6c8541bc5cb66af6ce (127.0.186.67:34263)
I20260812 08:00:22.123446  3900 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.67:48697
I20260812 08:00:22.133702   745 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:00:22.144146  3900 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45070:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
I20260812 08:00:22.166146  4033 tablet_service.cc:1511] Processing CreateTablet for tablet 4b616eae379e4df49dca75d91474181e (DEFAULT_TABLE table=TestTable [id=c1452f5335c342e5b3339e84628496fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:22.166639  4175 tablet_service.cc:1511] Processing CreateTablet for tablet 4b616eae379e4df49dca75d91474181e (DEFAULT_TABLE table=TestTable [id=c1452f5335c342e5b3339e84628496fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:22.167280  4033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b616eae379e4df49dca75d91474181e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:22.167563  4175 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b616eae379e4df49dca75d91474181e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:22.172955  4421 tablet_bootstrap.cc:492] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43: Bootstrap starting.
I20260812 08:00:22.172983  4420 tablet_bootstrap.cc:492] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045: Bootstrap starting.
I20260812 08:00:22.175529  4420 tablet_bootstrap.cc:654] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:22.175529  4421 tablet_bootstrap.cc:654] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:22.176316  4421 log.cc:826] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:22.176316  4420 log.cc:826] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:22.178112  4420 tablet_bootstrap.cc:492] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045: No bootstrap required, opened a new log
I20260812 08:00:22.178117  4421 tablet_bootstrap.cc:492] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43: No bootstrap required, opened a new log
I20260812 08:00:22.178318  4421 ts_tablet_manager.cc:1403] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43: Time spent bootstrapping tablet: real 0.006s	user 0.004s	sys 0.000s
I20260812 08:00:22.178318  4420 ts_tablet_manager.cc:1403] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045: Time spent bootstrapping tablet: real 0.006s	user 0.004s	sys 0.000s
I20260812 08:00:22.180917  4421 raft_consensus.cc:359] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } }
I20260812 08:00:22.180888  4420 raft_consensus.cc:359] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } }
I20260812 08:00:22.181114  4420 raft_consensus.cc:385] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:22.181116  4421 raft_consensus.cc:385] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:22.181180  4421 raft_consensus.cc:740] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f95519b8ee347baa084e494a1933f43, State: Initialized, Role: FOLLOWER
I20260812 08:00:22.181192  4420 raft_consensus.cc:740] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e7b0e672d5e40008295c761b59dc045, State: Initialized, Role: FOLLOWER
I20260812 08:00:22.181648  4420 consensus_queue.cc:260] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } }
I20260812 08:00:22.181672  4421 consensus_queue.cc:260] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } }
I20260812 08:00:22.182358  4249 heartbeater.cc:499] Master 127.0.186.126:40423 was elected leader, sending a full tablet report...
I20260812 08:00:22.182359  4106 heartbeater.cc:499] Master 127.0.186.126:40423 was elected leader, sending a full tablet report...
I20260812 08:00:22.182591  4420 ts_tablet_manager.cc:1434] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260812 08:00:22.183627  4421 ts_tablet_manager.cc:1434] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
W20260812 08:00:22.198544  4107 tablet.cc:2406] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:00:22.223569  4425 raft_consensus.cc:493] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:00:22.223809  4425 raft_consensus.cc:515] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } }
I20260812 08:00:22.224898  4425 leader_election.cc:290] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e7b0e672d5e40008295c761b59dc045 (127.0.186.66:39541)
I20260812 08:00:22.228609  4197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b616eae379e4df49dca75d91474181e" candidate_uuid: "3f95519b8ee347baa084e494a1933f43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e7b0e672d5e40008295c761b59dc045" is_pre_election: true
I20260812 08:00:22.228927  4197 raft_consensus.cc:2468] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f95519b8ee347baa084e494a1933f43 in term 0.
I20260812 08:00:22.229820  3986 leader_election.cc:304] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 3f95519b8ee347baa084e494a1933f43, 6e7b0e672d5e40008295c761b59dc045; no voters: 
I20260812 08:00:22.230103  4425 raft_consensus.cc:2804] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:00:22.230186  4425 raft_consensus.cc:493] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:00:22.230232  4425 raft_consensus.cc:3060] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:22.231207  4425 raft_consensus.cc:515] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } }
I20260812 08:00:22.231554  4425 leader_election.cc:290] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43 [CANDIDATE]: Term 1 election: Requested vote from peers 6e7b0e672d5e40008295c761b59dc045 (127.0.186.66:39541)
I20260812 08:00:22.232041  4197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b616eae379e4df49dca75d91474181e" candidate_uuid: "3f95519b8ee347baa084e494a1933f43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e7b0e672d5e40008295c761b59dc045"
I20260812 08:00:22.232177  4197 raft_consensus.cc:3060] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:22.233644  4197 raft_consensus.cc:2468] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f95519b8ee347baa084e494a1933f43 in term 1.
I20260812 08:00:22.234045  3986 leader_election.cc:304] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 3f95519b8ee347baa084e494a1933f43, 6e7b0e672d5e40008295c761b59dc045; no voters: 
I20260812 08:00:22.234283  4425 raft_consensus.cc:2804] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:00:22.234531  4425 raft_consensus.cc:697] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43 [term 1 LEADER]: Becoming Leader. State: Replica: 3f95519b8ee347baa084e494a1933f43, State: Running, Role: LEADER
I20260812 08:00:22.234897  4425 consensus_queue.cc:237] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } }
I20260812 08:00:22.238153  3900 catalog_manager.cc:5719] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f95519b8ee347baa084e494a1933f43 (127.0.186.65). New cstate: current_term: 1 leader_uuid: "3f95519b8ee347baa084e494a1933f43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } health_report { overall_health: HEALTHY } } }
I20260812 08:00:22.262187   745 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:00:22.264118   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 3f95519b8ee347baa084e494a1933f43 to finish bootstrapping
I20260812 08:00:22.270605   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 6e7b0e672d5e40008295c761b59dc045 to finish bootstrapping
I20260812 08:00:22.285422   745 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 1.1
I20260812 08:00:22.310711  4425 consensus_queue.cc:1048] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:00:22.381289  4424 raft_consensus.cc:493] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f95519b8ee347baa084e494a1933f43)
I20260812 08:00:22.381524  4424 raft_consensus.cc:515] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } }
I20260812 08:00:22.382684  4424 leader_election.cc:290] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f95519b8ee347baa084e494a1933f43 (127.0.186.65:36339)
I20260812 08:00:22.386544  4059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b616eae379e4df49dca75d91474181e" candidate_uuid: "6e7b0e672d5e40008295c761b59dc045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f95519b8ee347baa084e494a1933f43" is_pre_election: true
I20260812 08:00:22.387430  4124 leader_election.cc:304] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 6e7b0e672d5e40008295c761b59dc045; no voters: 3f95519b8ee347baa084e494a1933f43
I20260812 08:00:22.387751  4424 raft_consensus.cc:2749] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 08:00:22.393172  4059 consensus_queue.cc:237] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } } peers { permanent_uuid: "dd4c13e70f154f6c8541bc5cb66af6ce" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 34263 } attrs { } }
I20260812 08:00:22.395339  4197 raft_consensus.cc:1275] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Refusing update from remote peer 3f95519b8ee347baa084e494a1933f43: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:00:22.395967  4427 consensus_queue.cc:1048] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:00:22.398499  4425 raft_consensus.cc:2955] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd4c13e70f154f6c8541bc5cb66af6ce (127.0.186.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } } peers { permanent_uuid: "dd4c13e70f154f6c8541bc5cb66af6ce" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 34263 } attrs { } } }
I20260812 08:00:22.399179  4197 raft_consensus.cc:2955] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd4c13e70f154f6c8541bc5cb66af6ce (127.0.186.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } } peers { permanent_uuid: "dd4c13e70f154f6c8541bc5cb66af6ce" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 34263 } attrs { } } }
I20260812 08:00:22.402210  3899 catalog_manager.cc:5719] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43 reported cstate change: config changed from index -1 to 2, NON_VOTER dd4c13e70f154f6c8541bc5cb66af6ce (127.0.186.67) added. New cstate: current_term: 1 leader_uuid: "3f95519b8ee347baa084e494a1933f43" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd4c13e70f154f6c8541bc5cb66af6ce" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 34263 } attrs { } health_report { overall_health: UNKNOWN } } }
W20260812 08:00:22.406778  3986 consensus_peers.cc:597] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43 -> Peer dd4c13e70f154f6c8541bc5cb66af6ce (127.0.186.67:34263): Couldn't send request to peer dd4c13e70f154f6c8541bc5cb66af6ce. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4b616eae379e4df49dca75d91474181e. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:00:22.469349  4439 ts_tablet_manager.cc:933] T 4b616eae379e4df49dca75d91474181e P dd4c13e70f154f6c8541bc5cb66af6ce: Initiating tablet copy from peer 3f95519b8ee347baa084e494a1933f43 (127.0.186.65:36339)
I20260812 08:00:22.470261  4439 tablet_copy_client.cc:323] T 4b616eae379e4df49dca75d91474181e P dd4c13e70f154f6c8541bc5cb66af6ce: tablet copy: Beginning tablet copy session from remote peer at address 127.0.186.65:36339
I20260812 08:00:22.474915  4079 tablet_copy_service.cc:140] P 3f95519b8ee347baa084e494a1933f43: Received BeginTabletCopySession request for tablet 4b616eae379e4df49dca75d91474181e from peer dd4c13e70f154f6c8541bc5cb66af6ce ({username='slave'} at 127.0.186.67:36137)
I20260812 08:00:22.475121  4079 tablet_copy_service.cc:161] P 3f95519b8ee347baa084e494a1933f43: Beginning new tablet copy session on tablet 4b616eae379e4df49dca75d91474181e from peer dd4c13e70f154f6c8541bc5cb66af6ce at {username='slave'} at 127.0.186.67:36137: session id = dd4c13e70f154f6c8541bc5cb66af6ce-4b616eae379e4df49dca75d91474181e
I20260812 08:00:22.476600  4079 tablet_copy_source_session.cc:215] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 08:00:22.480852  4439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b616eae379e4df49dca75d91474181e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:22.486546  4439 tablet_copy_client.cc:806] T 4b616eae379e4df49dca75d91474181e P dd4c13e70f154f6c8541bc5cb66af6ce: tablet copy: Starting download of 0 data blocks...
I20260812 08:00:22.486795  4439 tablet_copy_client.cc:670] T 4b616eae379e4df49dca75d91474181e P dd4c13e70f154f6c8541bc5cb66af6ce: tablet copy: Starting download of 1 WAL segments...
I20260812 08:00:22.489180  4439 tablet_copy_client.cc:538] T 4b616eae379e4df49dca75d91474181e P dd4c13e70f154f6c8541bc5cb66af6ce: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 08:00:22.491220  4439 tablet_bootstrap.cc:492] T 4b616eae379e4df49dca75d91474181e P dd4c13e70f154f6c8541bc5cb66af6ce: Bootstrap starting.
I20260812 08:00:22.496796  4439 log.cc:826] T 4b616eae379e4df49dca75d91474181e P dd4c13e70f154f6c8541bc5cb66af6ce: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:22.500422  4439 tablet_bootstrap.cc:492] T 4b616eae379e4df49dca75d91474181e P dd4c13e70f154f6c8541bc5cb66af6ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:00:22.500783  4439 tablet_bootstrap.cc:492] T 4b616eae379e4df49dca75d91474181e P dd4c13e70f154f6c8541bc5cb66af6ce: Bootstrap complete.
I20260812 08:00:22.501121  4439 ts_tablet_manager.cc:1403] T 4b616eae379e4df49dca75d91474181e P dd4c13e70f154f6c8541bc5cb66af6ce: Time spent bootstrapping tablet: real 0.010s	user 0.008s	sys 0.000s
I20260812 08:00:22.504020  4439 raft_consensus.cc:359] T 4b616eae379e4df49dca75d91474181e P dd4c13e70f154f6c8541bc5cb66af6ce [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } } peers { permanent_uuid: "dd4c13e70f154f6c8541bc5cb66af6ce" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 34263 } attrs { } }
I20260812 08:00:22.504294  4439 raft_consensus.cc:740] T 4b616eae379e4df49dca75d91474181e P dd4c13e70f154f6c8541bc5cb66af6ce [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd4c13e70f154f6c8541bc5cb66af6ce, State: Initialized, Role: LEARNER
I20260812 08:00:22.504629  4439 consensus_queue.cc:260] T 4b616eae379e4df49dca75d91474181e P dd4c13e70f154f6c8541bc5cb66af6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } } peers { permanent_uuid: "dd4c13e70f154f6c8541bc5cb66af6ce" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 34263 } attrs { } }
I20260812 08:00:22.505419  4403 heartbeater.cc:499] Master 127.0.186.126:40423 was elected leader, sending a full tablet report...
I20260812 08:00:22.505650  4439 ts_tablet_manager.cc:1434] T 4b616eae379e4df49dca75d91474181e P dd4c13e70f154f6c8541bc5cb66af6ce: Time spent starting tablet: real 0.004s	user 0.008s	sys 0.000s
I20260812 08:00:22.506752  4079 tablet_copy_service.cc:342] P 3f95519b8ee347baa084e494a1933f43: Request end of tablet copy session dd4c13e70f154f6c8541bc5cb66af6ce-4b616eae379e4df49dca75d91474181e received from {username='slave'} at 127.0.186.67:36137
I20260812 08:00:22.506870  4079 tablet_copy_service.cc:434] P 3f95519b8ee347baa084e494a1933f43: ending tablet copy session dd4c13e70f154f6c8541bc5cb66af6ce-4b616eae379e4df49dca75d91474181e on tablet 4b616eae379e4df49dca75d91474181e with peer dd4c13e70f154f6c8541bc5cb66af6ce
I20260812 08:00:22.582567  4424 raft_consensus.cc:493] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f95519b8ee347baa084e494a1933f43)
I20260812 08:00:22.582753  4424 raft_consensus.cc:515] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } } peers { permanent_uuid: "dd4c13e70f154f6c8541bc5cb66af6ce" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 34263 } attrs { } }
I20260812 08:00:22.583263  4424 leader_election.cc:290] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f95519b8ee347baa084e494a1933f43 (127.0.186.65:36339)
W20260812 08:00:22.659832  4124 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.186.65:36339 timed out after 0.076s (SENT)
W20260812 08:00:22.737033  4124 leader_election.cc:336] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3f95519b8ee347baa084e494a1933f43 (127.0.186.65:36339): Timed out: RequestConsensusVote RPC to 127.0.186.65:36339 timed out after 0.076s (SENT)
I20260812 08:00:22.737175  4124 leader_election.cc:304] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 6e7b0e672d5e40008295c761b59dc045; no voters: 3f95519b8ee347baa084e494a1933f43
I20260812 08:00:22.737499  4424 raft_consensus.cc:2749] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 08:00:22.818322  4424 raft_consensus.cc:493] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f95519b8ee347baa084e494a1933f43)
I20260812 08:00:22.818521  4424 raft_consensus.cc:515] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } } peers { permanent_uuid: "dd4c13e70f154f6c8541bc5cb66af6ce" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 34263 } attrs { } }
I20260812 08:00:22.819056  4424 leader_election.cc:290] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f95519b8ee347baa084e494a1933f43 (127.0.186.65:36339)
W20260812 08:00:22.976593  4124 leader_election.cc:336] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3f95519b8ee347baa084e494a1933f43 (127.0.186.65:36339): Timed out: RequestConsensusVote RPC to 127.0.186.65:36339 timed out after 0.078s (SENT)
I20260812 08:00:22.976727  4124 leader_election.cc:304] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 6e7b0e672d5e40008295c761b59dc045; no voters: 3f95519b8ee347baa084e494a1933f43
I20260812 08:00:22.977108  4424 raft_consensus.cc:2749] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 08:00:23.086265  4424 raft_consensus.cc:493] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f95519b8ee347baa084e494a1933f43)
I20260812 08:00:23.086452  4424 raft_consensus.cc:515] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } } peers { permanent_uuid: "dd4c13e70f154f6c8541bc5cb66af6ce" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 34263 } attrs { } }
I20260812 08:00:23.087018  4424 leader_election.cc:290] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f95519b8ee347baa084e494a1933f43 (127.0.186.65:36339)
W20260812 08:00:23.270673  4124 leader_election.cc:336] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3f95519b8ee347baa084e494a1933f43 (127.0.186.65:36339): Timed out: RequestConsensusVote RPC to 127.0.186.65:36339 timed out after 0.091s (SENT)
I20260812 08:00:23.270848  4124 leader_election.cc:304] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 6e7b0e672d5e40008295c761b59dc045; no voters: 3f95519b8ee347baa084e494a1933f43
I20260812 08:00:23.271126  4424 raft_consensus.cc:2749] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 08:00:23.374477  4424 raft_consensus.cc:493] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f95519b8ee347baa084e494a1933f43)
I20260812 08:00:23.374725  4424 raft_consensus.cc:515] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } } peers { permanent_uuid: "dd4c13e70f154f6c8541bc5cb66af6ce" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 34263 } attrs { } }
I20260812 08:00:23.375301  4424 leader_election.cc:290] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f95519b8ee347baa084e494a1933f43 (127.0.186.65:36339)
W20260812 08:00:23.995922  4124 leader_election.cc:336] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3f95519b8ee347baa084e494a1933f43 (127.0.186.65:36339): Timed out: RequestConsensusVote RPC to 127.0.186.65:36339 timed out after 0.309s (SENT)
I20260812 08:00:23.996044  4124 leader_election.cc:304] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 6e7b0e672d5e40008295c761b59dc045; no voters: 3f95519b8ee347baa084e494a1933f43
I20260812 08:00:23.996449  4446 raft_consensus.cc:2749] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 08:00:24.119639  4446 raft_consensus.cc:493] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f95519b8ee347baa084e494a1933f43)
I20260812 08:00:24.119841  4446 raft_consensus.cc:515] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } } peers { permanent_uuid: "dd4c13e70f154f6c8541bc5cb66af6ce" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 34263 } attrs { } }
I20260812 08:00:24.120362  4446 leader_election.cc:290] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f95519b8ee347baa084e494a1933f43 (127.0.186.65:36339)
W20260812 08:00:24.406364  4124 leader_election.cc:336] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3f95519b8ee347baa084e494a1933f43 (127.0.186.65:36339): Timed out: RequestConsensusVote RPC to 127.0.186.65:36339 timed out after 0.142s (SENT)
I20260812 08:00:24.406502  4124 leader_election.cc:304] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 6e7b0e672d5e40008295c761b59dc045; no voters: 3f95519b8ee347baa084e494a1933f43
I20260812 08:00:24.406744  4446 raft_consensus.cc:2749] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 08:00:24.892225  4446 raft_consensus.cc:493] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f95519b8ee347baa084e494a1933f43)
I20260812 08:00:24.892400  4446 raft_consensus.cc:515] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } } peers { permanent_uuid: "dd4c13e70f154f6c8541bc5cb66af6ce" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 34263 } attrs { } }
I20260812 08:00:24.892834  4446 leader_election.cc:290] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f95519b8ee347baa084e494a1933f43 (127.0.186.65:36339)
W20260812 08:00:25.437373  4124 leader_election.cc:336] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3f95519b8ee347baa084e494a1933f43 (127.0.186.65:36339): Timed out: RequestConsensusVote RPC to 127.0.186.65:36339 timed out after 0.271s (SENT)
I20260812 08:00:25.437537  4124 leader_election.cc:304] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 6e7b0e672d5e40008295c761b59dc045; no voters: 3f95519b8ee347baa084e494a1933f43
I20260812 08:00:25.437973  4450 raft_consensus.cc:2749] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 08:00:26.162078  4451 raft_consensus.cc:493] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f95519b8ee347baa084e494a1933f43)
I20260812 08:00:26.162269  4451 raft_consensus.cc:515] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } } peers { permanent_uuid: "dd4c13e70f154f6c8541bc5cb66af6ce" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 34263 } attrs { } }
I20260812 08:00:26.162742  4451 leader_election.cc:290] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f95519b8ee347baa084e494a1933f43 (127.0.186.65:36339)
W20260812 08:00:27.521245  4124 leader_election.cc:336] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3f95519b8ee347baa084e494a1933f43 (127.0.186.65:36339): Timed out: RequestConsensusVote RPC to 127.0.186.65:36339 timed out after 0.678s (SENT)
I20260812 08:00:27.521353  4124 leader_election.cc:304] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 6e7b0e672d5e40008295c761b59dc045; no voters: 3f95519b8ee347baa084e494a1933f43
I20260812 08:00:27.521736  4453 raft_consensus.cc:2749] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 08:00:28.303822  4454 raft_consensus.cc:493] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f95519b8ee347baa084e494a1933f43)
I20260812 08:00:28.303975  4454 raft_consensus.cc:515] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } } peers { permanent_uuid: "dd4c13e70f154f6c8541bc5cb66af6ce" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 34263 } attrs { } }
I20260812 08:00:28.304369  4454 leader_election.cc:290] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f95519b8ee347baa084e494a1933f43 (127.0.186.65:36339)
W20260812 08:00:28.496063  4124 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.186.65:36339 timed out after 0.191s (SENT) [suppressed 6 similar messages]
W20260812 08:00:28.688441  4124 leader_election.cc:336] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3f95519b8ee347baa084e494a1933f43 (127.0.186.65:36339): Timed out: RequestConsensusVote RPC to 127.0.186.65:36339 timed out after 0.191s (SENT)
I20260812 08:00:28.688550  4124 leader_election.cc:304] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 6e7b0e672d5e40008295c761b59dc045; no voters: 3f95519b8ee347baa084e494a1933f43
I20260812 08:00:28.688783  4454 raft_consensus.cc:2749] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 08:00:30.022347  4456 raft_consensus.cc:493] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f95519b8ee347baa084e494a1933f43)
I20260812 08:00:30.022500  4456 raft_consensus.cc:515] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } } peers { permanent_uuid: "dd4c13e70f154f6c8541bc5cb66af6ce" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 34263 } attrs { } }
I20260812 08:00:30.022895  4456 leader_election.cc:290] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f95519b8ee347baa084e494a1933f43 (127.0.186.65:36339)
I20260812 08:00:30.080964  4059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b616eae379e4df49dca75d91474181e" candidate_uuid: "6e7b0e672d5e40008295c761b59dc045" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3f95519b8ee347baa084e494a1933f43" is_pre_election: true
I20260812 08:00:30.081288  4059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b616eae379e4df49dca75d91474181e" candidate_uuid: "6e7b0e672d5e40008295c761b59dc045" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3f95519b8ee347baa084e494a1933f43" is_pre_election: true
I20260812 08:00:30.081701  4059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b616eae379e4df49dca75d91474181e" candidate_uuid: "6e7b0e672d5e40008295c761b59dc045" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3f95519b8ee347baa084e494a1933f43" is_pre_election: true
I20260812 08:00:30.081844  4035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b616eae379e4df49dca75d91474181e" candidate_uuid: "6e7b0e672d5e40008295c761b59dc045" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3f95519b8ee347baa084e494a1933f43" is_pre_election: true
I20260812 08:00:30.081890  4059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b616eae379e4df49dca75d91474181e" candidate_uuid: "6e7b0e672d5e40008295c761b59dc045" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3f95519b8ee347baa084e494a1933f43" is_pre_election: true
I20260812 08:00:30.082036  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b616eae379e4df49dca75d91474181e" candidate_uuid: "6e7b0e672d5e40008295c761b59dc045" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3f95519b8ee347baa084e494a1933f43" is_pre_election: true
I20260812 08:00:30.082319  4037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b616eae379e4df49dca75d91474181e" candidate_uuid: "6e7b0e672d5e40008295c761b59dc045" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3f95519b8ee347baa084e494a1933f43" is_pre_election: true
I20260812 08:00:30.082026  4035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b616eae379e4df49dca75d91474181e" candidate_uuid: "6e7b0e672d5e40008295c761b59dc045" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3f95519b8ee347baa084e494a1933f43" is_pre_election: true
I20260812 08:00:30.082885  4034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b616eae379e4df49dca75d91474181e" candidate_uuid: "6e7b0e672d5e40008295c761b59dc045" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3f95519b8ee347baa084e494a1933f43" is_pre_election: true
I20260812 08:00:30.083009  4038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b616eae379e4df49dca75d91474181e" candidate_uuid: "6e7b0e672d5e40008295c761b59dc045" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3f95519b8ee347baa084e494a1933f43" is_pre_election: true
I20260812 08:00:30.083101  4036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b616eae379e4df49dca75d91474181e" candidate_uuid: "6e7b0e672d5e40008295c761b59dc045" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3f95519b8ee347baa084e494a1933f43" is_pre_election: true
I20260812 08:00:30.083719  4351 raft_consensus.cc:1217] T 4b616eae379e4df49dca75d91474181e P dd4c13e70f154f6c8541bc5cb66af6ce [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260812 08:00:30.084323  4034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b616eae379e4df49dca75d91474181e" candidate_uuid: "6e7b0e672d5e40008295c761b59dc045" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3f95519b8ee347baa084e494a1933f43" is_pre_election: true
I20260812 08:00:30.084551  4036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b616eae379e4df49dca75d91474181e" candidate_uuid: "6e7b0e672d5e40008295c761b59dc045" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3f95519b8ee347baa084e494a1933f43" is_pre_election: true
I20260812 08:00:30.084604  4042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b616eae379e4df49dca75d91474181e" candidate_uuid: "6e7b0e672d5e40008295c761b59dc045" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3f95519b8ee347baa084e494a1933f43" is_pre_election: true
I20260812 08:00:30.084801  4038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b616eae379e4df49dca75d91474181e" candidate_uuid: "6e7b0e672d5e40008295c761b59dc045" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3f95519b8ee347baa084e494a1933f43" is_pre_election: true
I20260812 08:00:30.084558  4046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b616eae379e4df49dca75d91474181e" candidate_uuid: "6e7b0e672d5e40008295c761b59dc045" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3f95519b8ee347baa084e494a1933f43" is_pre_election: true
I20260812 08:00:30.082028  4059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b616eae379e4df49dca75d91474181e" candidate_uuid: "6e7b0e672d5e40008295c761b59dc045" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3f95519b8ee347baa084e494a1933f43" is_pre_election: true
I20260812 08:00:30.086906  4124 leader_election.cc:304] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 6e7b0e672d5e40008295c761b59dc045; no voters: 3f95519b8ee347baa084e494a1933f43
I20260812 08:00:30.087458  4456 raft_consensus.cc:2749] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 08:00:30.088207  4100 generic_service.cc:139] {username='slave'} at 127.0.0.1:38468 forcing change of non-runtime-safe flag enable_leader_failure_detection
I20260812 08:00:30.088296  4100 generic_service.cc:175] {username='slave'} at 127.0.0.1:38468 changed flags via RPC: enable_leader_failure_detection from 'true' to 'false'
W20260812 08:00:30.089404  4243 generic_service.cc:139] {username='slave'} at 127.0.0.1:52924 forcing change of non-runtime-safe flag enable_leader_failure_detection
I20260812 08:00:30.089522  4243 generic_service.cc:175] {username='slave'} at 127.0.0.1:52924 changed flags via RPC: enable_leader_failure_detection from 'true' to 'false'
W20260812 08:00:30.093668  4396 generic_service.cc:139] {username='slave'} at 127.0.0.1:39984 forcing change of non-runtime-safe flag enable_leader_failure_detection
I20260812 08:00:30.093755  4396 generic_service.cc:175] {username='slave'} at 127.0.0.1:39984 changed flags via RPC: enable_leader_failure_detection from 'true' to 'false'
I20260812 08:00:30.097150   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 4253
I20260812 08:00:30.109699   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.67:34263
--local_ip_for_outbound_sockets=127.0.186.67
--tserver_master_addrs=127.0.186.126:40423
--webserver_port=44063
--webserver_interface=127.0.186.67
--builtin_ntp_servers=127.0.186.84:38065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=64
--leader_failure_max_missed_heartbeat_periods=1 with env {}
I20260812 08:00:30.153237  4456 raft_consensus.cc:493] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f95519b8ee347baa084e494a1933f43)
I20260812 08:00:30.153405  4456 raft_consensus.cc:515] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } } peers { permanent_uuid: "dd4c13e70f154f6c8541bc5cb66af6ce" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 34263 } attrs { } }
I20260812 08:00:30.153864  4456 leader_election.cc:290] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f95519b8ee347baa084e494a1933f43 (127.0.186.65:36339)
I20260812 08:00:30.154444  4039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b616eae379e4df49dca75d91474181e" candidate_uuid: "6e7b0e672d5e40008295c761b59dc045" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3f95519b8ee347baa084e494a1933f43" is_pre_election: true
I20260812 08:00:30.154882  4124 leader_election.cc:304] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 6e7b0e672d5e40008295c761b59dc045; no voters: 3f95519b8ee347baa084e494a1933f43
I20260812 08:00:30.155057  4456 raft_consensus.cc:2749] T 4b616eae379e4df49dca75d91474181e P 6e7b0e672d5e40008295c761b59dc045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 08:00:30.163375  3986 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.67:34263: connect: Connection refused (error 111)
W20260812 08:00:30.164248  3986 consensus_peers.cc:597] T 4b616eae379e4df49dca75d91474181e P 3f95519b8ee347baa084e494a1933f43 -> Peer dd4c13e70f154f6c8541bc5cb66af6ce (127.0.186.67:34263): Couldn't send request to peer dd4c13e70f154f6c8541bc5cb66af6ce. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:34263: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260812 08:00:30.217881  4464 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:30.218137  4464 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:30.218187  4464 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:30.221655  4464 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:30.221779  4464 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.67
I20260812 08:00:30.225708  4464 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:38065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_max_missed_heartbeat_periods=1
--raft_heartbeat_interval_ms=64
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.67:34263
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.186.67
--webserver_port=44063
--tserver_master_addrs=127.0.186.126:40423
--never_fsync=true
--heap_profile_path=/tmp/kudu.4464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:30.226821  4464 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:30.227974  4464 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:00:30.231164  4464 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 32
W20260812 08:00:30.235095  4475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:30.235085  4472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:30.235136  4464 server_base.cc:1061] running on GCE node
W20260812 08:00:30.235085  4473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:30.235667  4464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:30.236217  4464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:30.237373  4464 hybrid_clock.cc:648] HybridClock initialized: now 1786521630237339 us; error 77 us; skew 500 ppm
I20260812 08:00:30.239203  4464 webserver.cc:533] Webserver started at http://127.0.186.67:44063/ using document root <none> and password file <none>
I20260812 08:00:30.239760  4464 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:30.239825  4464 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:30.243230  4464 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.000s	sys 0.003s
I20260812 08:00:30.245045  4482 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:30.246022  4464 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.001s
I20260812 08:00:30.246140  4464 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "dd4c13e70f154f6c8541bc5cb66af6ce"
format_stamp: "Formatted at 2026-08-12 08:00:22 on dist-test-slave-49h3"
I20260812 08:00:30.246505  4464 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:30.275169  4464 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:30.275880  4464 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:30.276057  4464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:30.276664  4464 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:30.277920  4490 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:00:30.281512  4464 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:00:30.281572  4464 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:00:30.281648  4464 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:00:30.283329  4464 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:00:30.283383  4464 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:00:30.283746  4490 tablet_bootstrap.cc:492] T 4b616eae379e4df49dca75d91474181e P dd4c13e70f154f6c8541bc5cb66af6ce: Bootstrap starting.
I20260812 08:00:30.293731  4464 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.67:34263
I20260812 08:00:30.293766  4616 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.67:34263 every 8 connection(s)
I20260812 08:00:30.294821  4464 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:00:30.295850   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 4464
I20260812 08:00:30.299981  4490 log.cc:826] T 4b616eae379e4df49dca75d91474181e P dd4c13e70f154f6c8541bc5cb66af6ce: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:30.304051  4490 tablet_bootstrap.cc:492] T 4b616eae379e4df49dca75d91474181e P dd4c13e70f154f6c8541bc5cb66af6ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:00:30.304441  4490 tablet_bootstrap.cc:492] T 4b616eae379e4df49dca75d91474181e P dd4c13e70f154f6c8541bc5cb66af6ce: Bootstrap complete.
I20260812 08:00:30.305051  4617 heartbeater.cc:344] Connected to a master server at 127.0.186.126:40423
I20260812 08:00:30.305271  4617 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:30.305792  4490 ts_tablet_manager.cc:1403] T 4b616eae379e4df49dca75d91474181e P dd4c13e70f154f6c8541bc5cb66af6ce: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.007s
I20260812 08:00:30.305832  4617 heartbeater.cc:507] Master 127.0.186.126:40423 requested a full tablet report, sending...
I20260812 08:00:30.308072  3899 ts_manager.cc:194] Re-registered known tserver with Master: dd4c13e70f154f6c8541bc5cb66af6ce (127.0.186.67:34263)
I20260812 08:00:30.308846  3899 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.67:33407
I20260812 08:00:30.308634  4490 raft_consensus.cc:359] T 4b616eae379e4df49dca75d91474181e P dd4c13e70f154f6c8541bc5cb66af6ce [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } } peers { permanent_uuid: "dd4c13e70f154f6c8541bc5cb66af6ce" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 34263 } attrs { } }
I20260812 08:00:30.309005  4490 raft_consensus.cc:740] T 4b616eae379e4df49dca75d91474181e P dd4c13e70f154f6c8541bc5cb66af6ce [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd4c13e70f154f6c8541bc5cb66af6ce, State: Initialized, Role: LEARNER
I20260812 08:00:30.309377  4490 consensus_queue.cc:260] T 4b616eae379e4df49dca75d91474181e P dd4c13e70f154f6c8541bc5cb66af6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e7b0e672d5e40008295c761b59dc045" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 39541 } } peers { permanent_uuid: "3f95519b8ee347baa084e494a1933f43" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36339 } } peers { permanent_uuid: "dd4c13e70f154f6c8541bc5cb66af6ce" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 34263 } attrs { } }
I20260812 08:00:30.310303  4490 ts_tablet_manager.cc:1434] T 4b616eae379e4df49dca75d91474181e P dd4c13e70f154f6c8541bc5cb66af6ce: Time spent starting tablet: real 0.004s	user 0.003s	sys 0.001s
I20260812 08:00:30.310673  4617 heartbeater.cc:499] Master 127.0.186.126:40423 was elected leader, sending a full tablet report...
I20260812 08:00:30.515412  4564 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4b616eae379e4df49dca75d91474181e"
dest_uuid: "dd4c13e70f154f6c8541bc5cb66af6ce"
 from {username='slave'} at 127.0.0.1:40010
I20260812 08:00:30.516723   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 3966
I20260812 08:00:30.527856   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 4109
I20260812 08:00:30.539057   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 4464
I20260812 08:00:30.549221   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 3865
2026-08-12T08:00:30Z chronyd exiting
[       OK ] RaftConsensusNonVoterITest.NonVoterReplicasDoNotVote (9333 ms)
[ RUN      ] RaftConsensusNonVoterITest.PromoteAndDemote
I20260812 08:00:30.575239   745 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:00:30.575285   745 ts_itest-base.cc:116] --------------
I20260812 08:00:30.575300   745 ts_itest-base.cc:117] 4 tablet servers
I20260812 08:00:30.575313   745 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:00:30.575325   745 ts_itest-base.cc:119] --------------
2026-08-12T08:00:30Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:00:30Z Disabled control of system clock
I20260812 08:00:30.584143   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:43507
--webserver_interface=127.0.186.126
--webserver_port=0
--builtin_ntp_servers=127.0.186.84:33477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:43507
--catalog_manager_wait_for_new_tablets_to_elect_leader=false
--catalog_manager_evict_excess_replicas=false with env {}
W20260812 08:00:30.687785  4635 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:30.688022  4635 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:30.688088  4635 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:30.691447  4635 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:00:30.691507  4635 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:30.691536  4635 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:00:30.691557  4635 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:00:30.695453  4635 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:33477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--catalog_manager_evict_excess_replicas=false
--catalog_manager_wait_for_new_tablets_to_elect_leader=false
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:43507
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:43507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.4635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:30.696615  4635 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:30.697809  4635 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:30.703227  4644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:30.703260  4642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:30.703332  4635 server_base.cc:1061] running on GCE node
W20260812 08:00:30.703279  4641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:30.703890  4635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:30.704752  4635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:30.705902  4635 hybrid_clock.cc:648] HybridClock initialized: now 1786521630705887 us; error 31 us; skew 500 ppm
I20260812 08:00:30.707657  4635 webserver.cc:533] Webserver started at http://127.0.186.126:42185/ using document root <none> and password file <none>
I20260812 08:00:30.708235  4635 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:30.708289  4635 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:30.708490  4635 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:30.710139  4635 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "7c2985dd0c2f4ac2aa304a7e389a73e5"
format_stamp: "Formatted at 2026-08-12 08:00:30 on dist-test-slave-49h3"
I20260812 08:00:30.710600  4635 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "7c2985dd0c2f4ac2aa304a7e389a73e5"
format_stamp: "Formatted at 2026-08-12 08:00:30 on dist-test-slave-49h3"
I20260812 08:00:30.713997  4635 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.001s
I20260812 08:00:30.716248  4652 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:30.717265  4635 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:00:30.717387  4635 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "7c2985dd0c2f4ac2aa304a7e389a73e5"
format_stamp: "Formatted at 2026-08-12 08:00:30 on dist-test-slave-49h3"
I20260812 08:00:30.717517  4635 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:30.746512  4635 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:30.747301  4635 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:30.747486  4635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:30.755139  4714 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:43507 every 8 connection(s)
I20260812 08:00:30.755144  4635 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:43507
I20260812 08:00:30.756273  4635 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:00:30.759413  4715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:30.759770   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 4635
I20260812 08:00:30.759868   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:00:30.764917  4715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c2985dd0c2f4ac2aa304a7e389a73e5: Bootstrap starting.
I20260812 08:00:30.767302  4715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c2985dd0c2f4ac2aa304a7e389a73e5: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:30.768002  4715 log.cc:826] T 00000000000000000000000000000000 P 7c2985dd0c2f4ac2aa304a7e389a73e5: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:30.769856  4715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c2985dd0c2f4ac2aa304a7e389a73e5: No bootstrap required, opened a new log
I20260812 08:00:30.772444  4715 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7c2985dd0c2f4ac2aa304a7e389a73e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2985dd0c2f4ac2aa304a7e389a73e5" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 43507 } }
I20260812 08:00:30.772639  4715 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7c2985dd0c2f4ac2aa304a7e389a73e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:30.772683  4715 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7c2985dd0c2f4ac2aa304a7e389a73e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c2985dd0c2f4ac2aa304a7e389a73e5, State: Initialized, Role: FOLLOWER
I20260812 08:00:30.773054  4715 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c2985dd0c2f4ac2aa304a7e389a73e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2985dd0c2f4ac2aa304a7e389a73e5" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 43507 } }
I20260812 08:00:30.773176  4715 raft_consensus.cc:399] T 00000000000000000000000000000000 P 7c2985dd0c2f4ac2aa304a7e389a73e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:00:30.773221  4715 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7c2985dd0c2f4ac2aa304a7e389a73e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:00:30.773299  4715 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7c2985dd0c2f4ac2aa304a7e389a73e5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:30.774091  4715 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7c2985dd0c2f4ac2aa304a7e389a73e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2985dd0c2f4ac2aa304a7e389a73e5" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 43507 } }
I20260812 08:00:30.774397  4715 leader_election.cc:304] T 00000000000000000000000000000000 P 7c2985dd0c2f4ac2aa304a7e389a73e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c2985dd0c2f4ac2aa304a7e389a73e5; no voters: 
I20260812 08:00:30.774657  4715 leader_election.cc:290] T 00000000000000000000000000000000 P 7c2985dd0c2f4ac2aa304a7e389a73e5 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:00:30.774784  4720 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7c2985dd0c2f4ac2aa304a7e389a73e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:00:30.774981  4720 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7c2985dd0c2f4ac2aa304a7e389a73e5 [term 1 LEADER]: Becoming Leader. State: Replica: 7c2985dd0c2f4ac2aa304a7e389a73e5, State: Running, Role: LEADER
I20260812 08:00:30.775388  4720 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c2985dd0c2f4ac2aa304a7e389a73e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2985dd0c2f4ac2aa304a7e389a73e5" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 43507 } }
I20260812 08:00:30.775815  4715 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7c2985dd0c2f4ac2aa304a7e389a73e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:00:30.776948  4724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c2985dd0c2f4ac2aa304a7e389a73e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c2985dd0c2f4ac2aa304a7e389a73e5. Latest consensus state: current_term: 1 leader_uuid: "7c2985dd0c2f4ac2aa304a7e389a73e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2985dd0c2f4ac2aa304a7e389a73e5" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 43507 } } }
I20260812 08:00:30.777070  4724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c2985dd0c2f4ac2aa304a7e389a73e5 [sys.catalog]: This master's current role is: LEADER
I20260812 08:00:30.777387  4723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c2985dd0c2f4ac2aa304a7e389a73e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c2985dd0c2f4ac2aa304a7e389a73e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2985dd0c2f4ac2aa304a7e389a73e5" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 43507 } } }
I20260812 08:00:30.777549  4723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c2985dd0c2f4ac2aa304a7e389a73e5 [sys.catalog]: This master's current role is: LEADER
I20260812 08:00:30.777931  4731 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:00:30.780572  4731 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:00:30.785107  4731 catalog_manager.cc:1383] Generated new cluster ID: e3576ae4538b4f3980556d8d56c3d4a9
I20260812 08:00:30.785174  4731 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:00:30.792224  4731 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:00:30.793404  4731 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:00:30.806313  4731 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 7c2985dd0c2f4ac2aa304a7e389a73e5: Generated new TSK 0
I20260812 08:00:30.806907  4731 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:00:30.810781   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:0
--local_ip_for_outbound_sockets=127.0.186.65
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:43507
--builtin_ntp_servers=127.0.186.84:33477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=50
--evict_failed_followers=false
--enable_leader_failure_detection=false with env {}
W20260812 08:00:30.917519  4742 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 08:00:30.917743  4742 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:30.917765  4742 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:30.917825  4742 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:30.921073  4742 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:30.921181  4742 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:00:30.925024  4742 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:33477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--evict_failed_followers=false
--raft_heartbeat_interval_ms=50
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:43507
--never_fsync=true
--heap_profile_path=/tmp/kudu.4742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:30.926221  4742 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:30.927402  4742 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:00:30.930320  4742 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 25
W20260812 08:00:30.933930  4749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:30.933987  4748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:30.934026  4751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:30.934520  4742 server_base.cc:1061] running on GCE node
I20260812 08:00:30.934881  4742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:30.935432  4742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:30.936604  4742 hybrid_clock.cc:648] HybridClock initialized: now 1786521630936584 us; error 34 us; skew 500 ppm
I20260812 08:00:30.938313  4742 webserver.cc:533] Webserver started at http://127.0.186.65:46301/ using document root <none> and password file <none>
I20260812 08:00:30.938879  4742 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:30.938932  4742 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:30.939113  4742 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:30.940678  4742 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac"
format_stamp: "Formatted at 2026-08-12 08:00:30 on dist-test-slave-49h3"
I20260812 08:00:30.941169  4742 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac"
format_stamp: "Formatted at 2026-08-12 08:00:30 on dist-test-slave-49h3"
I20260812 08:00:30.944737  4742 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:00:30.947018  4757 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:30.948000  4742 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:00:30.948103  4742 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac"
format_stamp: "Formatted at 2026-08-12 08:00:30 on dist-test-slave-49h3"
I20260812 08:00:30.948201  4742 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:30.962213  4742 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:30.962833  4742 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:30.962999  4742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:30.963572  4742 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:30.964586  4742 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:30.964635  4742 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:30.964675  4742 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:30.964711  4742 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:30.974437  4742 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:35701
I20260812 08:00:30.974519  4886 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:35701 every 8 connection(s)
I20260812 08:00:30.975556  4742 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:00:30.977253   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 4742
I20260812 08:00:30.977361   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:00:30.979833   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.66:0
--local_ip_for_outbound_sockets=127.0.186.66
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:43507
--builtin_ntp_servers=127.0.186.84:33477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=50
--evict_failed_followers=false
--enable_leader_failure_detection=false with env {}
I20260812 08:00:30.986076  4887 heartbeater.cc:344] Connected to a master server at 127.0.186.126:43507
I20260812 08:00:30.986346  4887 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:30.986917  4887 heartbeater.cc:507] Master 127.0.186.126:43507 requested a full tablet report, sending...
I20260812 08:00:30.988729  4673 ts_manager.cc:194] Registered new tserver with Master: 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65:35701)
I20260812 08:00:30.990716  4673 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.65:39615
W20260812 08:00:31.090232  4891 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 08:00:31.090444  4891 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:31.090466  4891 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:31.090503  4891 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:31.093699  4891 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:31.093789  4891 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.66
I20260812 08:00:31.097525  4891 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:33477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--evict_failed_followers=false
--raft_heartbeat_interval_ms=50
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:43507
--never_fsync=true
--heap_profile_path=/tmp/kudu.4891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:31.098577  4891 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:31.099658  4891 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:00:31.102476  4891 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 25
W20260812 08:00:31.106184  4901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:31.106184  4899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:31.106271  4891 server_base.cc:1061] running on GCE node
W20260812 08:00:31.106204  4898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:31.106830  4891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:31.107365  4891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:31.108507  4891 hybrid_clock.cc:648] HybridClock initialized: now 1786521631108493 us; error 32 us; skew 500 ppm
I20260812 08:00:31.110163  4891 webserver.cc:533] Webserver started at http://127.0.186.66:32809/ using document root <none> and password file <none>
I20260812 08:00:31.110713  4891 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:31.110766  4891 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:31.110951  4891 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:31.112490  4891 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "c63354224ecd4288b891afbd82a3da21"
format_stamp: "Formatted at 2026-08-12 08:00:31 on dist-test-slave-49h3"
I20260812 08:00:31.113154  4891 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "c63354224ecd4288b891afbd82a3da21"
format_stamp: "Formatted at 2026-08-12 08:00:31 on dist-test-slave-49h3"
I20260812 08:00:31.116380  4891 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.000s
I20260812 08:00:31.118414  4907 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:31.119364  4891 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:00:31.119468  4891 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "c63354224ecd4288b891afbd82a3da21"
format_stamp: "Formatted at 2026-08-12 08:00:31 on dist-test-slave-49h3"
I20260812 08:00:31.119546  4891 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:31.136955  4891 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:31.137645  4891 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:31.137811  4891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:31.138429  4891 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:31.139499  4891 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:31.139549  4891 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:31.139604  4891 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:31.139675  4891 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:31.148815  4891 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.66:40797
I20260812 08:00:31.148870  5037 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.66:40797 every 8 connection(s)
I20260812 08:00:31.149827  4891 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:00:31.157526   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 4891
I20260812 08:00:31.157651   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:00:31.158502  5038 heartbeater.cc:344] Connected to a master server at 127.0.186.126:43507
I20260812 08:00:31.158759  5038 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:31.159502  5038 heartbeater.cc:507] Master 127.0.186.126:43507 requested a full tablet report, sending...
I20260812 08:00:31.159951   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.67:0
--local_ip_for_outbound_sockets=127.0.186.67
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:43507
--builtin_ntp_servers=127.0.186.84:33477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=50
--evict_failed_followers=false
--enable_leader_failure_detection=false with env {}
I20260812 08:00:31.160609  4673 ts_manager.cc:194] Registered new tserver with Master: c63354224ecd4288b891afbd82a3da21 (127.0.186.66:40797)
I20260812 08:00:31.161235  4673 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.66:51387
W20260812 08:00:31.268850  5041 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 08:00:31.269099  5041 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:31.269140  5041 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:31.269201  5041 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:31.272512  5041 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:31.272639  5041 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.67
I20260812 08:00:31.276594  5041 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:33477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--evict_failed_followers=false
--raft_heartbeat_interval_ms=50
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:43507
--never_fsync=true
--heap_profile_path=/tmp/kudu.5041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:31.277885  5041 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:31.279066  5041 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:00:31.281978  5041 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 25
W20260812 08:00:31.285655  5048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:31.285665  5050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:31.285748  5046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:31.285893  5041 server_base.cc:1061] running on GCE node
I20260812 08:00:31.286262  5041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:31.286791  5041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:31.287940  5041 hybrid_clock.cc:648] HybridClock initialized: now 1786521631287916 us; error 34 us; skew 500 ppm
I20260812 08:00:31.289736  5041 webserver.cc:533] Webserver started at http://127.0.186.67:38295/ using document root <none> and password file <none>
I20260812 08:00:31.290297  5041 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:31.290354  5041 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:31.290531  5041 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:31.292047  5041 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "891acef050c7407a852c2416b86af163"
format_stamp: "Formatted at 2026-08-12 08:00:31 on dist-test-slave-49h3"
I20260812 08:00:31.292513  5041 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "891acef050c7407a852c2416b86af163"
format_stamp: "Formatted at 2026-08-12 08:00:31 on dist-test-slave-49h3"
I20260812 08:00:31.295732  5041 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.003s
I20260812 08:00:31.297812  5056 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:31.298763  5041 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:00:31.298875  5041 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "891acef050c7407a852c2416b86af163"
format_stamp: "Formatted at 2026-08-12 08:00:31 on dist-test-slave-49h3"
I20260812 08:00:31.298959  5041 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:31.316298  5041 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:31.317004  5041 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:31.317191  5041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:31.317843  5041 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:31.318926  5041 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:31.318977  5041 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:31.319031  5041 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:31.319056  5041 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:31.328244  5041 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.67:41995
I20260812 08:00:31.328279  5187 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.67:41995 every 8 connection(s)
I20260812 08:00:31.329245  5041 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:00:31.337628  5188 heartbeater.cc:344] Connected to a master server at 127.0.186.126:43507
I20260812 08:00:31.337723   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 5041
I20260812 08:00:31.337827   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:00:31.337905  5188 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:31.338506  5188 heartbeater.cc:507] Master 127.0.186.126:43507 requested a full tablet report, sending...
I20260812 08:00:31.339557  4673 ts_manager.cc:194] Registered new tserver with Master: 891acef050c7407a852c2416b86af163 (127.0.186.67:41995)
I20260812 08:00:31.340035   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.68:0
--local_ip_for_outbound_sockets=127.0.186.68
--webserver_interface=127.0.186.68
--webserver_port=0
--tserver_master_addrs=127.0.186.126:43507
--builtin_ntp_servers=127.0.186.84:33477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=50
--evict_failed_followers=false
--enable_leader_failure_detection=false with env {}
I20260812 08:00:31.340113  4673 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.67:56791
W20260812 08:00:31.447273  5192 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 08:00:31.447510  5192 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:31.447530  5192 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:31.447576  5192 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:31.450948  5192 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:31.451061  5192 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.68
I20260812 08:00:31.454896  5192 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:33477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--evict_failed_followers=false
--raft_heartbeat_interval_ms=50
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.0.186.68
--webserver_port=0
--tserver_master_addrs=127.0.186.126:43507
--never_fsync=true
--heap_profile_path=/tmp/kudu.5192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.68

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:31.455993  5192 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:31.457127  5192 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:00:31.459997  5192 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 25
W20260812 08:00:31.463570  5197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:31.463570  5200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:31.463579  5198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:31.463872  5192 server_base.cc:1061] running on GCE node
I20260812 08:00:31.464252  5192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:31.464771  5192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:31.465916  5192 hybrid_clock.cc:648] HybridClock initialized: now 1786521631465901 us; error 30 us; skew 500 ppm
I20260812 08:00:31.467638  5192 webserver.cc:533] Webserver started at http://127.0.186.68:43211/ using document root <none> and password file <none>
I20260812 08:00:31.468190  5192 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:31.468248  5192 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:31.468427  5192 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:31.470018  5192 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "100406046c54412097105da287cb4797"
format_stamp: "Formatted at 2026-08-12 08:00:31 on dist-test-slave-49h3"
I20260812 08:00:31.470479  5192 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "100406046c54412097105da287cb4797"
format_stamp: "Formatted at 2026-08-12 08:00:31 on dist-test-slave-49h3"
I20260812 08:00:31.473748  5192 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:00:31.475955  5208 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:31.477010  5192 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:00:31.477123  5192 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "100406046c54412097105da287cb4797"
format_stamp: "Formatted at 2026-08-12 08:00:31 on dist-test-slave-49h3"
I20260812 08:00:31.477211  5192 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:31.491623  5192 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:31.492280  5192 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:31.492448  5192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:31.493019  5192 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:31.494107  5192 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:31.494158  5192 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:31.494211  5192 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:31.494238  5192 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:31.503160  5192 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.68:40325
I20260812 08:00:31.503204  5342 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.68:40325 every 8 connection(s)
I20260812 08:00:31.504249  5192 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20260812 08:00:31.506611   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 5192
I20260812 08:00:31.506722   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20260812 08:00:31.514087  5343 heartbeater.cc:344] Connected to a master server at 127.0.186.126:43507
I20260812 08:00:31.514354  5343 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:31.514914  5343 heartbeater.cc:507] Master 127.0.186.126:43507 requested a full tablet report, sending...
I20260812 08:00:31.516002  4673 ts_manager.cc:194] Registered new tserver with Master: 100406046c54412097105da287cb4797 (127.0.186.68:40325)
I20260812 08:00:31.516549  4673 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.68:56625
I20260812 08:00:31.521421   745 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20260812 08:00:31.530131  4673 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53004:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
I20260812 08:00:31.548975  5115 tablet_service.cc:1511] Processing CreateTablet for tablet 77237ae0d0ea471193508c0773fac8c7 (DEFAULT_TABLE table=TestTable [id=3dcf2b5e1861491cb607ee1d948003fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:31.548975  5264 tablet_service.cc:1511] Processing CreateTablet for tablet 77237ae0d0ea471193508c0773fac8c7 (DEFAULT_TABLE table=TestTable [id=3dcf2b5e1861491cb607ee1d948003fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:31.550005  5115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77237ae0d0ea471193508c0773fac8c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:31.550005  5264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77237ae0d0ea471193508c0773fac8c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:31.555060  5362 tablet_bootstrap.cc:492] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163: Bootstrap starting.
I20260812 08:00:31.554977  4961 tablet_service.cc:1511] Processing CreateTablet for tablet 77237ae0d0ea471193508c0773fac8c7 (DEFAULT_TABLE table=TestTable [id=3dcf2b5e1861491cb607ee1d948003fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:31.556033  4961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77237ae0d0ea471193508c0773fac8c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:31.556209  5365 tablet_bootstrap.cc:492] T 77237ae0d0ea471193508c0773fac8c7 P 100406046c54412097105da287cb4797: Bootstrap starting.
I20260812 08:00:31.557158  5362 tablet_bootstrap.cc:654] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:31.558027  5362 log.cc:826] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:31.558338  5365 tablet_bootstrap.cc:654] T 77237ae0d0ea471193508c0773fac8c7 P 100406046c54412097105da287cb4797: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:31.559155  5365 log.cc:826] T 77237ae0d0ea471193508c0773fac8c7 P 100406046c54412097105da287cb4797: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:31.559620  5362 tablet_bootstrap.cc:492] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163: No bootstrap required, opened a new log
I20260812 08:00:31.559827  5362 ts_tablet_manager.cc:1403] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:00:31.560891  5365 tablet_bootstrap.cc:492] T 77237ae0d0ea471193508c0773fac8c7 P 100406046c54412097105da287cb4797: No bootstrap required, opened a new log
I20260812 08:00:31.561079  5365 ts_tablet_manager.cc:1403] T 77237ae0d0ea471193508c0773fac8c7 P 100406046c54412097105da287cb4797: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:00:31.561903  5368 tablet_bootstrap.cc:492] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21: Bootstrap starting.
I20260812 08:00:31.562490  5362 raft_consensus.cc:359] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } }
I20260812 08:00:31.562724  5362 raft_consensus.cc:740] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 891acef050c7407a852c2416b86af163, State: Initialized, Role: FOLLOWER
I20260812 08:00:31.563077  5362 consensus_queue.cc:260] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } }
I20260812 08:00:31.563975  5362 ts_tablet_manager.cc:1434] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:00:31.563930  5365 raft_consensus.cc:359] T 77237ae0d0ea471193508c0773fac8c7 P 100406046c54412097105da287cb4797 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } }
I20260812 08:00:31.564175  5368 tablet_bootstrap.cc:654] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:31.564175  5365 raft_consensus.cc:740] T 77237ae0d0ea471193508c0773fac8c7 P 100406046c54412097105da287cb4797 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 100406046c54412097105da287cb4797, State: Initialized, Role: FOLLOWER
I20260812 08:00:31.564360  5188 heartbeater.cc:499] Master 127.0.186.126:43507 was elected leader, sending a full tablet report...
I20260812 08:00:31.564453  5365 consensus_queue.cc:260] T 77237ae0d0ea471193508c0773fac8c7 P 100406046c54412097105da287cb4797 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } }
I20260812 08:00:31.565091  5343 heartbeater.cc:499] Master 127.0.186.126:43507 was elected leader, sending a full tablet report...
I20260812 08:00:31.565251  5365 ts_tablet_manager.cc:1434] T 77237ae0d0ea471193508c0773fac8c7 P 100406046c54412097105da287cb4797: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:00:31.566332  5368 log.cc:826] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:31.568039  5368 tablet_bootstrap.cc:492] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21: No bootstrap required, opened a new log
I20260812 08:00:31.568256  5368 ts_tablet_manager.cc:1403] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21: Time spent bootstrapping tablet: real 0.007s	user 0.002s	sys 0.002s
I20260812 08:00:31.571344  5368 raft_consensus.cc:359] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } }
I20260812 08:00:31.571698  5368 raft_consensus.cc:740] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c63354224ecd4288b891afbd82a3da21, State: Initialized, Role: FOLLOWER
I20260812 08:00:31.572032  5368 consensus_queue.cc:260] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } }
I20260812 08:00:31.572745  5038 heartbeater.cc:499] Master 127.0.186.126:43507 was elected leader, sending a full tablet report...
I20260812 08:00:31.572902  5368 ts_tablet_manager.cc:1434] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.002s
I20260812 08:00:31.575356   745 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20260812 08:00:31.576882   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver c63354224ecd4288b891afbd82a3da21 to finish bootstrapping
W20260812 08:00:31.580147  5190 tablet.cc:2406] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:00:31.583003   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 891acef050c7407a852c2416b86af163 to finish bootstrapping
I20260812 08:00:31.588294   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 100406046c54412097105da287cb4797 to finish bootstrapping
I20260812 08:00:31.596531  5137 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "77237ae0d0ea471193508c0773fac8c7"
dest_uuid: "891acef050c7407a852c2416b86af163"
 from {username='slave'} at 127.0.0.1:34794
I20260812 08:00:31.596756  5137 raft_consensus.cc:493] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 08:00:31.596853  5137 raft_consensus.cc:3060] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:31.597816  5137 raft_consensus.cc:515] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } }
I20260812 08:00:31.598877  5137 leader_election.cc:290] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [CANDIDATE]: Term 1 election: Requested vote from peers 100406046c54412097105da287cb4797 (127.0.186.68:40325), c63354224ecd4288b891afbd82a3da21 (127.0.186.66:40797)
I20260812 08:00:31.603255  5287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77237ae0d0ea471193508c0773fac8c7" candidate_uuid: "891acef050c7407a852c2416b86af163" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "100406046c54412097105da287cb4797"
I20260812 08:00:31.603255  4985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77237ae0d0ea471193508c0773fac8c7" candidate_uuid: "891acef050c7407a852c2416b86af163" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c63354224ecd4288b891afbd82a3da21"
I20260812 08:00:31.603502  5287 raft_consensus.cc:3060] T 77237ae0d0ea471193508c0773fac8c7 P 100406046c54412097105da287cb4797 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:31.603502  4985 raft_consensus.cc:3060] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:31.604686  5287 raft_consensus.cc:2468] T 77237ae0d0ea471193508c0773fac8c7 P 100406046c54412097105da287cb4797 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 891acef050c7407a852c2416b86af163 in term 1.
I20260812 08:00:31.604686  4985 raft_consensus.cc:2468] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 891acef050c7407a852c2416b86af163 in term 1.
I20260812 08:00:31.605216  5057 leader_election.cc:304] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 891acef050c7407a852c2416b86af163, c63354224ecd4288b891afbd82a3da21; no voters: 
I20260812 08:00:31.605513  5369 raft_consensus.cc:2804] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:00:31.605765  5369 raft_consensus.cc:697] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [term 1 LEADER]: Becoming Leader. State: Replica: 891acef050c7407a852c2416b86af163, State: Running, Role: LEADER
I20260812 08:00:31.606134  5369 consensus_queue.cc:237] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } }
I20260812 08:00:31.609862  4673 catalog_manager.cc:5719] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 reported cstate change: term changed from 0 to 1, leader changed from <none> to 891acef050c7407a852c2416b86af163 (127.0.186.67). New cstate: current_term: 1 leader_uuid: "891acef050c7407a852c2416b86af163" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } health_report { overall_health: HEALTHY } } }
I20260812 08:00:31.618155   745 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 1.1 0.0
I20260812 08:00:31.646939  5369 consensus_queue.cc:1048] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Connected to new peer: Peer: permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 08:00:31.650926  5039 tablet.cc:2406] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:00:31.652910  5369 consensus_queue.cc:1048] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:00:31.725375  5137 consensus_queue.cc:237] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } }
I20260812 08:00:31.727308  4985 raft_consensus.cc:1275] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21 [term 1 FOLLOWER]: Refusing update from remote peer 891acef050c7407a852c2416b86af163: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:00:31.727308  5287 raft_consensus.cc:1275] T 77237ae0d0ea471193508c0773fac8c7 P 100406046c54412097105da287cb4797 [term 1 FOLLOWER]: Refusing update from remote peer 891acef050c7407a852c2416b86af163: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:00:31.727793  5380 consensus_queue.cc:1048] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Connected to new peer: Peer: permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:00:31.728011  5369 consensus_queue.cc:1048] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:00:31.730100  5380 raft_consensus.cc:2955] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } } }
I20260812 08:00:31.730577  5287 raft_consensus.cc:2955] T 77237ae0d0ea471193508c0773fac8c7 P 100406046c54412097105da287cb4797 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } } }
I20260812 08:00:31.731802  4985 raft_consensus.cc:2955] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } } }
I20260812 08:00:31.734025  4670 catalog_manager.cc:5719] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 reported cstate change: config changed from index -1 to 2, NON_VOTER 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) added. New cstate: current_term: 1 leader_uuid: "891acef050c7407a852c2416b86af163" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } health_report { overall_health: UNKNOWN } } }
W20260812 08:00:31.734598  5059 consensus_peers.cc:597] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 -> Peer 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65:35701): Couldn't send request to peer 1ef10e8ced0d4cc98922aeecd5a4b5ac. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77237ae0d0ea471193508c0773fac8c7. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:00:31.776562  5385 ts_tablet_manager.cc:933] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac: Initiating tablet copy from peer 891acef050c7407a852c2416b86af163 (127.0.186.67:41995)
I20260812 08:00:31.777240  5385 tablet_copy_client.cc:323] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac: tablet copy: Beginning tablet copy session from remote peer at address 127.0.186.67:41995
I20260812 08:00:31.780910  5159 tablet_copy_service.cc:140] P 891acef050c7407a852c2416b86af163: Received BeginTabletCopySession request for tablet 77237ae0d0ea471193508c0773fac8c7 from peer 1ef10e8ced0d4cc98922aeecd5a4b5ac ({username='slave'} at 127.0.186.65:43983)
I20260812 08:00:31.781039  5159 tablet_copy_service.cc:161] P 891acef050c7407a852c2416b86af163: Beginning new tablet copy session on tablet 77237ae0d0ea471193508c0773fac8c7 from peer 1ef10e8ced0d4cc98922aeecd5a4b5ac at {username='slave'} at 127.0.186.65:43983: session id = 1ef10e8ced0d4cc98922aeecd5a4b5ac-77237ae0d0ea471193508c0773fac8c7
I20260812 08:00:31.782333  5159 tablet_copy_source_session.cc:215] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 08:00:31.785876  5385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77237ae0d0ea471193508c0773fac8c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:31.790563  5385 tablet_copy_client.cc:806] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac: tablet copy: Starting download of 0 data blocks...
I20260812 08:00:31.790771  5385 tablet_copy_client.cc:670] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac: tablet copy: Starting download of 1 WAL segments...
I20260812 08:00:31.792536  5385 tablet_copy_client.cc:538] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 08:00:31.794313  5385 tablet_bootstrap.cc:492] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac: Bootstrap starting.
I20260812 08:00:31.798645  5385 log.cc:826] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:31.800822  5385 tablet_bootstrap.cc:492] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:00:31.801041  5385 tablet_bootstrap.cc:492] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac: Bootstrap complete.
I20260812 08:00:31.801276  5385 ts_tablet_manager.cc:1403] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac: Time spent bootstrapping tablet: real 0.007s	user 0.005s	sys 0.000s
I20260812 08:00:31.803544  5385 raft_consensus.cc:359] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } }
I20260812 08:00:31.803753  5385 raft_consensus.cc:740] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1ef10e8ced0d4cc98922aeecd5a4b5ac, State: Initialized, Role: LEARNER
I20260812 08:00:31.804001  5385 consensus_queue.cc:260] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } }
I20260812 08:00:31.804580  4887 heartbeater.cc:499] Master 127.0.186.126:43507 was elected leader, sending a full tablet report...
I20260812 08:00:31.804702  5385 ts_tablet_manager.cc:1434] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac: Time spent starting tablet: real 0.003s	user 0.002s	sys 0.003s
I20260812 08:00:31.805475  5159 tablet_copy_service.cc:342] P 891acef050c7407a852c2416b86af163: Request end of tablet copy session 1ef10e8ced0d4cc98922aeecd5a4b5ac-77237ae0d0ea471193508c0773fac8c7 received from {username='slave'} at 127.0.186.65:43983
I20260812 08:00:31.805590  5159 tablet_copy_service.cc:434] P 891acef050c7407a852c2416b86af163: ending tablet copy session 1ef10e8ced0d4cc98922aeecd5a4b5ac-77237ae0d0ea471193508c0773fac8c7 on tablet 77237ae0d0ea471193508c0773fac8c7 with peer 1ef10e8ced0d4cc98922aeecd5a4b5ac
I20260812 08:00:31.813901  5137 consensus_queue.cc:237] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } }
I20260812 08:00:31.815414  5287 raft_consensus.cc:1275] T 77237ae0d0ea471193508c0773fac8c7 P 100406046c54412097105da287cb4797 [term 1 FOLLOWER]: Refusing update from remote peer 891acef050c7407a852c2416b86af163: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 08:00:31.815443  4985 raft_consensus.cc:1275] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21 [term 1 FOLLOWER]: Refusing update from remote peer 891acef050c7407a852c2416b86af163: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 08:00:31.815584  4835 raft_consensus.cc:1275] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac [term 1 LEARNER]: Refusing update from remote peer 891acef050c7407a852c2416b86af163: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 08:00:31.815829  5376 consensus_queue.cc:1048] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Connected to new peer: Peer: permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 08:00:31.816061  5369 consensus_queue.cc:1048] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 08:00:31.816223  5380 consensus_queue.cc:1048] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 08:00:31.818706  5376 raft_consensus.cc:2955] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } } }
I20260812 08:00:31.819511  4835 raft_consensus.cc:2955] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } } }
I20260812 08:00:31.820340  5287 raft_consensus.cc:2955] T 77237ae0d0ea471193508c0773fac8c7 P 100406046c54412097105da287cb4797 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } } }
I20260812 08:00:31.820556  4985 raft_consensus.cc:2955] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } } }
I20260812 08:00:31.822417  4673 catalog_manager.cc:5719] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 reported cstate change: config changed from index 2 to 3, 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "891acef050c7407a852c2416b86af163" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } health_report { overall_health: HEALTHY } } }
I20260812 08:00:31.838802  5137 consensus_queue.cc:237] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } }
I20260812 08:00:31.840039  4985 raft_consensus.cc:1275] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21 [term 1 FOLLOWER]: Refusing update from remote peer 891acef050c7407a852c2416b86af163: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 08:00:31.840060  5287 raft_consensus.cc:1275] T 77237ae0d0ea471193508c0773fac8c7 P 100406046c54412097105da287cb4797 [term 1 FOLLOWER]: Refusing update from remote peer 891acef050c7407a852c2416b86af163: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 08:00:31.840060  4835 raft_consensus.cc:1275] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac [term 1 FOLLOWER]: Refusing update from remote peer 891acef050c7407a852c2416b86af163: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 08:00:31.840435  5369 consensus_queue.cc:1048] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 08:00:31.840627  5380 consensus_queue.cc:1048] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 08:00:31.840796  5376 consensus_queue.cc:1048] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Connected to new peer: Peer: permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 08:00:31.842381  5369 raft_consensus.cc:2955] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) changed from VOTER to NON_VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } } }
I20260812 08:00:31.842792  4835 raft_consensus.cc:2955] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac [term 1 LEARNER]: Committing config change with OpId 1.4: config changed from index 3 to 4, 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) changed from VOTER to NON_VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } } }
I20260812 08:00:31.842798  5287 raft_consensus.cc:2955] T 77237ae0d0ea471193508c0773fac8c7 P 100406046c54412097105da287cb4797 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) changed from VOTER to NON_VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } } }
I20260812 08:00:31.843621  4984 raft_consensus.cc:2955] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) changed from VOTER to NON_VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } } }
I20260812 08:00:31.845244  4672 catalog_manager.cc:5719] T 77237ae0d0ea471193508c0773fac8c7 P 100406046c54412097105da287cb4797 reported cstate change: config changed from index 3 to 4, 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) changed from VOTER to NON_VOTER. New cstate: current_term: 1 leader_uuid: "891acef050c7407a852c2416b86af163" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } } }
I20260812 08:00:31.854391   745 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
I20260812 08:00:31.900345  5137 consensus_queue.cc:237] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } }
I20260812 08:00:31.902297  4985 raft_consensus.cc:1275] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21 [term 1 FOLLOWER]: Refusing update from remote peer 891acef050c7407a852c2416b86af163: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260812 08:00:31.902783  5380 consensus_queue.cc:1048] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 08:00:31.904027  4835 raft_consensus.cc:1275] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac [term 1 LEARNER]: Refusing update from remote peer 891acef050c7407a852c2416b86af163: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260812 08:00:31.904450  5380 consensus_queue.cc:1048] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 08:00:31.904595  5287 raft_consensus.cc:1275] T 77237ae0d0ea471193508c0773fac8c7 P 100406046c54412097105da287cb4797 [term 1 FOLLOWER]: Refusing update from remote peer 891acef050c7407a852c2416b86af163: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260812 08:00:31.905093  5426 consensus_queue.cc:1048] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Connected to new peer: Peer: permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260812 08:00:31.907058  5376 raft_consensus.cc:2955] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 4 to 10, 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } } }
I20260812 08:00:31.907881  4835 raft_consensus.cc:2955] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 4 to 10, 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } } }
I20260812 08:00:31.910466  5287 raft_consensus.cc:2955] T 77237ae0d0ea471193508c0773fac8c7 P 100406046c54412097105da287cb4797 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 4 to 10, 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } } }
I20260812 08:00:31.911101  4672 catalog_manager.cc:5719] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac reported cstate change: config changed from index 4 to 10, 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "891acef050c7407a852c2416b86af163" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } } }
I20260812 08:00:31.919178  4985 raft_consensus.cc:2955] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 4 to 10, 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } } }
I20260812 08:00:31.938015  5137 consensus_queue.cc:237] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 1.18, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } }
I20260812 08:00:31.939815  4834 raft_consensus.cc:1275] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac [term 1 FOLLOWER]: Refusing update from remote peer 891acef050c7407a852c2416b86af163: Log matching property violated. Preceding OpId in replica: term: 1 index: 18. Preceding OpId from leader: term: 1 index: 19. (index mismatch)
I20260812 08:00:31.940349  5369 consensus_queue.cc:1048] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260812 08:00:31.940805  4984 raft_consensus.cc:1275] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21 [term 1 FOLLOWER]: Refusing update from remote peer 891acef050c7407a852c2416b86af163: Log matching property violated. Preceding OpId in replica: term: 1 index: 18. Preceding OpId from leader: term: 1 index: 19. (index mismatch)
I20260812 08:00:31.941717  5369 consensus_queue.cc:1048] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 15, Time since last communication: 0.000s
I20260812 08:00:31.947788  5380 raft_consensus.cc:2955] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [term 1 LEADER]: Committing config change with OpId 1.19: config changed from index 10 to 19, 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) changed from VOTER to NON_VOTER. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } } }
I20260812 08:00:31.948227  4984 raft_consensus.cc:2955] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21 [term 1 FOLLOWER]: Committing config change with OpId 1.19: config changed from index 10 to 19, 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) changed from VOTER to NON_VOTER. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } } }
I20260812 08:00:31.950434  4834 raft_consensus.cc:2955] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac [term 1 LEARNER]: Committing config change with OpId 1.19: config changed from index 10 to 19, 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) changed from VOTER to NON_VOTER. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } } }
I20260812 08:00:31.951087  4661 catalog_manager.cc:5719] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21 reported cstate change: config changed from index 10 to 19, 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) changed from VOTER to NON_VOTER. New cstate: current_term: 1 leader_uuid: "891acef050c7407a852c2416b86af163" committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } } }
I20260812 08:00:31.952415  5287 raft_consensus.cc:1275] T 77237ae0d0ea471193508c0773fac8c7 P 100406046c54412097105da287cb4797 [term 1 FOLLOWER]: Refusing update from remote peer 891acef050c7407a852c2416b86af163: Log matching property violated. Preceding OpId in replica: term: 1 index: 18. Preceding OpId from leader: term: 1 index: 19. (index mismatch)
I20260812 08:00:31.953495  5369 consensus_queue.cc:1048] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Connected to new peer: Peer: permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.000s
I20260812 08:00:31.962044  5287 raft_consensus.cc:2955] T 77237ae0d0ea471193508c0773fac8c7 P 100406046c54412097105da287cb4797 [term 1 FOLLOWER]: Committing config change with OpId 1.19: config changed from index 10 to 19, 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) changed from VOTER to NON_VOTER. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } } }
I20260812 08:00:32.082674  4810 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:00:32.083755  5115 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260812 08:00:32.089788  4961 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:00:32.090349  5264 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |       Address       | Status
----------------------------------+---------------------+---------
 7c2985dd0c2f4ac2aa304a7e389a73e5 | 127.0.186.126:43507 | HEALTHY

Unusual flags for Master:
                         Flag                         |                                                                                          Value                                                                                           |      Tags       |         Master
------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 catalog_manager_evict_excess_replicas                | false                                                                                                                                                                                    | runtime,hidden  | all 1 server(s) checked
 catalog_manager_wait_for_new_tablets_to_elect_leader | false                                                                                                                                                                                    | hidden          | all 1 server(s) checked
 ipki_ca_key_size                                     | 768                                                                                                                                                                                      | experimental    | all 1 server(s) checked
 ipki_server_key_size                                 | 768                                                                                                                                                                                      | experimental    | all 1 server(s) checked
 never_fsync                                          | true                                                                                                                                                                                     | unsafe,advanced | all 1 server(s) checked
 openssl_security_level_override                      | 0                                                                                                                                                                                        | unsafe,hidden   | all 1 server(s) checked
 rpc_reuseport                                        | true                                                                                                                                                                                     | experimental    | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports                     | true                                                                                                                                                                                     | unsafe          | all 1 server(s) checked
 server_dump_info_format                              | pb                                                                                                                                                                                       | hidden          | all 1 server(s) checked
 server_dump_info_path                                | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden          | all 1 server(s) checked
 tsk_num_rsa_bits                                     | 512                                                                                                                                                                                      | experimental    | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |       Value        |         Master
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.0.186.84:33477 | all 1 server(s) checked
 time_source         | builtin            | all 1 server(s) checked

Tablet Server Summary
               UUID               |      Address       | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
 100406046c54412097105da287cb4797 | 127.0.186.68:40325 | HEALTHY | <none>   |       0        |       0
 1ef10e8ced0d4cc98922aeecd5a4b5ac | 127.0.186.65:35701 | HEALTHY | <none>   |       0        |       0
 891acef050c7407a852c2416b86af163 | 127.0.186.67:41995 | HEALTHY | <none>   |       1        |       0
 c63354224ecd4288b891afbd82a3da21 | 127.0.186.66:40797 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       4

Unusual flags for Tablet Server:
               Flag               |                                                                                        Value                                                                                         |      Tags       |      Tablet Server
----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 enable_leader_failure_detection  | false                                                                                                                                                                                | unsafe          | all 4 server(s) checked
 ipki_server_key_size             | 768                                                                                                                                                                                  | experimental    | all 4 server(s) checked
 local_ip_for_outbound_sockets    | 127.0.186.65                                                                                                                                                                         | experimental    | 127.0.186.65:35701
 local_ip_for_outbound_sockets    | 127.0.186.66                                                                                                                                                                         | experimental    | 127.0.186.66:40797
 local_ip_for_outbound_sockets    | 127.0.186.67                                                                                                                                                                         | experimental    | 127.0.186.67:41995
 local_ip_for_outbound_sockets    | 127.0.186.68                                                                                                                                                                         | experimental    | 127.0.186.68:40325
 never_fsync                      | true                                                                                                                                                                                 | unsafe,advanced | all 4 server(s) checked
 openssl_security_level_override  | 0                                                                                                                                                                                    | unsafe,hidden   | all 4 server(s) checked
 rpc_server_allow_ephemeral_ports | true                                                                                                                                                                                 | unsafe          | all 4 server(s) checked
 server_dump_info_format          | pb                                                                                                                                                                                   | hidden          | all 4 server(s) checked
 server_dump_info_path            | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden          | 127.0.186.65:35701
 server_dump_info_path            | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden          | 127.0.186.66:40797
 server_dump_info_path            | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden          | 127.0.186.67:41995
 server_dump_info_path            | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden          | 127.0.186.68:40325

Flags of checked categories for Tablet Server:
        Flag         |       Value        |      Tablet Server
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.0.186.84:33477 | all 4 server(s) checked
 time_source         | builtin            | all 4 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 5 server(s) checked

Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
   Name    | RF | Status  | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-----------+----+---------+---------------+---------+------------+------------------+-------------
 TestTable | 3  | HEALTHY | 1             | 1       | 0          | 0                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 4
 Tables         | 1
 Tablets        | 1
 Replicas       | 4

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

OK
I20260812 08:00:32.115527   745 log_verifier.cc:126] Checking tablet 77237ae0d0ea471193508c0773fac8c7
I20260812 08:00:32.166385   745 log_verifier.cc:177] Verified matching terms for 37 ops in tablet 77237ae0d0ea471193508c0773fac8c7
I20260812 08:00:32.189951  5137 consensus_queue.cc:237] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 37, Committed index: 37, Last appended: 1.37, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } }
I20260812 08:00:32.191339  4985 raft_consensus.cc:1275] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21 [term 1 FOLLOWER]: Refusing update from remote peer 891acef050c7407a852c2416b86af163: Log matching property violated. Preceding OpId in replica: term: 1 index: 37. Preceding OpId from leader: term: 1 index: 38. (index mismatch)
I20260812 08:00:32.191756  5426 consensus_queue.cc:1048] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 37, Time since last communication: 0.000s
I20260812 08:00:32.191810  4834 raft_consensus.cc:1275] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac [term 1 LEARNER]: Refusing update from remote peer 891acef050c7407a852c2416b86af163: Log matching property violated. Preceding OpId in replica: term: 1 index: 37. Preceding OpId from leader: term: 1 index: 38. (index mismatch)
I20260812 08:00:32.191886  5285 raft_consensus.cc:1275] T 77237ae0d0ea471193508c0773fac8c7 P 100406046c54412097105da287cb4797 [term 1 FOLLOWER]: Refusing update from remote peer 891acef050c7407a852c2416b86af163: Log matching property violated. Preceding OpId in replica: term: 1 index: 37. Preceding OpId from leader: term: 1 index: 38. (index mismatch)
I20260812 08:00:32.192299  5426 consensus_queue.cc:1048] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 37, Time since last communication: 0.000s
I20260812 08:00:32.192498  5380 consensus_queue.cc:1048] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Connected to new peer: Peer: permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 37, Time since last communication: 0.000s
I20260812 08:00:32.194550  5380 raft_consensus.cc:2955] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [term 1 LEADER]: Committing config change with OpId 1.38: config changed from index 19 to 38, 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) changed from NON_VOTER to VOTER. New config: { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } } }
I20260812 08:00:32.195125  4834 raft_consensus.cc:2955] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac [term 1 FOLLOWER]: Committing config change with OpId 1.38: config changed from index 19 to 38, 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) changed from NON_VOTER to VOTER. New config: { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } } }
I20260812 08:00:32.197113  4673 catalog_manager.cc:5719] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 reported cstate change: config changed from index 19 to 38, 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "891acef050c7407a852c2416b86af163" committed_config { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } health_report { overall_health: HEALTHY } } }
I20260812 08:00:32.197371  4985 raft_consensus.cc:2955] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21 [term 1 FOLLOWER]: Committing config change with OpId 1.38: config changed from index 19 to 38, 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) changed from NON_VOTER to VOTER. New config: { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } } }
I20260812 08:00:32.198279  5285 raft_consensus.cc:2955] T 77237ae0d0ea471193508c0773fac8c7 P 100406046c54412097105da287cb4797 [term 1 FOLLOWER]: Committing config change with OpId 1.38: config changed from index 19 to 38, 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) changed from NON_VOTER to VOTER. New config: { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } peers { permanent_uuid: "1ef10e8ced0d4cc98922aeecd5a4b5ac" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35701 } attrs { } } }
I20260812 08:00:32.207736  5137 consensus_queue.cc:237] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 38, Committed index: 38, Last appended: 1.38, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } }
I20260812 08:00:32.208822  4985 raft_consensus.cc:1275] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21 [term 1 FOLLOWER]: Refusing update from remote peer 891acef050c7407a852c2416b86af163: Log matching property violated. Preceding OpId in replica: term: 1 index: 38. Preceding OpId from leader: term: 1 index: 39. (index mismatch)
I20260812 08:00:32.208976  5285 raft_consensus.cc:1275] T 77237ae0d0ea471193508c0773fac8c7 P 100406046c54412097105da287cb4797 [term 1 FOLLOWER]: Refusing update from remote peer 891acef050c7407a852c2416b86af163: Log matching property violated. Preceding OpId in replica: term: 1 index: 38. Preceding OpId from leader: term: 1 index: 39. (index mismatch)
I20260812 08:00:32.209198  5426 consensus_queue.cc:1048] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 39, Last known committed idx: 38, Time since last communication: 0.000s
I20260812 08:00:32.209388  5369 consensus_queue.cc:1048] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [LEADER]: Connected to new peer: Peer: permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 39, Last known committed idx: 38, Time since last communication: 0.000s
I20260812 08:00:32.210851  5426 raft_consensus.cc:2955] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 [term 1 LEADER]: Committing config change with OpId 1.39: config changed from index 38 to 39, VOTER 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) evicted. New config: { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } }
I20260812 08:00:32.211516  4985 raft_consensus.cc:2955] T 77237ae0d0ea471193508c0773fac8c7 P c63354224ecd4288b891afbd82a3da21 [term 1 FOLLOWER]: Committing config change with OpId 1.39: config changed from index 38 to 39, VOTER 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) evicted. New config: { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } }
I20260812 08:00:32.211725  5285 raft_consensus.cc:2955] T 77237ae0d0ea471193508c0773fac8c7 P 100406046c54412097105da287cb4797 [term 1 FOLLOWER]: Committing config change with OpId 1.39: config changed from index 38 to 39, VOTER 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) evicted. New config: { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } } }
I20260812 08:00:32.213572  4661 catalog_manager.cc:5719] T 77237ae0d0ea471193508c0773fac8c7 P 891acef050c7407a852c2416b86af163 reported cstate change: config changed from index 38 to 39, VOTER 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65) evicted. New cstate: current_term: 1 leader_uuid: "891acef050c7407a852c2416b86af163" committed_config { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "100406046c54412097105da287cb4797" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 40325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c63354224ecd4288b891afbd82a3da21" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 40797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "891acef050c7407a852c2416b86af163" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 41995 } health_report { overall_health: HEALTHY } } }
I20260812 08:00:32.220052  4810 tablet_service.cc:1558] Processing DeleteTablet for tablet 77237ae0d0ea471193508c0773fac8c7 with delete_type TABLET_DATA_TOMBSTONED (TS 1ef10e8ced0d4cc98922aeecd5a4b5ac not found in new config with opid_index 39) from {username='slave'} at 127.0.0.1:58336
I20260812 08:00:32.220551  5467 tablet_replica.cc:333] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac: stopping tablet replica
I20260812 08:00:32.220837  5467 raft_consensus.cc:2243] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 08:00:32.221068  5467 raft_consensus.cc:2272] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 08:00:32.222869  5467 ts_tablet_manager.cc:1916] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 08:00:32.224696  5467 ts_tablet_manager.cc:1929] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.38
I20260812 08:00:32.224776  5467 log.cc:1199] T 77237ae0d0ea471193508c0773fac8c7 P 1ef10e8ced0d4cc98922aeecd5a4b5ac: Deleting WAL directory at /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromoteAndDemote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/77237ae0d0ea471193508c0773fac8c7
I20260812 08:00:32.225368  4655 catalog_manager.cc:5050] TS 1ef10e8ced0d4cc98922aeecd5a4b5ac (127.0.186.65:35701): tablet 77237ae0d0ea471193508c0773fac8c7 (table TestTable [id=3dcf2b5e1861491cb607ee1d948003fa]) successfully deleted
I20260812 08:00:32.228909   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 4742
I20260812 08:00:32.244839   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 4891
I20260812 08:00:32.254733   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 5041
I20260812 08:00:32.264074   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 5192
I20260812 08:00:32.275694   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 4635
2026-08-12T08:00:32Z chronyd exiting
[       OK ] RaftConsensusNonVoterITest.PromoteAndDemote (1727 ms)
[ RUN      ] RaftConsensusNonVoterITest.PromotedReplicaCanVote
I20260812 08:00:32.302672   745 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:00:32.302723   745 ts_itest-base.cc:116] --------------
I20260812 08:00:32.302734   745 ts_itest-base.cc:117] 4 tablet servers
I20260812 08:00:32.302743   745 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:00:32.302752   745 ts_itest-base.cc:119] --------------
2026-08-12T08:00:32Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:00:32Z Disabled control of system clock
I20260812 08:00:32.311853   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:42559
--webserver_interface=127.0.186.126
--webserver_port=0
--builtin_ntp_servers=127.0.186.84:45089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:42559
--catalog_manager_evict_excess_replicas=false with env {}
W20260812 08:00:32.416975  5476 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:32.417228  5476 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:32.417282  5476 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:32.420595  5476 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:00:32.420667  5476 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:32.420696  5476 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:00:32.420717  5476 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:00:32.424691  5476 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:45089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--catalog_manager_evict_excess_replicas=false
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:42559
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:42559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.5476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:32.425961  5476 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:32.427275  5476 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:32.433071  5486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:32.433080  5482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:32.433184  5476 server_base.cc:1061] running on GCE node
W20260812 08:00:32.433092  5484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:32.433867  5476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:32.434813  5476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:32.435993  5476 hybrid_clock.cc:648] HybridClock initialized: now 1786521632435975 us; error 39 us; skew 500 ppm
I20260812 08:00:32.437822  5476 webserver.cc:533] Webserver started at http://127.0.186.126:37317/ using document root <none> and password file <none>
I20260812 08:00:32.438407  5476 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:32.438467  5476 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:32.438661  5476 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:32.440260  5476 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "4cc359db13014b3f8ffe9a1e162c2cb9"
format_stamp: "Formatted at 2026-08-12 08:00:32 on dist-test-slave-49h3"
I20260812 08:00:32.440739  5476 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "4cc359db13014b3f8ffe9a1e162c2cb9"
format_stamp: "Formatted at 2026-08-12 08:00:32 on dist-test-slave-49h3"
I20260812 08:00:32.444084  5476 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.000s	sys 0.005s
I20260812 08:00:32.446287  5493 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:32.447340  5476 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260812 08:00:32.447461  5476 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "4cc359db13014b3f8ffe9a1e162c2cb9"
format_stamp: "Formatted at 2026-08-12 08:00:32 on dist-test-slave-49h3"
I20260812 08:00:32.447556  5476 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:32.462801  5476 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:32.463498  5476 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:32.463666  5476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:32.471004  5551 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:42559 every 8 connection(s)
I20260812 08:00:32.471017  5476 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:42559
I20260812 08:00:32.472076  5476 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:00:32.474942  5552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:32.477202   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 5476
I20260812 08:00:32.477324   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:00:32.479972  5552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cc359db13014b3f8ffe9a1e162c2cb9: Bootstrap starting.
I20260812 08:00:32.482117  5552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cc359db13014b3f8ffe9a1e162c2cb9: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:32.482807  5552 log.cc:826] T 00000000000000000000000000000000 P 4cc359db13014b3f8ffe9a1e162c2cb9: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:32.484683  5552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cc359db13014b3f8ffe9a1e162c2cb9: No bootstrap required, opened a new log
I20260812 08:00:32.487478  5552 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4cc359db13014b3f8ffe9a1e162c2cb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc359db13014b3f8ffe9a1e162c2cb9" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 42559 } }
I20260812 08:00:32.487660  5552 raft_consensus.cc:385] T 00000000000000000000000000000000 P 4cc359db13014b3f8ffe9a1e162c2cb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:32.487720  5552 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4cc359db13014b3f8ffe9a1e162c2cb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cc359db13014b3f8ffe9a1e162c2cb9, State: Initialized, Role: FOLLOWER
I20260812 08:00:32.488117  5552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cc359db13014b3f8ffe9a1e162c2cb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc359db13014b3f8ffe9a1e162c2cb9" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 42559 } }
I20260812 08:00:32.488255  5552 raft_consensus.cc:399] T 00000000000000000000000000000000 P 4cc359db13014b3f8ffe9a1e162c2cb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:00:32.488361  5552 raft_consensus.cc:493] T 00000000000000000000000000000000 P 4cc359db13014b3f8ffe9a1e162c2cb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:00:32.488487  5552 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4cc359db13014b3f8ffe9a1e162c2cb9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:32.489264  5552 raft_consensus.cc:515] T 00000000000000000000000000000000 P 4cc359db13014b3f8ffe9a1e162c2cb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc359db13014b3f8ffe9a1e162c2cb9" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 42559 } }
I20260812 08:00:32.489615  5552 leader_election.cc:304] T 00000000000000000000000000000000 P 4cc359db13014b3f8ffe9a1e162c2cb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4cc359db13014b3f8ffe9a1e162c2cb9; no voters: 
I20260812 08:00:32.489893  5552 leader_election.cc:290] T 00000000000000000000000000000000 P 4cc359db13014b3f8ffe9a1e162c2cb9 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:00:32.489990  5559 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 4cc359db13014b3f8ffe9a1e162c2cb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:00:32.490196  5559 raft_consensus.cc:697] T 00000000000000000000000000000000 P 4cc359db13014b3f8ffe9a1e162c2cb9 [term 1 LEADER]: Becoming Leader. State: Replica: 4cc359db13014b3f8ffe9a1e162c2cb9, State: Running, Role: LEADER
I20260812 08:00:32.490485  5559 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cc359db13014b3f8ffe9a1e162c2cb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc359db13014b3f8ffe9a1e162c2cb9" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 42559 } }
I20260812 08:00:32.490934  5552 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4cc359db13014b3f8ffe9a1e162c2cb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:00:32.492044  5560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cc359db13014b3f8ffe9a1e162c2cb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4cc359db13014b3f8ffe9a1e162c2cb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc359db13014b3f8ffe9a1e162c2cb9" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 42559 } } }
I20260812 08:00:32.492177  5560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cc359db13014b3f8ffe9a1e162c2cb9 [sys.catalog]: This master's current role is: LEADER
I20260812 08:00:32.493048  5561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cc359db13014b3f8ffe9a1e162c2cb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cc359db13014b3f8ffe9a1e162c2cb9. Latest consensus state: current_term: 1 leader_uuid: "4cc359db13014b3f8ffe9a1e162c2cb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc359db13014b3f8ffe9a1e162c2cb9" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 42559 } } }
I20260812 08:00:32.493237  5561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cc359db13014b3f8ffe9a1e162c2cb9 [sys.catalog]: This master's current role is: LEADER
W20260812 08:00:32.495515  5577 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 4cc359db13014b3f8ffe9a1e162c2cb9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 08:00:32.495581  5577 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 08:00:32.495639  5570 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:00:32.496337  5570 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:00:32.501508  5570 catalog_manager.cc:1383] Generated new cluster ID: f4d61bbd7b594d5a9a5c3ce1ec9b0581
I20260812 08:00:32.501562  5570 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:00:32.506955  5570 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:00:32.508019  5570 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:00:32.521939  5570 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 4cc359db13014b3f8ffe9a1e162c2cb9: Generated new TSK 0
I20260812 08:00:32.522686  5570 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:00:32.531292   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:0
--local_ip_for_outbound_sockets=127.0.186.65
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:42559
--builtin_ntp_servers=127.0.186.84:45089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=50
--evict_failed_followers=false with env {}
W20260812 08:00:32.637418  5581 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:32.637674  5581 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:32.637737  5581 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:32.641073  5581 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:32.641198  5581 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:00:32.645015  5581 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:45089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--evict_failed_followers=false
--raft_heartbeat_interval_ms=50
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:42559
--never_fsync=true
--heap_profile_path=/tmp/kudu.5581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:32.646171  5581 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:32.647323  5581 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:00:32.650055  5581 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 25
W20260812 08:00:32.653633  5587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:32.653584  5586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:32.653585  5591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:32.654047  5581 server_base.cc:1061] running on GCE node
I20260812 08:00:32.654405  5581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:32.654944  5581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:32.656095  5581 hybrid_clock.cc:648] HybridClock initialized: now 1786521632656079 us; error 32 us; skew 500 ppm
I20260812 08:00:32.657727  5581 webserver.cc:533] Webserver started at http://127.0.186.65:36037/ using document root <none> and password file <none>
I20260812 08:00:32.658277  5581 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:32.658334  5581 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:32.658511  5581 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:32.660009  5581 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "28637390589f4854bc0c4d1916be3269"
format_stamp: "Formatted at 2026-08-12 08:00:32 on dist-test-slave-49h3"
I20260812 08:00:32.660475  5581 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "28637390589f4854bc0c4d1916be3269"
format_stamp: "Formatted at 2026-08-12 08:00:32 on dist-test-slave-49h3"
I20260812 08:00:32.663609  5581 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:00:32.665643  5600 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:32.666596  5581 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:00:32.666697  5581 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "28637390589f4854bc0c4d1916be3269"
format_stamp: "Formatted at 2026-08-12 08:00:32 on dist-test-slave-49h3"
I20260812 08:00:32.666786  5581 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:32.703642  5581 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:32.704386  5581 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:32.704561  5581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:32.705166  5581 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:32.706275  5581 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:32.706326  5581 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:32.706379  5581 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:32.706403  5581 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:32.715289  5581 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:36089
I20260812 08:00:32.715351  5728 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:36089 every 8 connection(s)
I20260812 08:00:32.716318  5581 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:00:32.718344   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 5581
I20260812 08:00:32.718467   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:00:32.720922   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.66:0
--local_ip_for_outbound_sockets=127.0.186.66
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:42559
--builtin_ntp_servers=127.0.186.84:45089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=50
--evict_failed_followers=false with env {}
I20260812 08:00:32.724948  5729 heartbeater.cc:344] Connected to a master server at 127.0.186.126:42559
I20260812 08:00:32.725178  5729 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:32.725732  5729 heartbeater.cc:507] Master 127.0.186.126:42559 requested a full tablet report, sending...
I20260812 08:00:32.727373  5510 ts_manager.cc:194] Registered new tserver with Master: 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089)
I20260812 08:00:32.728995  5510 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.65:37333
W20260812 08:00:32.827152  5733 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:32.827365  5733 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:32.827406  5733 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:32.830659  5733 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:32.830761  5733 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.66
I20260812 08:00:32.834491  5733 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:45089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--evict_failed_followers=false
--raft_heartbeat_interval_ms=50
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:42559
--never_fsync=true
--heap_profile_path=/tmp/kudu.5733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:32.835528  5733 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:32.836632  5733 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:00:32.839381  5733 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 25
W20260812 08:00:32.842815  5741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:32.842799  5740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:32.842801  5743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:32.842923  5733 server_base.cc:1061] running on GCE node
I20260812 08:00:32.843380  5733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:32.843887  5733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:32.845032  5733 hybrid_clock.cc:648] HybridClock initialized: now 1786521632845020 us; error 24 us; skew 500 ppm
I20260812 08:00:32.846634  5733 webserver.cc:533] Webserver started at http://127.0.186.66:34577/ using document root <none> and password file <none>
I20260812 08:00:32.847177  5733 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:32.847254  5733 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:32.847453  5733 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:32.849014  5733 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "81f7779748444854812a4497693823fd"
format_stamp: "Formatted at 2026-08-12 08:00:32 on dist-test-slave-49h3"
I20260812 08:00:32.849519  5733 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "81f7779748444854812a4497693823fd"
format_stamp: "Formatted at 2026-08-12 08:00:32 on dist-test-slave-49h3"
I20260812 08:00:32.852751  5733 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.001s	sys 0.001s
I20260812 08:00:32.854934  5752 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:32.855890  5733 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:00:32.855986  5733 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "81f7779748444854812a4497693823fd"
format_stamp: "Formatted at 2026-08-12 08:00:32 on dist-test-slave-49h3"
I20260812 08:00:32.856073  5733 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:32.871969  5733 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:32.872610  5733 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:32.872774  5733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:32.873332  5733 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:32.874387  5733 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:32.874437  5733 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:32.874482  5733 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:32.874511  5733 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:32.883144  5733 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.66:35669
I20260812 08:00:32.883179  5885 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.66:35669 every 8 connection(s)
I20260812 08:00:32.884084  5733 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:00:32.887982   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 5733
I20260812 08:00:32.888087   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:00:32.890309   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.67:0
--local_ip_for_outbound_sockets=127.0.186.67
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:42559
--builtin_ntp_servers=127.0.186.84:45089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=50
--evict_failed_followers=false with env {}
I20260812 08:00:32.892329  5886 heartbeater.cc:344] Connected to a master server at 127.0.186.126:42559
I20260812 08:00:32.892601  5886 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:32.893102  5886 heartbeater.cc:507] Master 127.0.186.126:42559 requested a full tablet report, sending...
I20260812 08:00:32.894158  5510 ts_manager.cc:194] Registered new tserver with Master: 81f7779748444854812a4497693823fd (127.0.186.66:35669)
I20260812 08:00:32.894703  5510 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.66:34819
W20260812 08:00:32.998271  5890 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:32.998486  5890 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:32.998544  5890 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:33.001829  5890 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:33.001943  5890 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.67
I20260812 08:00:33.005738  5890 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:45089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--evict_failed_followers=false
--raft_heartbeat_interval_ms=50
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:42559
--never_fsync=true
--heap_profile_path=/tmp/kudu.5890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:33.006832  5890 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:33.007982  5890 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:00:33.010869  5890 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 25
W20260812 08:00:33.014506  5897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:33.014515  5899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:33.014660  5901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:33.014745  5890 server_base.cc:1061] running on GCE node
I20260812 08:00:33.015084  5890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:33.015601  5890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:33.016752  5890 hybrid_clock.cc:648] HybridClock initialized: now 1786521633016733 us; error 29 us; skew 500 ppm
I20260812 08:00:33.018461  5890 webserver.cc:533] Webserver started at http://127.0.186.67:33257/ using document root <none> and password file <none>
I20260812 08:00:33.019001  5890 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:33.019049  5890 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:33.019217  5890 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:33.020727  5890 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "f9541f9d66774e57922e780d5740351c"
format_stamp: "Formatted at 2026-08-12 08:00:33 on dist-test-slave-49h3"
I20260812 08:00:33.021211  5890 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "f9541f9d66774e57922e780d5740351c"
format_stamp: "Formatted at 2026-08-12 08:00:33 on dist-test-slave-49h3"
I20260812 08:00:33.024842  5890 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.000s
I20260812 08:00:33.026947  5908 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:33.027890  5890 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:00:33.027997  5890 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "f9541f9d66774e57922e780d5740351c"
format_stamp: "Formatted at 2026-08-12 08:00:33 on dist-test-slave-49h3"
I20260812 08:00:33.028081  5890 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:33.059690  5890 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:33.060397  5890 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:33.060565  5890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:33.061164  5890 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:33.062297  5890 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:33.062348  5890 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:33.062402  5890 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:33.062426  5890 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:33.071322  5890 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.67:37017
I20260812 08:00:33.071362  6043 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.67:37017 every 8 connection(s)
I20260812 08:00:33.072284  5890 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:00:33.077013   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 5890
I20260812 08:00:33.077136   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:00:33.079493   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.68:0
--local_ip_for_outbound_sockets=127.0.186.68
--webserver_interface=127.0.186.68
--webserver_port=0
--tserver_master_addrs=127.0.186.126:42559
--builtin_ntp_servers=127.0.186.84:45089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=50
--evict_failed_followers=false with env {}
I20260812 08:00:33.080425  6044 heartbeater.cc:344] Connected to a master server at 127.0.186.126:42559
I20260812 08:00:33.080651  6044 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:33.081143  6044 heartbeater.cc:507] Master 127.0.186.126:42559 requested a full tablet report, sending...
I20260812 08:00:33.082278  5510 ts_manager.cc:194] Registered new tserver with Master: f9541f9d66774e57922e780d5740351c (127.0.186.67:37017)
I20260812 08:00:33.082820  5510 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.67:52387
W20260812 08:00:33.186777  6048 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:33.186990  6048 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:33.187033  6048 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:33.190337  6048 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:33.190451  6048 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.68
I20260812 08:00:33.194440  6048 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:45089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--evict_failed_followers=false
--raft_heartbeat_interval_ms=50
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.0.186.68
--webserver_port=0
--tserver_master_addrs=127.0.186.126:42559
--never_fsync=true
--heap_profile_path=/tmp/kudu.6048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.68

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:33.195570  6048 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:33.196659  6048 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:00:33.199448  6048 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 25
W20260812 08:00:33.202905  6056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:33.202939  6055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:33.202965  6058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:33.203315  6048 server_base.cc:1061] running on GCE node
I20260812 08:00:33.203656  6048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:33.204173  6048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:33.205331  6048 hybrid_clock.cc:648] HybridClock initialized: now 1786521633205316 us; error 35 us; skew 500 ppm
I20260812 08:00:33.207127  6048 webserver.cc:533] Webserver started at http://127.0.186.68:41223/ using document root <none> and password file <none>
I20260812 08:00:33.207701  6048 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:33.207759  6048 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:33.207937  6048 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:33.209410  6048 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "e54ed57d03b94188a017315e98e22e29"
format_stamp: "Formatted at 2026-08-12 08:00:33 on dist-test-slave-49h3"
I20260812 08:00:33.209904  6048 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "e54ed57d03b94188a017315e98e22e29"
format_stamp: "Formatted at 2026-08-12 08:00:33 on dist-test-slave-49h3"
I20260812 08:00:33.212966  6048 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.001s	sys 0.002s
I20260812 08:00:33.215288  6066 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:33.216281  6048 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:00:33.216393  6048 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "e54ed57d03b94188a017315e98e22e29"
format_stamp: "Formatted at 2026-08-12 08:00:33 on dist-test-slave-49h3"
I20260812 08:00:33.216480  6048 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:33.240860  6048 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:33.241698  6048 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:33.241868  6048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:33.242467  6048 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:33.243517  6048 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:33.243577  6048 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:33.243631  6048 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:33.243654  6048 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:33.252516  6048 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.68:36743
I20260812 08:00:33.252580  6197 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.68:36743 every 8 connection(s)
I20260812 08:00:33.253512  6048 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20260812 08:00:33.256955   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 6048
I20260812 08:00:33.257102   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.PromotedReplicaCanVote.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20260812 08:00:33.262449  6198 heartbeater.cc:344] Connected to a master server at 127.0.186.126:42559
I20260812 08:00:33.262677  6198 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:33.263167  6198 heartbeater.cc:507] Master 127.0.186.126:42559 requested a full tablet report, sending...
I20260812 08:00:33.264148  5510 ts_manager.cc:194] Registered new tserver with Master: e54ed57d03b94188a017315e98e22e29 (127.0.186.68:36743)
I20260812 08:00:33.264708  5510 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.68:48383
I20260812 08:00:33.271880   745 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20260812 08:00:33.280591  5510 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52278:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
I20260812 08:00:33.299860  5654 tablet_service.cc:1511] Processing CreateTablet for tablet 3e819892c07d46f29e0e6483dcce9cd0 (DEFAULT_TABLE table=TestTable [id=7cb66f810a2247a79c994f994e824528]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:33.299891  6120 tablet_service.cc:1511] Processing CreateTablet for tablet 3e819892c07d46f29e0e6483dcce9cd0 (DEFAULT_TABLE table=TestTable [id=7cb66f810a2247a79c994f994e824528]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:33.299891  5807 tablet_service.cc:1511] Processing CreateTablet for tablet 3e819892c07d46f29e0e6483dcce9cd0 (DEFAULT_TABLE table=TestTable [id=7cb66f810a2247a79c994f994e824528]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:33.300779  5654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e819892c07d46f29e0e6483dcce9cd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:33.300823  5807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e819892c07d46f29e0e6483dcce9cd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:33.300823  6120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e819892c07d46f29e0e6483dcce9cd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:33.305815  6223 tablet_bootstrap.cc:492] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269: Bootstrap starting.
I20260812 08:00:33.306833  6224 tablet_bootstrap.cc:492] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29: Bootstrap starting.
I20260812 08:00:33.306962  6225 tablet_bootstrap.cc:492] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd: Bootstrap starting.
I20260812 08:00:33.308240  6223 tablet_bootstrap.cc:654] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:33.308931  6225 tablet_bootstrap.cc:654] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:33.308931  6224 tablet_bootstrap.cc:654] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:33.308929  6223 log.cc:826] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:33.309757  6225 log.cc:826] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:33.309754  6224 log.cc:826] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:33.310597  6223 tablet_bootstrap.cc:492] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269: No bootstrap required, opened a new log
I20260812 08:00:33.310784  6223 ts_tablet_manager.cc:1403] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:00:33.311352  6225 tablet_bootstrap.cc:492] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd: No bootstrap required, opened a new log
I20260812 08:00:33.311352  6224 tablet_bootstrap.cc:492] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29: No bootstrap required, opened a new log
I20260812 08:00:33.311527  6225 ts_tablet_manager.cc:1403] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.000s
I20260812 08:00:33.311527  6224 ts_tablet_manager.cc:1403] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.000s
I20260812 08:00:33.313236  6223 raft_consensus.cc:359] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } }
I20260812 08:00:33.313488  6223 raft_consensus.cc:385] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:33.313551  6223 raft_consensus.cc:740] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28637390589f4854bc0c4d1916be3269, State: Initialized, Role: FOLLOWER
I20260812 08:00:33.313987  6223 consensus_queue.cc:260] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } }
I20260812 08:00:33.314458  6225 raft_consensus.cc:359] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } }
I20260812 08:00:33.314458  6224 raft_consensus.cc:359] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } }
I20260812 08:00:33.314605  5729 heartbeater.cc:499] Master 127.0.186.126:42559 was elected leader, sending a full tablet report...
I20260812 08:00:33.314646  6224 raft_consensus.cc:385] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:33.314646  6225 raft_consensus.cc:385] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:33.314702  6224 raft_consensus.cc:740] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e54ed57d03b94188a017315e98e22e29, State: Initialized, Role: FOLLOWER
I20260812 08:00:33.314702  6225 raft_consensus.cc:740] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81f7779748444854812a4497693823fd, State: Initialized, Role: FOLLOWER
I20260812 08:00:33.315099  6225 consensus_queue.cc:260] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } }
I20260812 08:00:33.315099  6224 consensus_queue.cc:260] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } }
I20260812 08:00:33.315794  5886 heartbeater.cc:499] Master 127.0.186.126:42559 was elected leader, sending a full tablet report...
I20260812 08:00:33.315934  6224 ts_tablet_manager.cc:1434] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:00:33.316020  6198 heartbeater.cc:499] Master 127.0.186.126:42559 was elected leader, sending a full tablet report...
I20260812 08:00:33.316659  6225 ts_tablet_manager.cc:1434] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260812 08:00:33.317248  6223 ts_tablet_manager.cc:1434] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260812 08:00:33.319098  6231 raft_consensus.cc:493] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:00:33.319263  6231 raft_consensus.cc:515] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } }
I20260812 08:00:33.320500  6231 leader_election.cc:290] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e54ed57d03b94188a017315e98e22e29 (127.0.186.68:36743), 81f7779748444854812a4497693823fd (127.0.186.66:35669)
I20260812 08:00:33.322333  6233 raft_consensus.cc:493] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:00:33.322479  6233 raft_consensus.cc:515] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } }
I20260812 08:00:33.324764  6233 leader_election.cc:290] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e54ed57d03b94188a017315e98e22e29 (127.0.186.68:36743), 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089)
I20260812 08:00:33.325314  6145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "28637390589f4854bc0c4d1916be3269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e54ed57d03b94188a017315e98e22e29" is_pre_election: true
I20260812 08:00:33.325589  6145 raft_consensus.cc:2468] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28637390589f4854bc0c4d1916be3269 in term 0.
I20260812 08:00:33.325834  5830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "28637390589f4854bc0c4d1916be3269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81f7779748444854812a4497693823fd" is_pre_election: true
I20260812 08:00:33.325999  5830 raft_consensus.cc:2468] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28637390589f4854bc0c4d1916be3269 in term 0.
I20260812 08:00:33.326087  5601 leader_election.cc:304] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28637390589f4854bc0c4d1916be3269, e54ed57d03b94188a017315e98e22e29; no voters: 
I20260812 08:00:33.326975  6231 raft_consensus.cc:2804] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:00:33.327044  6231 raft_consensus.cc:493] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:00:33.327075  6231 raft_consensus.cc:3060] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:33.327893  6231 raft_consensus.cc:515] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } }
I20260812 08:00:33.328224  6231 leader_election.cc:290] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [CANDIDATE]: Term 1 election: Requested vote from peers e54ed57d03b94188a017315e98e22e29 (127.0.186.68:36743), 81f7779748444854812a4497693823fd (127.0.186.66:35669)
I20260812 08:00:33.328540  6145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "28637390589f4854bc0c4d1916be3269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e54ed57d03b94188a017315e98e22e29"
I20260812 08:00:33.328660  6145 raft_consensus.cc:3060] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:33.328755  5677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "81f7779748444854812a4497693823fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28637390589f4854bc0c4d1916be3269" is_pre_election: true
I20260812 08:00:33.328949  5677 raft_consensus.cc:2393] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 81f7779748444854812a4497693823fd in current term 1: Already voted for candidate 28637390589f4854bc0c4d1916be3269 in this term.
I20260812 08:00:33.329330  5830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "28637390589f4854bc0c4d1916be3269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81f7779748444854812a4497693823fd"
I20260812 08:00:33.329439  5830 raft_consensus.cc:3060] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:33.329579  6144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "81f7779748444854812a4497693823fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e54ed57d03b94188a017315e98e22e29" is_pre_election: true
I20260812 08:00:33.329770  6145 raft_consensus.cc:2468] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28637390589f4854bc0c4d1916be3269 in term 1.
I20260812 08:00:33.330075  5601 leader_election.cc:304] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28637390589f4854bc0c4d1916be3269, e54ed57d03b94188a017315e98e22e29; no voters: 
I20260812 08:00:33.330265  6231 raft_consensus.cc:2804] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:00:33.330255  5753 leader_election.cc:304] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 81f7779748444854812a4497693823fd; no voters: 28637390589f4854bc0c4d1916be3269, e54ed57d03b94188a017315e98e22e29
I20260812 08:00:33.330454  6231 raft_consensus.cc:697] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [term 1 LEADER]: Becoming Leader. State: Replica: 28637390589f4854bc0c4d1916be3269, State: Running, Role: LEADER
I20260812 08:00:33.330478  5830 raft_consensus.cc:2468] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28637390589f4854bc0c4d1916be3269 in term 1.
I20260812 08:00:33.330569  6233 raft_consensus.cc:2749] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 08:00:33.330755  6231 consensus_queue.cc:237] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } }
I20260812 08:00:33.333787  5508 catalog_manager.cc:5719] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28637390589f4854bc0c4d1916be3269 (127.0.186.65). New cstate: current_term: 1 leader_uuid: "28637390589f4854bc0c4d1916be3269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } health_report { overall_health: HEALTHY } } }
I20260812 08:00:33.343187   745 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20260812 08:00:33.344524   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 28637390589f4854bc0c4d1916be3269 to finish bootstrapping
I20260812 08:00:33.349601   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 81f7779748444854812a4497693823fd to finish bootstrapping
I20260812 08:00:33.354251   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver e54ed57d03b94188a017315e98e22e29 to finish bootstrapping
I20260812 08:00:33.369277   745 cluster_itest_util.cc:258] Not converged past 1 yet: 1.1 0.0 0.0
W20260812 08:00:33.385160  5888 tablet.cc:2406] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:00:33.388532  6231 consensus_queue.cc:1048] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:00:33.393714  6239 consensus_queue.cc:1048] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:00:33.477541  5677 consensus_queue.cc:237] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } }
I20260812 08:00:33.479480  6145 raft_consensus.cc:1275] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Refusing update from remote peer 28637390589f4854bc0c4d1916be3269: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:00:33.479858  5830 raft_consensus.cc:1275] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Refusing update from remote peer 28637390589f4854bc0c4d1916be3269: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:00:33.480024  6239 consensus_queue.cc:1048] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:00:33.480791  6239 consensus_queue.cc:1048] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:00:33.482483  6251 raft_consensus.cc:2955] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f9541f9d66774e57922e780d5740351c (127.0.186.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } } }
I20260812 08:00:33.482954  6145 raft_consensus.cc:2955] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f9541f9d66774e57922e780d5740351c (127.0.186.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } } }
I20260812 08:00:33.485622  5510 catalog_manager.cc:5719] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 reported cstate change: config changed from index -1 to 2, NON_VOTER f9541f9d66774e57922e780d5740351c (127.0.186.67) added. New cstate: current_term: 1 leader_uuid: "28637390589f4854bc0c4d1916be3269" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } health_report { overall_health: UNKNOWN } } }
W20260812 08:00:33.487054  5604 consensus_peers.cc:597] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 -> Peer f9541f9d66774e57922e780d5740351c (127.0.186.67:37017): Couldn't send request to peer f9541f9d66774e57922e780d5740351c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e819892c07d46f29e0e6483dcce9cd0. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:00:33.487979  5830 raft_consensus.cc:2955] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f9541f9d66774e57922e780d5740351c (127.0.186.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } } }
I20260812 08:00:33.520921  6254 ts_tablet_manager.cc:933] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c: Initiating tablet copy from peer 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089)
I20260812 08:00:33.521616  6254 tablet_copy_client.cc:323] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.186.65:36089
I20260812 08:00:33.525017  5698 tablet_copy_service.cc:140] P 28637390589f4854bc0c4d1916be3269: Received BeginTabletCopySession request for tablet 3e819892c07d46f29e0e6483dcce9cd0 from peer f9541f9d66774e57922e780d5740351c ({username='slave'} at 127.0.186.67:60983)
I20260812 08:00:33.525169  5698 tablet_copy_service.cc:161] P 28637390589f4854bc0c4d1916be3269: Beginning new tablet copy session on tablet 3e819892c07d46f29e0e6483dcce9cd0 from peer f9541f9d66774e57922e780d5740351c at {username='slave'} at 127.0.186.67:60983: session id = f9541f9d66774e57922e780d5740351c-3e819892c07d46f29e0e6483dcce9cd0
I20260812 08:00:33.526391  5698 tablet_copy_source_session.cc:215] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 08:00:33.530164  6254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e819892c07d46f29e0e6483dcce9cd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:33.535125  6254 tablet_copy_client.cc:806] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c: tablet copy: Starting download of 0 data blocks...
I20260812 08:00:33.535323  6254 tablet_copy_client.cc:670] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c: tablet copy: Starting download of 1 WAL segments...
I20260812 08:00:33.537192  6254 tablet_copy_client.cc:538] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 08:00:33.538905  6254 tablet_bootstrap.cc:492] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c: Bootstrap starting.
I20260812 08:00:33.543387  6254 log.cc:826] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:33.545720  6254 tablet_bootstrap.cc:492] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:00:33.545971  6254 tablet_bootstrap.cc:492] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c: Bootstrap complete.
I20260812 08:00:33.546244  6254 ts_tablet_manager.cc:1403] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c: Time spent bootstrapping tablet: real 0.008s	user 0.007s	sys 0.001s
I20260812 08:00:33.548560  6254 raft_consensus.cc:359] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } }
I20260812 08:00:33.548794  6254 raft_consensus.cc:740] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f9541f9d66774e57922e780d5740351c, State: Initialized, Role: LEARNER
I20260812 08:00:33.549115  6254 consensus_queue.cc:260] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } }
I20260812 08:00:33.549831  6044 heartbeater.cc:499] Master 127.0.186.126:42559 was elected leader, sending a full tablet report...
I20260812 08:00:33.550005  6254 ts_tablet_manager.cc:1434] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260812 08:00:33.550956  5698 tablet_copy_service.cc:342] P 28637390589f4854bc0c4d1916be3269: Request end of tablet copy session f9541f9d66774e57922e780d5740351c-3e819892c07d46f29e0e6483dcce9cd0 received from {username='slave'} at 127.0.186.67:60983
I20260812 08:00:33.551077  5698 tablet_copy_service.cc:434] P 28637390589f4854bc0c4d1916be3269: ending tablet copy session f9541f9d66774e57922e780d5740351c-3e819892c07d46f29e0e6483dcce9cd0 on tablet 3e819892c07d46f29e0e6483dcce9cd0 with peer f9541f9d66774e57922e780d5740351c
I20260812 08:00:33.555627  5677 consensus_queue.cc:237] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } }
I20260812 08:00:33.557075  6145 raft_consensus.cc:1275] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Refusing update from remote peer 28637390589f4854bc0c4d1916be3269: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 08:00:33.557365  5990 raft_consensus.cc:1275] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [term 1 LEARNER]: Refusing update from remote peer 28637390589f4854bc0c4d1916be3269: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 08:00:33.557556  6251 consensus_queue.cc:1048] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 08:00:33.557935  5830 raft_consensus.cc:1275] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Refusing update from remote peer 28637390589f4854bc0c4d1916be3269: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 08:00:33.557960  6251 consensus_queue.cc:1048] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 08:00:33.558321  6251 consensus_queue.cc:1048] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 08:00:33.560760  6239 raft_consensus.cc:2955] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f9541f9d66774e57922e780d5740351c (127.0.186.67) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } } }
I20260812 08:00:33.561571  5990 raft_consensus.cc:2955] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f9541f9d66774e57922e780d5740351c (127.0.186.67) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } } }
I20260812 08:00:33.562397  5830 raft_consensus.cc:2955] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f9541f9d66774e57922e780d5740351c (127.0.186.67) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } } }
I20260812 08:00:33.562866  6145 raft_consensus.cc:2955] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f9541f9d66774e57922e780d5740351c (127.0.186.67) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } } }
I20260812 08:00:33.564800  5509 catalog_manager.cc:5719] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c reported cstate change: config changed from index 2 to 3, f9541f9d66774e57922e780d5740351c (127.0.186.67) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "28637390589f4854bc0c4d1916be3269" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } } }
I20260812 08:00:33.565963   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 28637390589f4854bc0c4d1916be3269 to finish bootstrapping
I20260812 08:00:33.567014   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 81f7779748444854812a4497693823fd to finish bootstrapping
I20260812 08:00:33.567991   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver f9541f9d66774e57922e780d5740351c to finish bootstrapping
I20260812 08:00:33.571866   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver e54ed57d03b94188a017315e98e22e29 to finish bootstrapping
I20260812 08:00:33.713680  6233 raft_consensus.cc:493] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28637390589f4854bc0c4d1916be3269)
I20260812 08:00:33.713724  6232 raft_consensus.cc:493] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28637390589f4854bc0c4d1916be3269)
I20260812 08:00:33.713855  6233 raft_consensus.cc:515] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } }
I20260812 08:00:33.713872  6232 raft_consensus.cc:515] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } }
I20260812 08:00:33.714480  6233 leader_election.cc:290] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e54ed57d03b94188a017315e98e22e29 (127.0.186.68:36743), 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089), f9541f9d66774e57922e780d5740351c (127.0.186.67:37017)
I20260812 08:00:33.714941  6145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "81f7779748444854812a4497693823fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e54ed57d03b94188a017315e98e22e29" is_pre_election: true
I20260812 08:00:33.715078  6145 raft_consensus.cc:2468] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81f7779748444854812a4497693823fd in term 1.
I20260812 08:00:33.715253  6232 leader_election.cc:290] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 81f7779748444854812a4497693823fd (127.0.186.66:35669), 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089), f9541f9d66774e57922e780d5740351c (127.0.186.67:37017)
I20260812 08:00:33.718556  5830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "e54ed57d03b94188a017315e98e22e29" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "81f7779748444854812a4497693823fd" is_pre_election: true
I20260812 08:00:33.718708  5830 raft_consensus.cc:2468] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e54ed57d03b94188a017315e98e22e29 in term 1.
W20260812 08:00:33.872220  6072 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: connection negotiation to 127.0.186.67:37017 for RPC RequestConsensusVote timed out after 0.156s (ON_OUTBOUND_QUEUE)
W20260812 08:00:33.925551  5753 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.186.65:36089 timed out after 0.210s (SENT)
W20260812 08:00:34.029373  6072 leader_election.cc:336] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f9541f9d66774e57922e780d5740351c (127.0.186.67:37017): Timed out: connection negotiation to 127.0.186.67:37017 for RPC RequestConsensusVote timed out after 0.156s (ON_OUTBOUND_QUEUE)
W20260812 08:00:34.029610  6068 leader_election.cc:336] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089): Timed out: connection negotiation to 127.0.186.65:36089 for RPC RequestConsensusVote timed out after 0.156s (ON_OUTBOUND_QUEUE)
I20260812 08:00:34.029683  6068 leader_election.cc:304] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 81f7779748444854812a4497693823fd, e54ed57d03b94188a017315e98e22e29; no voters: 28637390589f4854bc0c4d1916be3269, f9541f9d66774e57922e780d5740351c
I20260812 08:00:34.029953  6232 raft_consensus.cc:2749] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 08:00:34.136699  5758 leader_election.cc:336] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f9541f9d66774e57922e780d5740351c (127.0.186.67:37017): Timed out: connection negotiation to 127.0.186.67:37017 for RPC RequestConsensusVote timed out after 0.210s (ON_OUTBOUND_QUEUE)
W20260812 08:00:34.136929  5753 leader_election.cc:336] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089): Timed out: RequestConsensusVote RPC to 127.0.186.65:36089 timed out after 0.210s (SENT)
I20260812 08:00:34.137032  5753 leader_election.cc:304] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 81f7779748444854812a4497693823fd, e54ed57d03b94188a017315e98e22e29; no voters: 28637390589f4854bc0c4d1916be3269, f9541f9d66774e57922e780d5740351c
I20260812 08:00:34.137243  6233 raft_consensus.cc:2749] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 08:00:34.182756  6232 raft_consensus.cc:493] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28637390589f4854bc0c4d1916be3269)
I20260812 08:00:34.182928  6232 raft_consensus.cc:515] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } }
I20260812 08:00:34.183656  6232 leader_election.cc:290] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 81f7779748444854812a4497693823fd (127.0.186.66:35669), 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089), f9541f9d66774e57922e780d5740351c (127.0.186.67:37017)
I20260812 08:00:34.184209  5830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "e54ed57d03b94188a017315e98e22e29" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "81f7779748444854812a4497693823fd" is_pre_election: true
I20260812 08:00:34.184379  5830 raft_consensus.cc:2468] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e54ed57d03b94188a017315e98e22e29 in term 1.
I20260812 08:00:34.437238  6233 raft_consensus.cc:493] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28637390589f4854bc0c4d1916be3269)
I20260812 08:00:34.437398  6233 raft_consensus.cc:515] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } }
I20260812 08:00:34.437959  6233 leader_election.cc:290] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e54ed57d03b94188a017315e98e22e29 (127.0.186.68:36743), 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089), f9541f9d66774e57922e780d5740351c (127.0.186.67:37017)
I20260812 08:00:34.438529  6145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "81f7779748444854812a4497693823fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e54ed57d03b94188a017315e98e22e29" is_pre_election: true
I20260812 08:00:34.438710  6145 raft_consensus.cc:2468] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81f7779748444854812a4497693823fd in term 1.
W20260812 08:00:34.675464  6072 leader_election.cc:336] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f9541f9d66774e57922e780d5740351c (127.0.186.67:37017): Timed out: connection negotiation to 127.0.186.67:37017 for RPC RequestConsensusVote timed out after 0.245s (ON_OUTBOUND_QUEUE)
W20260812 08:00:34.676004  6068 leader_election.cc:336] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089): Timed out: connection negotiation to 127.0.186.65:36089 for RPC RequestConsensusVote timed out after 0.245s (ON_OUTBOUND_QUEUE)
I20260812 08:00:34.676071  6068 leader_election.cc:304] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 81f7779748444854812a4497693823fd, e54ed57d03b94188a017315e98e22e29; no voters: 28637390589f4854bc0c4d1916be3269, f9541f9d66774e57922e780d5740351c
I20260812 08:00:34.676326  6232 raft_consensus.cc:2749] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 08:00:34.839777  5758 leader_election.cc:336] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f9541f9d66774e57922e780d5740351c (127.0.186.67:37017): Timed out: connection negotiation to 127.0.186.67:37017 for RPC RequestConsensusVote timed out after 0.200s (ON_OUTBOUND_QUEUE)
W20260812 08:00:34.839936  5753 leader_election.cc:336] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089): Timed out: RequestConsensusVote RPC to 127.0.186.65:36089 timed out after 0.200s (SENT)
I20260812 08:00:34.839998  5753 leader_election.cc:304] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 81f7779748444854812a4497693823fd, e54ed57d03b94188a017315e98e22e29; no voters: 28637390589f4854bc0c4d1916be3269, f9541f9d66774e57922e780d5740351c
I20260812 08:00:34.840214  6233 raft_consensus.cc:2749] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 08:00:34.870316  6232 raft_consensus.cc:493] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28637390589f4854bc0c4d1916be3269)
I20260812 08:00:34.870491  6232 raft_consensus.cc:515] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } }
I20260812 08:00:34.871109  6232 leader_election.cc:290] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 81f7779748444854812a4497693823fd (127.0.186.66:35669), 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089), f9541f9d66774e57922e780d5740351c (127.0.186.67:37017)
I20260812 08:00:34.871623  5830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "e54ed57d03b94188a017315e98e22e29" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "81f7779748444854812a4497693823fd" is_pre_election: true
I20260812 08:00:34.871783  5830 raft_consensus.cc:2468] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e54ed57d03b94188a017315e98e22e29 in term 1.
I20260812 08:00:35.362470  6267 raft_consensus.cc:493] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28637390589f4854bc0c4d1916be3269)
I20260812 08:00:35.362637  6267 raft_consensus.cc:515] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } }
I20260812 08:00:35.363204  6267 leader_election.cc:290] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e54ed57d03b94188a017315e98e22e29 (127.0.186.68:36743), 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089), f9541f9d66774e57922e780d5740351c (127.0.186.67:37017)
I20260812 08:00:35.363722  6145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "81f7779748444854812a4497693823fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e54ed57d03b94188a017315e98e22e29" is_pre_election: true
I20260812 08:00:35.363869  6145 raft_consensus.cc:2468] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81f7779748444854812a4497693823fd in term 1.
W20260812 08:00:35.557363  6072 leader_election.cc:336] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f9541f9d66774e57922e780d5740351c (127.0.186.67:37017): Timed out: connection negotiation to 127.0.186.67:37017 for RPC RequestConsensusVote timed out after 0.342s (ON_OUTBOUND_QUEUE)
W20260812 08:00:35.557564  6068 leader_election.cc:336] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089): Timed out: connection negotiation to 127.0.186.65:36089 for RPC RequestConsensusVote timed out after 0.342s (ON_OUTBOUND_QUEUE)
I20260812 08:00:35.557687  6068 leader_election.cc:304] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 81f7779748444854812a4497693823fd, e54ed57d03b94188a017315e98e22e29; no voters: 28637390589f4854bc0c4d1916be3269, f9541f9d66774e57922e780d5740351c
I20260812 08:00:35.558085  6268 raft_consensus.cc:2749] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 08:00:35.791193  6268 raft_consensus.cc:493] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28637390589f4854bc0c4d1916be3269)
I20260812 08:00:35.791381  6268 raft_consensus.cc:515] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } }
I20260812 08:00:35.791993  6268 leader_election.cc:290] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 81f7779748444854812a4497693823fd (127.0.186.66:35669), 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089), f9541f9d66774e57922e780d5740351c (127.0.186.67:37017)
I20260812 08:00:35.792516  5830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "e54ed57d03b94188a017315e98e22e29" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "81f7779748444854812a4497693823fd" is_pre_election: true
I20260812 08:00:35.792696  5830 raft_consensus.cc:2468] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e54ed57d03b94188a017315e98e22e29 in term 1.
W20260812 08:00:36.059468  5758 leader_election.cc:336] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f9541f9d66774e57922e780d5740351c (127.0.186.67:37017): Timed out: connection negotiation to 127.0.186.67:37017 for RPC RequestConsensusVote timed out after 0.347s (ON_OUTBOUND_QUEUE)
W20260812 08:00:36.059629  5753 leader_election.cc:336] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089): Timed out: RequestConsensusVote RPC to 127.0.186.65:36089 timed out after 0.347s (SENT)
I20260812 08:00:36.059692  5753 leader_election.cc:304] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 81f7779748444854812a4497693823fd, e54ed57d03b94188a017315e98e22e29; no voters: 28637390589f4854bc0c4d1916be3269, f9541f9d66774e57922e780d5740351c
I20260812 08:00:36.060060  6270 raft_consensus.cc:2749] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 08:00:36.267012  6270 raft_consensus.cc:493] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28637390589f4854bc0c4d1916be3269)
I20260812 08:00:36.267184  6270 raft_consensus.cc:515] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } }
I20260812 08:00:36.267709  6270 leader_election.cc:290] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e54ed57d03b94188a017315e98e22e29 (127.0.186.68:36743), 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089), f9541f9d66774e57922e780d5740351c (127.0.186.67:37017)
I20260812 08:00:36.268227  6145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "81f7779748444854812a4497693823fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e54ed57d03b94188a017315e98e22e29" is_pre_election: true
I20260812 08:00:36.268384  6145 raft_consensus.cc:2468] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81f7779748444854812a4497693823fd in term 1.
W20260812 08:00:36.645488  6068 leader_election.cc:336] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089): Timed out: connection negotiation to 127.0.186.65:36089 for RPC RequestConsensusVote timed out after 0.426s (ON_OUTBOUND_QUEUE)
W20260812 08:00:36.646171  6072 leader_election.cc:336] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f9541f9d66774e57922e780d5740351c (127.0.186.67:37017): Timed out: connection negotiation to 127.0.186.67:37017 for RPC RequestConsensusVote timed out after 0.426s (ON_OUTBOUND_QUEUE)
I20260812 08:00:36.646255  6072 leader_election.cc:304] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 81f7779748444854812a4497693823fd, e54ed57d03b94188a017315e98e22e29; no voters: 28637390589f4854bc0c4d1916be3269, f9541f9d66774e57922e780d5740351c
I20260812 08:00:36.646596  6272 raft_consensus.cc:2749] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 08:00:36.898203  6260 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:00:33.715481 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.186.67:37017 (local address 127.0.186.68:56139)
0812 08:00:33.715773 (+   292us) negotiation.cc:107] Waiting for socket to connect
0812 08:00:33.715786 (+    13us) client_negotiation.cc:175] Beginning negotiation
0812 08:00:33.715860 (+    74us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 08:00:36.897680 (+3181820us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.186.67:37017: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":96,"threads_started":1}
W20260812 08:00:36.898200  6238 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:00:33.714811 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.186.67:37017 (local address 127.0.186.66:35731)
0812 08:00:33.715012 (+   201us) negotiation.cc:107] Waiting for socket to connect
0812 08:00:33.715027 (+    15us) client_negotiation.cc:175] Beginning negotiation
0812 08:00:33.715123 (+    96us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 08:00:36.897689 (+3182566us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.186.67:37017: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":112}
W20260812 08:00:36.898216  6261 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:00:33.715370 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.186.65:36089 (local address 127.0.186.68:49073)
0812 08:00:33.715840 (+   470us) negotiation.cc:107] Waiting for socket to connect
0812 08:00:33.715855 (+    15us) client_negotiation.cc:175] Beginning negotiation
0812 08:00:33.715926 (+    71us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 08:00:36.897686 (+3181760us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.186.65:36089: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":189,"thread_start_us":97,"threads_started":1}
I20260812 08:00:36.919839  6272 raft_consensus.cc:493] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28637390589f4854bc0c4d1916be3269)
I20260812 08:00:36.919996  6272 raft_consensus.cc:515] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } }
I20260812 08:00:36.920578  6272 leader_election.cc:290] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 81f7779748444854812a4497693823fd (127.0.186.66:35669), 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089), f9541f9d66774e57922e780d5740351c (127.0.186.67:37017)
I20260812 08:00:36.921041  5830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "e54ed57d03b94188a017315e98e22e29" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "81f7779748444854812a4497693823fd" is_pre_election: true
I20260812 08:00:36.921209  5830 raft_consensus.cc:2468] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e54ed57d03b94188a017315e98e22e29 in term 1.
W20260812 08:00:37.336462  6072 leader_election.cc:336] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f9541f9d66774e57922e780d5740351c (127.0.186.67:37017): Timed out: connection negotiation to 127.0.186.67:37017 for RPC RequestConsensusVote timed out after 0.207s (ON_OUTBOUND_QUEUE)
W20260812 08:00:37.336668  6068 leader_election.cc:336] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089): Timed out: connection negotiation to 127.0.186.65:36089 for RPC RequestConsensusVote timed out after 0.207s (ON_OUTBOUND_QUEUE)
I20260812 08:00:37.336760  6068 leader_election.cc:304] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 81f7779748444854812a4497693823fd, e54ed57d03b94188a017315e98e22e29; no voters: 28637390589f4854bc0c4d1916be3269, f9541f9d66774e57922e780d5740351c
I20260812 08:00:37.336969  6272 raft_consensus.cc:2749] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 08:00:37.498795  6272 raft_consensus.cc:493] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28637390589f4854bc0c4d1916be3269)
I20260812 08:00:37.498963  6272 raft_consensus.cc:515] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } }
I20260812 08:00:37.499481  6272 leader_election.cc:290] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 81f7779748444854812a4497693823fd (127.0.186.66:35669), 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089), f9541f9d66774e57922e780d5740351c (127.0.186.67:37017)
I20260812 08:00:37.500001  5830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "e54ed57d03b94188a017315e98e22e29" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "81f7779748444854812a4497693823fd" is_pre_election: true
I20260812 08:00:37.500149  5830 raft_consensus.cc:2468] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e54ed57d03b94188a017315e98e22e29 in term 1.
W20260812 08:00:37.615728  5758 leader_election.cc:336] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f9541f9d66774e57922e780d5740351c (127.0.186.67:37017): Timed out: connection negotiation to 127.0.186.67:37017 for RPC RequestConsensusVote timed out after 0.715s (ON_OUTBOUND_QUEUE)
W20260812 08:00:37.701090  5753 leader_election.cc:336] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089): Timed out: RequestConsensusVote RPC to 127.0.186.65:36089 timed out after 0.715s (SENT)
I20260812 08:00:37.701208  5753 leader_election.cc:304] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 81f7779748444854812a4497693823fd, e54ed57d03b94188a017315e98e22e29; no voters: 28637390589f4854bc0c4d1916be3269, f9541f9d66774e57922e780d5740351c
I20260812 08:00:37.701615  6276 raft_consensus.cc:2749] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 08:00:37.951858  6276 raft_consensus.cc:493] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28637390589f4854bc0c4d1916be3269)
I20260812 08:00:37.952050  6276 raft_consensus.cc:515] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } }
I20260812 08:00:37.952708  6276 leader_election.cc:290] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e54ed57d03b94188a017315e98e22e29 (127.0.186.68:36743), 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089), f9541f9d66774e57922e780d5740351c (127.0.186.67:37017)
I20260812 08:00:37.953377  6145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "81f7779748444854812a4497693823fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e54ed57d03b94188a017315e98e22e29" is_pre_election: true
I20260812 08:00:37.953574  6145 raft_consensus.cc:2468] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81f7779748444854812a4497693823fd in term 1.
W20260812 08:00:38.656253  6072 leader_election.cc:336] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f9541f9d66774e57922e780d5740351c (127.0.186.67:37017): Timed out: connection negotiation to 127.0.186.67:37017 for RPC RequestConsensusVote timed out after 0.577s (ON_OUTBOUND_QUEUE)
W20260812 08:00:38.656466  6068 leader_election.cc:336] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089): Timed out: connection negotiation to 127.0.186.65:36089 for RPC RequestConsensusVote timed out after 0.577s (ON_OUTBOUND_QUEUE)
I20260812 08:00:38.656562  6068 leader_election.cc:304] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 81f7779748444854812a4497693823fd, e54ed57d03b94188a017315e98e22e29; no voters: 28637390589f4854bc0c4d1916be3269, f9541f9d66774e57922e780d5740351c
I20260812 08:00:38.656874  6279 raft_consensus.cc:2749] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 08:00:39.059751  5758 leader_election.cc:336] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f9541f9d66774e57922e780d5740351c (127.0.186.67:37017): Timed out: connection negotiation to 127.0.186.67:37017 for RPC RequestConsensusVote timed out after 0.552s (ON_OUTBOUND_QUEUE)
W20260812 08:00:39.059900  5753 leader_election.cc:336] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089): Timed out: RequestConsensusVote RPC to 127.0.186.65:36089 timed out after 0.552s (SENT)
I20260812 08:00:39.060003  5753 leader_election.cc:304] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 81f7779748444854812a4497693823fd, e54ed57d03b94188a017315e98e22e29; no voters: 28637390589f4854bc0c4d1916be3269, f9541f9d66774e57922e780d5740351c
I20260812 08:00:39.060369  6280 raft_consensus.cc:2749] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 08:00:39.812085  6281 raft_consensus.cc:493] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28637390589f4854bc0c4d1916be3269)
I20260812 08:00:39.812278  6281 raft_consensus.cc:515] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } }
I20260812 08:00:39.812829  6281 leader_election.cc:290] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 81f7779748444854812a4497693823fd (127.0.186.66:35669), 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089), f9541f9d66774e57922e780d5740351c (127.0.186.67:37017)
I20260812 08:00:39.813337  5830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "e54ed57d03b94188a017315e98e22e29" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "81f7779748444854812a4497693823fd" is_pre_election: true
I20260812 08:00:39.813508  5830 raft_consensus.cc:2468] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e54ed57d03b94188a017315e98e22e29 in term 1.
W20260812 08:00:39.969820  6260 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:00:36.920900 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.186.67:37017 (local address 127.0.186.68:43133)
0812 08:00:36.921057 (+   157us) negotiation.cc:107] Waiting for socket to connect
0812 08:00:36.921069 (+    12us) client_negotiation.cc:175] Beginning negotiation
0812 08:00:36.921179 (+   110us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 08:00:39.969692 (+3048513us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.186.67:37017: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":89,"spinlock_wait_cycles":10624}
W20260812 08:00:39.970041  6072 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: Client connection negotiation failed: client connection to 127.0.186.67:37017: received 0 of 4 requested bytes [suppressed 11 similar messages]
W20260812 08:00:39.973690  6261 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:00:36.920840 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.186.65:36089 (local address 127.0.186.68:49163)
0812 08:00:36.921023 (+   183us) negotiation.cc:107] Waiting for socket to connect
0812 08:00:36.921040 (+    17us) client_negotiation.cc:175] Beginning negotiation
0812 08:00:36.921181 (+   141us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 08:00:39.973588 (+3052407us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.186.65:36089: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":84,"spinlock_wait_cycles":37376}
W20260812 08:00:39.973737  6238 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:00:36.899074 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.186.67:37017 (local address 127.0.186.66:39165)
0812 08:00:36.899186 (+   112us) negotiation.cc:107] Waiting for socket to connect
0812 08:00:36.899199 (+    13us) client_negotiation.cc:175] Beginning negotiation
0812 08:00:36.899296 (+    97us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 08:00:39.973641 (+3074345us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.186.67:37017: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 08:00:40.621572  6283 raft_consensus.cc:493] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28637390589f4854bc0c4d1916be3269)
I20260812 08:00:40.621728  6283 raft_consensus.cc:515] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } }
I20260812 08:00:40.622231  6283 leader_election.cc:290] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e54ed57d03b94188a017315e98e22e29 (127.0.186.68:36743), 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089), f9541f9d66774e57922e780d5740351c (127.0.186.67:37017)
I20260812 08:00:40.622692  6145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "81f7779748444854812a4497693823fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e54ed57d03b94188a017315e98e22e29" is_pre_election: true
I20260812 08:00:40.622840  6145 raft_consensus.cc:2468] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81f7779748444854812a4497693823fd in term 1.
W20260812 08:00:41.365741  6072 leader_election.cc:336] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f9541f9d66774e57922e780d5740351c (127.0.186.67:37017): Timed out: connection negotiation to 127.0.186.67:37017 for RPC RequestConsensusVote timed out after 1.394s (ON_OUTBOUND_QUEUE)
W20260812 08:00:41.370014  6068 leader_election.cc:336] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089): Timed out: connection negotiation to 127.0.186.65:36089 for RPC RequestConsensusVote timed out after 1.394s (ON_OUTBOUND_QUEUE)
I20260812 08:00:41.370172  6068 leader_election.cc:304] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 81f7779748444854812a4497693823fd, e54ed57d03b94188a017315e98e22e29; no voters: 28637390589f4854bc0c4d1916be3269, f9541f9d66774e57922e780d5740351c
I20260812 08:00:41.370559  6285 raft_consensus.cc:2749] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 08:00:41.447042  5753 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.186.65:36089 timed out after 0.823s (SENT) [suppressed 9 similar messages]
W20260812 08:00:42.161890  6289 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:00:42.160411 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.186.68:56139 (local address 127.0.186.67:37017)
0812 08:00:42.161350 (+   939us) server_negotiation.cc:207] Beginning negotiation
0812 08:00:42.161354 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 08:00:42.161370 (+    16us) server_negotiation.cc:408] Connection header received
0812 08:00:42.161417 (+    47us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 08:00:42.161420 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 08:00:42.161504 (+    84us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 08:00:42.161638 (+   134us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.68:56139: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":875,"thread_start_us":101,"threads_started":1}
W20260812 08:00:42.163218  6253 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:00:42.161541 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.186.66:39165 (local address 127.0.186.67:37017)
0812 08:00:42.162874 (+  1333us) server_negotiation.cc:207] Beginning negotiation
0812 08:00:42.162879 (+     5us) server_negotiation.cc:400] Waiting for connection header
0812 08:00:42.162892 (+    13us) server_negotiation.cc:408] Connection header received
0812 08:00:42.162930 (+    38us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 08:00:42.162933 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 08:00:42.162993 (+    60us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 08:00:42.163094 (+   101us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.66:39165: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":1259,"spinlock_wait_cycles":10624,"thread_start_us":85,"threads_started":1}
W20260812 08:00:42.164376  6292 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:00:42.161677 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.186.68:43133 (local address 127.0.186.67:37017)
0812 08:00:42.162881 (+  1204us) server_negotiation.cc:207] Beginning negotiation
0812 08:00:42.162885 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 08:00:42.162895 (+    10us) server_negotiation.cc:408] Connection header received
0812 08:00:42.162930 (+    35us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 08:00:42.162933 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 08:00:42.162993 (+    60us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 08:00:42.163094 (+   101us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.68:43133: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":1152,"spinlock_wait_cycles":11904,"thread_start_us":71,"threads_started":1}
W20260812 08:00:42.161891  6287 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:00:42.160353 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.186.66:35731 (local address 127.0.186.67:37017)
0812 08:00:42.160892 (+   539us) server_negotiation.cc:207] Beginning negotiation
0812 08:00:42.160896 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 08:00:42.160910 (+    14us) server_negotiation.cc:408] Connection header received
0812 08:00:42.160958 (+    48us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 08:00:42.160962 (+     4us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 08:00:42.161021 (+    59us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 08:00:42.161239 (+   218us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.66:35731: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":443}
I20260812 08:00:42.166738  6288 raft_consensus.cc:493] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28637390589f4854bc0c4d1916be3269)
I20260812 08:00:42.166921  6288 raft_consensus.cc:515] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } }
I20260812 08:00:42.169129  6288 leader_election.cc:290] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e54ed57d03b94188a017315e98e22e29 (127.0.186.68:36743), 81f7779748444854812a4497693823fd (127.0.186.66:35669), 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089)
I20260812 08:00:42.169646  5987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "81f7779748444854812a4497693823fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f9541f9d66774e57922e780d5740351c" is_pre_election: true
I20260812 08:00:42.169826  5987 raft_consensus.cc:2468] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81f7779748444854812a4497693823fd in term 1.
I20260812 08:00:42.170190  5758 leader_election.cc:304] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 81f7779748444854812a4497693823fd, e54ed57d03b94188a017315e98e22e29, f9541f9d66774e57922e780d5740351c; no voters: 
I20260812 08:00:42.170527  6302 raft_consensus.cc:2804] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 08:00:42.170625  6302 raft_consensus.cc:493] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 28637390589f4854bc0c4d1916be3269)
I20260812 08:00:42.170696  6302 raft_consensus.cc:3060] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:00:42.171839  6302 raft_consensus.cc:515] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } }
I20260812 08:00:42.172425  6302 leader_election.cc:290] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 2 election: Requested vote from peers e54ed57d03b94188a017315e98e22e29 (127.0.186.68:36743), 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089), f9541f9d66774e57922e780d5740351c (127.0.186.67:37017)
I20260812 08:00:42.172571  5987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "81f7779748444854812a4497693823fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f9541f9d66774e57922e780d5740351c"
I20260812 08:00:42.172684  5987 raft_consensus.cc:3060] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:00:42.172737  6145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "81f7779748444854812a4497693823fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e54ed57d03b94188a017315e98e22e29"
I20260812 08:00:42.172863  6145 raft_consensus.cc:3060] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:00:42.173761  5987 raft_consensus.cc:2468] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81f7779748444854812a4497693823fd in term 2.
I20260812 08:00:42.173873  6145 raft_consensus.cc:2468] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81f7779748444854812a4497693823fd in term 2.
I20260812 08:00:42.174168  5830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "f9541f9d66774e57922e780d5740351c" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "81f7779748444854812a4497693823fd" is_pre_election: true
I20260812 08:00:42.174327  5753 leader_election.cc:304] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 81f7779748444854812a4497693823fd, e54ed57d03b94188a017315e98e22e29, f9541f9d66774e57922e780d5740351c; no voters: 
I20260812 08:00:42.174338  5830 raft_consensus.cc:2393] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f9541f9d66774e57922e780d5740351c in current term 2: Already voted for candidate 81f7779748444854812a4497693823fd in this term.
I20260812 08:00:42.174520  6302 raft_consensus.cc:2804] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:00:42.174892  6302 raft_consensus.cc:697] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 2 LEADER]: Becoming Leader. State: Replica: 81f7779748444854812a4497693823fd, State: Running, Role: LEADER
I20260812 08:00:42.174889  6145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "f9541f9d66774e57922e780d5740351c" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e54ed57d03b94188a017315e98e22e29" is_pre_election: true
I20260812 08:00:42.175034  6145 raft_consensus.cc:2393] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f9541f9d66774e57922e780d5740351c in current term 2: Already voted for candidate 81f7779748444854812a4497693823fd in this term.
I20260812 08:00:42.175321  5909 leader_election.cc:304] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f9541f9d66774e57922e780d5740351c; no voters: 81f7779748444854812a4497693823fd, e54ed57d03b94188a017315e98e22e29
I20260812 08:00:42.175390  6302 consensus_queue.cc:237] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } }
I20260812 08:00:42.175509  6288 raft_consensus.cc:2749] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 08:00:42.177702  5507 catalog_manager.cc:5719] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd reported cstate change: term changed from 1 to 2, leader changed from 28637390589f4854bc0c4d1916be3269 (127.0.186.65) to 81f7779748444854812a4497693823fd (127.0.186.66). New cstate: current_term: 2 leader_uuid: "81f7779748444854812a4497693823fd" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } health_report { overall_health: UNKNOWN } } }
I20260812 08:00:42.220520  6145 raft_consensus.cc:1275] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 2 FOLLOWER]: Refusing update from remote peer 81f7779748444854812a4497693823fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260812 08:00:42.221131  6302 consensus_queue.cc:1048] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 08:00:42.235682  5987 raft_consensus.cc:1275] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [term 2 FOLLOWER]: Refusing update from remote peer 81f7779748444854812a4497693823fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260812 08:00:42.236167  6302 consensus_queue.cc:1048] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260812 08:00:42.272222  5753 leader_election.cc:336] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089): Timed out: RequestConsensusVote RPC to 127.0.186.65:36089 timed out after 0.823s (SENT)
W20260812 08:00:42.359863  5909 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.186.65:36089 timed out after 0.190s (SENT)
I20260812 08:00:42.372040  5675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "81f7779748444854812a4497693823fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "28637390589f4854bc0c4d1916be3269" is_pre_election: true
I20260812 08:00:42.372835  5677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "f9541f9d66774e57922e780d5740351c" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "28637390589f4854bc0c4d1916be3269" is_pre_election: true
I20260812 08:00:42.373624  5664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "81f7779748444854812a4497693823fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "28637390589f4854bc0c4d1916be3269" is_pre_election: true
I20260812 08:00:42.374172  5676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "f9541f9d66774e57922e780d5740351c" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "28637390589f4854bc0c4d1916be3269" is_pre_election: true
I20260812 08:00:42.374274  5663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "81f7779748444854812a4497693823fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "28637390589f4854bc0c4d1916be3269" is_pre_election: true
I20260812 08:00:42.374392  5830 raft_consensus.cc:1240] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 2 LEADER]: Rejecting Update request from peer 28637390589f4854bc0c4d1916be3269 for earlier term 1. Current term is 2. Ops: []
I20260812 08:00:42.374814  5671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "81f7779748444854812a4497693823fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "28637390589f4854bc0c4d1916be3269" is_pre_election: true
I20260812 08:00:42.375099  5667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "81f7779748444854812a4497693823fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "28637390589f4854bc0c4d1916be3269" is_pre_election: true
I20260812 08:00:42.375289  5674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "81f7779748444854812a4497693823fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "28637390589f4854bc0c4d1916be3269" is_pre_election: true
I20260812 08:00:42.375502  5673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "81f7779748444854812a4497693823fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "28637390589f4854bc0c4d1916be3269" is_pre_election: true
I20260812 08:00:42.375612  5668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "81f7779748444854812a4497693823fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "28637390589f4854bc0c4d1916be3269" is_pre_election: true
I20260812 08:00:42.376091  5661 raft_consensus.cc:3055] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 08:00:42.376155  5661 raft_consensus.cc:740] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28637390589f4854bc0c4d1916be3269, State: Running, Role: LEADER
I20260812 08:00:42.376142  5662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "81f7779748444854812a4497693823fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "28637390589f4854bc0c4d1916be3269"
I20260812 08:00:42.376317  5661 consensus_queue.cc:260] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } }
I20260812 08:00:42.376385  5665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "81f7779748444854812a4497693823fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "28637390589f4854bc0c4d1916be3269" is_pre_election: true
I20260812 08:00:42.376503  5661 raft_consensus.cc:3060] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:00:42.376626  5666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "81f7779748444854812a4497693823fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "28637390589f4854bc0c4d1916be3269" is_pre_election: true
I20260812 08:00:42.376854  5670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "81f7779748444854812a4497693823fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "28637390589f4854bc0c4d1916be3269" is_pre_election: true
I20260812 08:00:42.377143  5672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "81f7779748444854812a4497693823fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "28637390589f4854bc0c4d1916be3269" is_pre_election: true
I20260812 08:00:42.377849  5661 raft_consensus.cc:1275] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [term 2 FOLLOWER]: Refusing update from remote peer 81f7779748444854812a4497693823fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260812 08:00:42.379148  6303 consensus_queue.cc:1048] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260812 08:00:42.379537  6321 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:00:42.377397 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.186.68:49073 (local address 127.0.186.65:36089)
0812 08:00:42.378615 (+  1218us) server_negotiation.cc:207] Beginning negotiation
0812 08:00:42.378620 (+     5us) server_negotiation.cc:400] Waiting for connection header
0812 08:00:42.378635 (+    15us) server_negotiation.cc:408] Connection header received
0812 08:00:42.378678 (+    43us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 08:00:42.378681 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 08:00:42.378728 (+    47us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 08:00:42.378882 (+   154us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.68:49073: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":1133,"thread_start_us":117,"threads_started":1}
W20260812 08:00:42.380066  6321 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:00:42.377586 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.186.68:49163 (local address 127.0.186.65:36089)
0812 08:00:42.379768 (+  2182us) server_negotiation.cc:207] Beginning negotiation
0812 08:00:42.379772 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 08:00:42.379786 (+    14us) server_negotiation.cc:408] Connection header received
0812 08:00:42.379826 (+    40us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 08:00:42.379829 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 08:00:42.379868 (+    39us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 08:00:42.379969 (+   101us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.68:49163: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":2116,"thread_start_us":48,"threads_started":1}
I20260812 08:00:42.487495  6310 raft_consensus.cc:493] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 81f7779748444854812a4497693823fd)
I20260812 08:00:42.487665  6310 raft_consensus.cc:515] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } }
I20260812 08:00:42.488292  6310 leader_election.cc:290] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 81f7779748444854812a4497693823fd (127.0.186.66:35669), 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089), f9541f9d66774e57922e780d5740351c (127.0.186.67:37017)
I20260812 08:00:42.488902  5661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "e54ed57d03b94188a017315e98e22e29" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "28637390589f4854bc0c4d1916be3269" is_pre_election: true
I20260812 08:00:42.488901  5987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "e54ed57d03b94188a017315e98e22e29" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f9541f9d66774e57922e780d5740351c" is_pre_election: true
I20260812 08:00:42.489112  5661 raft_consensus.cc:2468] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e54ed57d03b94188a017315e98e22e29 in term 2.
I20260812 08:00:42.503748  6288 raft_consensus.cc:493] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 81f7779748444854812a4497693823fd)
I20260812 08:00:42.503887  6288 raft_consensus.cc:515] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } }
I20260812 08:00:42.504384  6288 leader_election.cc:290] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e54ed57d03b94188a017315e98e22e29 (127.0.186.68:36743), 81f7779748444854812a4497693823fd (127.0.186.66:35669), 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089)
I20260812 08:00:42.504882  6145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "f9541f9d66774e57922e780d5740351c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e54ed57d03b94188a017315e98e22e29" is_pre_election: true
I20260812 08:00:42.504884  5661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "f9541f9d66774e57922e780d5740351c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "28637390589f4854bc0c4d1916be3269" is_pre_election: true
I20260812 08:00:42.505043  5661 raft_consensus.cc:2468] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9541f9d66774e57922e780d5740351c in term 2.
I20260812 08:00:42.505043  6145 raft_consensus.cc:2468] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9541f9d66774e57922e780d5740351c in term 2.
I20260812 08:00:42.505425  5909 leader_election.cc:304] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28637390589f4854bc0c4d1916be3269, e54ed57d03b94188a017315e98e22e29, f9541f9d66774e57922e780d5740351c; no voters: 
I20260812 08:00:42.505611  6288 raft_consensus.cc:2804] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 08:00:42.505673  6288 raft_consensus.cc:493] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 81f7779748444854812a4497693823fd)
I20260812 08:00:42.505702  6288 raft_consensus.cc:3060] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [term 2 FOLLOWER]: Advancing to term 3
I20260812 08:00:42.506587  6288 raft_consensus.cc:515] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } }
I20260812 08:00:42.506971  6288 leader_election.cc:290] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [CANDIDATE]: Term 3 election: Requested vote from peers e54ed57d03b94188a017315e98e22e29 (127.0.186.68:36743), 81f7779748444854812a4497693823fd (127.0.186.66:35669), 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089)
I20260812 08:00:42.507287  6145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "f9541f9d66774e57922e780d5740351c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e54ed57d03b94188a017315e98e22e29"
I20260812 08:00:42.507350  5661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "f9541f9d66774e57922e780d5740351c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "28637390589f4854bc0c4d1916be3269"
I20260812 08:00:42.507413  6145 raft_consensus.cc:3060] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 2 FOLLOWER]: Advancing to term 3
I20260812 08:00:42.507455  5661 raft_consensus.cc:3060] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [term 2 FOLLOWER]: Advancing to term 3
I20260812 08:00:42.508359  5661 raft_consensus.cc:2468] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9541f9d66774e57922e780d5740351c in term 3.
I20260812 08:00:42.508358  6145 raft_consensus.cc:2468] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9541f9d66774e57922e780d5740351c in term 3.
I20260812 08:00:42.508675  5909 leader_election.cc:304] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28637390589f4854bc0c4d1916be3269, e54ed57d03b94188a017315e98e22e29, f9541f9d66774e57922e780d5740351c; no voters: 
I20260812 08:00:42.508849  6288 raft_consensus.cc:2804] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [term 3 FOLLOWER]: Leader election won for term 3
I20260812 08:00:42.509104  6288 raft_consensus.cc:697] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [term 3 LEADER]: Becoming Leader. State: Replica: f9541f9d66774e57922e780d5740351c, State: Running, Role: LEADER
I20260812 08:00:42.509397  6288 consensus_queue.cc:237] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } }
I20260812 08:00:42.511848  5507 catalog_manager.cc:5719] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c reported cstate change: term changed from 2 to 3, leader changed from 81f7779748444854812a4497693823fd (127.0.186.66) to f9541f9d66774e57922e780d5740351c (127.0.186.67). New cstate: current_term: 3 leader_uuid: "f9541f9d66774e57922e780d5740351c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } health_report { overall_health: HEALTHY } } }
I20260812 08:00:42.558688  6145 raft_consensus.cc:1275] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 3 FOLLOWER]: Refusing update from remote peer f9541f9d66774e57922e780d5740351c: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260812 08:00:42.559262  6288 consensus_queue.cc:1048] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 08:00:42.566509  5661 raft_consensus.cc:1275] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [term 3 FOLLOWER]: Refusing update from remote peer f9541f9d66774e57922e780d5740351c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260812 08:00:42.566939  6288 consensus_queue.cc:1048] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [LEADER]: Connected to new peer: Peer: permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 08:00:42.591742  6145 raft_consensus.cc:1240] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 3 FOLLOWER]: Rejecting Update request from peer 81f7779748444854812a4497693823fd for earlier term 2. Current term is 3. Ops: []
I20260812 08:00:42.591795  5661 raft_consensus.cc:1217] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [term 3 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[2.4-2.4]   Dedup: 2.4->[]
I20260812 08:00:42.591986  5661 raft_consensus.cc:1240] T 3e819892c07d46f29e0e6483dcce9cd0 P 28637390589f4854bc0c4d1916be3269 [term 3 FOLLOWER]: Rejecting Update request from peer 81f7779748444854812a4497693823fd for earlier term 2. Current term is 3. Ops: [2.4-2.4]
I20260812 08:00:42.592286  5987 raft_consensus.cc:1240] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [term 3 LEADER]: Rejecting Update request from peer 81f7779748444854812a4497693823fd for earlier term 2. Current term is 3. Ops: []
I20260812 08:00:42.592361  6303 consensus_queue.cc:1059] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 08:00:42.592804  5828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "f9541f9d66774e57922e780d5740351c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "81f7779748444854812a4497693823fd"
I20260812 08:00:42.592890  5829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "f9541f9d66774e57922e780d5740351c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "81f7779748444854812a4497693823fd" is_pre_election: true
I20260812 08:00:42.592860  6302 consensus_queue.cc:1059] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 08:00:42.593010  6302 consensus_queue.cc:1059] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 08:00:42.593087  6302 consensus_queue.cc:1243] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [LEADER]: Peer f9541f9d66774e57922e780d5740351c log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260812 08:00:42.593102  5830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e819892c07d46f29e0e6483dcce9cd0" candidate_uuid: "e54ed57d03b94188a017315e98e22e29" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "81f7779748444854812a4497693823fd" is_pre_election: true
I20260812 08:00:42.593320  5826 raft_consensus.cc:3055] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 2 LEADER]: Stepping down as leader of term 2
I20260812 08:00:42.593377  5826 raft_consensus.cc:740] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 81f7779748444854812a4497693823fd, State: Running, Role: LEADER
I20260812 08:00:42.593573  5826 consensus_queue.cc:260] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e54ed57d03b94188a017315e98e22e29" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 36743 } } peers { permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 } } peers { permanent_uuid: "28637390589f4854bc0c4d1916be3269" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36089 } } peers { permanent_uuid: "f9541f9d66774e57922e780d5740351c" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 37017 } attrs { } }
I20260812 08:00:42.593619  6071 leader_election.cc:304] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28637390589f4854bc0c4d1916be3269, e54ed57d03b94188a017315e98e22e29; no voters: 81f7779748444854812a4497693823fd, f9541f9d66774e57922e780d5740351c
I20260812 08:00:42.593817  6310 raft_consensus.cc:2749] T 3e819892c07d46f29e0e6483dcce9cd0 P e54ed57d03b94188a017315e98e22e29 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 08:00:42.593909  5826 raft_consensus.cc:3060] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 2 FOLLOWER]: Advancing to term 3
I20260812 08:00:42.594002   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 5581
I20260812 08:00:42.595005  5826 raft_consensus.cc:1275] T 3e819892c07d46f29e0e6483dcce9cd0 P 81f7779748444854812a4497693823fd [term 3 FOLLOWER]: Refusing update from remote peer f9541f9d66774e57922e780d5740351c: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260812 08:00:42.595628  6303 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260812 08:00:42.597332  6288 consensus_queue.cc:1048] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c [LEADER]: Connected to new peer: Peer: permanent_uuid: "81f7779748444854812a4497693823fd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 35669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 08:00:42.608639   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 5733
W20260812 08:00:42.614624  5909 consensus_peers.cc:597] T 3e819892c07d46f29e0e6483dcce9cd0 P f9541f9d66774e57922e780d5740351c -> Peer 28637390589f4854bc0c4d1916be3269 (127.0.186.65:36089): Couldn't send request to peer 28637390589f4854bc0c4d1916be3269. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:36089: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 08:00:42.620601   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 5890
I20260812 08:00:42.630357   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 6048
I20260812 08:00:42.639479   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 5476
2026-08-12T08:00:42Z chronyd exiting
[       OK ] RaftConsensusNonVoterITest.PromotedReplicaCanVote (10365 ms)
[ RUN      ] RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter
I20260812 08:00:42.667909   745 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:00:42.667958   745 ts_itest-base.cc:116] --------------
I20260812 08:00:42.667973   745 ts_itest-base.cc:117] 4 tablet servers
I20260812 08:00:42.667985   745 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:00:42.667999   745 ts_itest-base.cc:119] --------------
2026-08-12T08:00:42Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:00:42Z Disabled control of system clock
I20260812 08:00:42.676961   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:41969
--webserver_interface=127.0.186.126
--webserver_port=0
--builtin_ntp_servers=127.0.186.84:43311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:41969
--raft_prepare_replacement_before_eviction=true
--catalog_manager_evict_excess_replicas=false with env {}
W20260812 08:00:42.780848  6339 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:42.781078  6339 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:42.781133  6339 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:42.784395  6339 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:00:42.784454  6339 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:42.784480  6339 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:00:42.784503  6339 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:00:42.788312  6339 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:43311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--catalog_manager_evict_excess_replicas=false
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:41969
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:41969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.6339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:42.789364  6339 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:42.790501  6339 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:42.796109  6344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:42.796177  6345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:42.796167  6339 server_base.cc:1061] running on GCE node
W20260812 08:00:42.796101  6348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:42.796761  6339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:42.797657  6339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:42.798806  6339 hybrid_clock.cc:648] HybridClock initialized: now 1786521642798798 us; error 23 us; skew 500 ppm
I20260812 08:00:42.800474  6339 webserver.cc:533] Webserver started at http://127.0.186.126:36289/ using document root <none> and password file <none>
I20260812 08:00:42.801014  6339 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:42.801074  6339 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:42.801265  6339 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:42.802783  6339 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "c96558a064714bc1b173a21f1a2e2a35"
format_stamp: "Formatted at 2026-08-12 08:00:42 on dist-test-slave-49h3"
I20260812 08:00:42.803251  6339 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "c96558a064714bc1b173a21f1a2e2a35"
format_stamp: "Formatted at 2026-08-12 08:00:42 on dist-test-slave-49h3"
I20260812 08:00:42.806509  6339 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:00:42.808548  6354 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:42.809510  6339 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:00:42.809615  6339 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "c96558a064714bc1b173a21f1a2e2a35"
format_stamp: "Formatted at 2026-08-12 08:00:42 on dist-test-slave-49h3"
I20260812 08:00:42.809697  6339 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:42.832535  6339 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:42.833191  6339 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:42.833351  6339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:42.840536  6339 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:41969
I20260812 08:00:42.840540  6412 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:41969 every 8 connection(s)
I20260812 08:00:42.841774  6339 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:00:42.842052   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 6339
I20260812 08:00:42.842160   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:00:42.844954  6414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:42.850914  6414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c96558a064714bc1b173a21f1a2e2a35: Bootstrap starting.
I20260812 08:00:42.853099  6414 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c96558a064714bc1b173a21f1a2e2a35: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:42.853855  6414 log.cc:826] T 00000000000000000000000000000000 P c96558a064714bc1b173a21f1a2e2a35: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:42.855445  6414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c96558a064714bc1b173a21f1a2e2a35: No bootstrap required, opened a new log
I20260812 08:00:42.857879  6414 raft_consensus.cc:359] T 00000000000000000000000000000000 P c96558a064714bc1b173a21f1a2e2a35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96558a064714bc1b173a21f1a2e2a35" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41969 } }
I20260812 08:00:42.858072  6414 raft_consensus.cc:385] T 00000000000000000000000000000000 P c96558a064714bc1b173a21f1a2e2a35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:42.858135  6414 raft_consensus.cc:740] T 00000000000000000000000000000000 P c96558a064714bc1b173a21f1a2e2a35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c96558a064714bc1b173a21f1a2e2a35, State: Initialized, Role: FOLLOWER
I20260812 08:00:42.858530  6414 consensus_queue.cc:260] T 00000000000000000000000000000000 P c96558a064714bc1b173a21f1a2e2a35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96558a064714bc1b173a21f1a2e2a35" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41969 } }
I20260812 08:00:42.858675  6414 raft_consensus.cc:399] T 00000000000000000000000000000000 P c96558a064714bc1b173a21f1a2e2a35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:00:42.858739  6414 raft_consensus.cc:493] T 00000000000000000000000000000000 P c96558a064714bc1b173a21f1a2e2a35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:00:42.858842  6414 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c96558a064714bc1b173a21f1a2e2a35 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:42.859587  6414 raft_consensus.cc:515] T 00000000000000000000000000000000 P c96558a064714bc1b173a21f1a2e2a35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96558a064714bc1b173a21f1a2e2a35" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41969 } }
I20260812 08:00:42.859903  6414 leader_election.cc:304] T 00000000000000000000000000000000 P c96558a064714bc1b173a21f1a2e2a35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c96558a064714bc1b173a21f1a2e2a35; no voters: 
I20260812 08:00:42.860162  6414 leader_election.cc:290] T 00000000000000000000000000000000 P c96558a064714bc1b173a21f1a2e2a35 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:00:42.860257  6420 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c96558a064714bc1b173a21f1a2e2a35 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:00:42.860456  6420 raft_consensus.cc:697] T 00000000000000000000000000000000 P c96558a064714bc1b173a21f1a2e2a35 [term 1 LEADER]: Becoming Leader. State: Replica: c96558a064714bc1b173a21f1a2e2a35, State: Running, Role: LEADER
I20260812 08:00:42.860733  6420 consensus_queue.cc:237] T 00000000000000000000000000000000 P c96558a064714bc1b173a21f1a2e2a35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96558a064714bc1b173a21f1a2e2a35" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41969 } }
I20260812 08:00:42.861249  6414 sys_catalog.cc:565] T 00000000000000000000000000000000 P c96558a064714bc1b173a21f1a2e2a35 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:00:42.862515  6423 sys_catalog.cc:455] T 00000000000000000000000000000000 P c96558a064714bc1b173a21f1a2e2a35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c96558a064714bc1b173a21f1a2e2a35. Latest consensus state: current_term: 1 leader_uuid: "c96558a064714bc1b173a21f1a2e2a35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96558a064714bc1b173a21f1a2e2a35" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41969 } } }
I20260812 08:00:42.862677  6423 sys_catalog.cc:458] T 00000000000000000000000000000000 P c96558a064714bc1b173a21f1a2e2a35 [sys.catalog]: This master's current role is: LEADER
I20260812 08:00:42.862725  6421 sys_catalog.cc:455] T 00000000000000000000000000000000 P c96558a064714bc1b173a21f1a2e2a35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c96558a064714bc1b173a21f1a2e2a35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96558a064714bc1b173a21f1a2e2a35" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41969 } } }
I20260812 08:00:42.862802  6421 sys_catalog.cc:458] T 00000000000000000000000000000000 P c96558a064714bc1b173a21f1a2e2a35 [sys.catalog]: This master's current role is: LEADER
I20260812 08:00:42.863108  6431 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:00:42.865844  6431 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:00:42.870936  6431 catalog_manager.cc:1383] Generated new cluster ID: 18fcd2a00b4545faa5100bd65e80b72d
I20260812 08:00:42.870993  6431 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:00:42.903245  6431 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:00:42.904155  6431 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:00:42.913535  6431 catalog_manager.cc:6092] T 00000000000000000000000000000000 P c96558a064714bc1b173a21f1a2e2a35: Generated new TSK 0
I20260812 08:00:42.914155  6431 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:00:42.917495   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:0
--local_ip_for_outbound_sockets=127.0.186.65
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:41969
--builtin_ntp_servers=127.0.186.84:43311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=5 with env {}
W20260812 08:00:43.027485  6443 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:43.027698  6443 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:43.027741  6443 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:43.031065  6443 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:43.031189  6443 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:00:43.035000  6443 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:43311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=5
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:41969
--never_fsync=true
--heap_profile_path=/tmp/kudu.6443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:43.036033  6443 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:43.037153  6443 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:43.043321  6450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:43.043367  6449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:43.043373  6453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:43.043864  6443 server_base.cc:1061] running on GCE node
I20260812 08:00:43.044226  6443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:43.044770  6443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:43.045924  6443 hybrid_clock.cc:648] HybridClock initialized: now 1786521643045906 us; error 31 us; skew 500 ppm
I20260812 08:00:43.047827  6443 webserver.cc:533] Webserver started at http://127.0.186.65:44399/ using document root <none> and password file <none>
I20260812 08:00:43.048411  6443 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:43.048472  6443 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:43.048669  6443 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:43.050269  6443 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "2d2face0e26e42d8806a902c15648764"
format_stamp: "Formatted at 2026-08-12 08:00:43 on dist-test-slave-49h3"
I20260812 08:00:43.050771  6443 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "2d2face0e26e42d8806a902c15648764"
format_stamp: "Formatted at 2026-08-12 08:00:43 on dist-test-slave-49h3"
I20260812 08:00:43.054190  6443 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.001s
I20260812 08:00:43.056342  6460 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:43.057331  6443 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 08:00:43.057464  6443 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "2d2face0e26e42d8806a902c15648764"
format_stamp: "Formatted at 2026-08-12 08:00:43 on dist-test-slave-49h3"
I20260812 08:00:43.057565  6443 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:43.077574  6443 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:43.078411  6443 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:43.078612  6443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:43.079231  6443 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:43.080317  6443 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:43.080370  6443 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:43.080426  6443 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:43.080453  6443 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:43.089462  6443 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:45739
I20260812 08:00:43.089507  6594 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:45739 every 8 connection(s)
I20260812 08:00:43.090479  6443 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:00:43.094365   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 6443
I20260812 08:00:43.094484   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:00:43.096745   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.66:0
--local_ip_for_outbound_sockets=127.0.186.66
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:41969
--builtin_ntp_servers=127.0.186.84:43311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=5 with env {}
I20260812 08:00:43.099224  6595 heartbeater.cc:344] Connected to a master server at 127.0.186.126:41969
I20260812 08:00:43.099485  6595 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:43.100016  6595 heartbeater.cc:507] Master 127.0.186.126:41969 requested a full tablet report, sending...
I20260812 08:00:43.101809  6374 ts_manager.cc:194] Registered new tserver with Master: 2d2face0e26e42d8806a902c15648764 (127.0.186.65:45739)
I20260812 08:00:43.103510  6374 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.65:33437
W20260812 08:00:43.204751  6600 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:43.204944  6600 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:43.204983  6600 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:43.208225  6600 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:43.208324  6600 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.66
I20260812 08:00:43.211966  6600 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:43311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=5
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:41969
--never_fsync=true
--heap_profile_path=/tmp/kudu.6600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:43.212991  6600 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:43.214186  6600 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:43.220281  6607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:43.220352  6609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:43.220286  6611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:43.220587  6600 server_base.cc:1061] running on GCE node
I20260812 08:00:43.220898  6600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:43.221392  6600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:43.222563  6600 hybrid_clock.cc:648] HybridClock initialized: now 1786521643222543 us; error 31 us; skew 500 ppm
I20260812 08:00:43.224192  6600 webserver.cc:533] Webserver started at http://127.0.186.66:44061/ using document root <none> and password file <none>
I20260812 08:00:43.224721  6600 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:43.224779  6600 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:43.224957  6600 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:43.226482  6600 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "2c0e5a656f71453f897427789d11f50b"
format_stamp: "Formatted at 2026-08-12 08:00:43 on dist-test-slave-49h3"
I20260812 08:00:43.226936  6600 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "2c0e5a656f71453f897427789d11f50b"
format_stamp: "Formatted at 2026-08-12 08:00:43 on dist-test-slave-49h3"
I20260812 08:00:43.230157  6600 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:00:43.232275  6620 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:43.233201  6600 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:00:43.233314  6600 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "2c0e5a656f71453f897427789d11f50b"
format_stamp: "Formatted at 2026-08-12 08:00:43 on dist-test-slave-49h3"
I20260812 08:00:43.233410  6600 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:43.248569  6600 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:43.249194  6600 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:43.249341  6600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:43.249948  6600 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:43.250979  6600 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:43.251029  6600 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:43.251067  6600 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:43.251087  6600 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:43.259848  6600 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.66:43239
I20260812 08:00:43.259881  6751 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.66:43239 every 8 connection(s)
I20260812 08:00:43.260855  6600 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:00:43.263325   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 6600
I20260812 08:00:43.263440   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:00:43.265933   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.67:0
--local_ip_for_outbound_sockets=127.0.186.67
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:41969
--builtin_ntp_servers=127.0.186.84:43311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=5 with env {}
I20260812 08:00:43.269977  6752 heartbeater.cc:344] Connected to a master server at 127.0.186.126:41969
I20260812 08:00:43.270251  6752 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:43.270816  6752 heartbeater.cc:507] Master 127.0.186.126:41969 requested a full tablet report, sending...
I20260812 08:00:43.271903  6374 ts_manager.cc:194] Registered new tserver with Master: 2c0e5a656f71453f897427789d11f50b (127.0.186.66:43239)
I20260812 08:00:43.272464  6374 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.66:58561
W20260812 08:00:43.370855  6757 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:43.371065  6757 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:43.371112  6757 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:43.374368  6757 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:43.374475  6757 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.67
I20260812 08:00:43.378175  6757 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:43311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=5
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:41969
--never_fsync=true
--heap_profile_path=/tmp/kudu.6757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:43.379236  6757 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:43.380318  6757 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:43.386271  6765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:43.386320  6763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:43.386411  6762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:43.386610  6757 server_base.cc:1061] running on GCE node
I20260812 08:00:43.386895  6757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:43.387358  6757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:43.388499  6757 hybrid_clock.cc:648] HybridClock initialized: now 1786521643388483 us; error 28 us; skew 500 ppm
I20260812 08:00:43.390107  6757 webserver.cc:533] Webserver started at http://127.0.186.67:34027/ using document root <none> and password file <none>
I20260812 08:00:43.390654  6757 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:43.390708  6757 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:43.390892  6757 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:43.392321  6757 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "14c944622556420bb0c8c990ed25b3a8"
format_stamp: "Formatted at 2026-08-12 08:00:43 on dist-test-slave-49h3"
I20260812 08:00:43.392766  6757 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "14c944622556420bb0c8c990ed25b3a8"
format_stamp: "Formatted at 2026-08-12 08:00:43 on dist-test-slave-49h3"
I20260812 08:00:43.395793  6757 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.000s	sys 0.005s
I20260812 08:00:43.397725  6772 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:43.398604  6757 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260812 08:00:43.398697  6757 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "14c944622556420bb0c8c990ed25b3a8"
format_stamp: "Formatted at 2026-08-12 08:00:43 on dist-test-slave-49h3"
I20260812 08:00:43.398778  6757 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:43.417925  6757 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:43.418520  6757 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:43.418687  6757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:43.419230  6757 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:43.420203  6757 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:43.420251  6757 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:43.420302  6757 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:43.420327  6757 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:43.428820  6757 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.67:42047
I20260812 08:00:43.428864  6907 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.67:42047 every 8 connection(s)
I20260812 08:00:43.429783  6757 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:00:43.432736   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 6757
I20260812 08:00:43.432827   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:00:43.435005   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.68:0
--local_ip_for_outbound_sockets=127.0.186.68
--webserver_interface=127.0.186.68
--webserver_port=0
--tserver_master_addrs=127.0.186.126:41969
--builtin_ntp_servers=127.0.186.84:43311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=5 with env {}
I20260812 08:00:43.437877  6908 heartbeater.cc:344] Connected to a master server at 127.0.186.126:41969
I20260812 08:00:43.438122  6908 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:43.438683  6908 heartbeater.cc:507] Master 127.0.186.126:41969 requested a full tablet report, sending...
I20260812 08:00:43.439829  6374 ts_manager.cc:194] Registered new tserver with Master: 14c944622556420bb0c8c990ed25b3a8 (127.0.186.67:42047)
I20260812 08:00:43.440368  6374 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.67:54275
W20260812 08:00:43.540741  6912 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:43.540936  6912 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:43.540980  6912 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:43.544229  6912 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:43.544341  6912 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.68
I20260812 08:00:43.548418  6912 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:43311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=5
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.0.186.68
--webserver_port=0
--tserver_master_addrs=127.0.186.126:41969
--never_fsync=true
--heap_profile_path=/tmp/kudu.6912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.68

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:43.549819  6912 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:43.551236  6912 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:43.558794  6920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:43.558914  6919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:43.558801  6922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:43.559259  6912 server_base.cc:1061] running on GCE node
I20260812 08:00:43.559639  6912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:43.560263  6912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:43.561435  6912 hybrid_clock.cc:648] HybridClock initialized: now 1786521643561419 us; error 34 us; skew 500 ppm
I20260812 08:00:43.563485  6912 webserver.cc:533] Webserver started at http://127.0.186.68:43683/ using document root <none> and password file <none>
I20260812 08:00:43.564185  6912 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:43.564258  6912 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:43.564476  6912 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:43.566485  6912 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "bcad432fa2c9426b9be411ad57aac42b"
format_stamp: "Formatted at 2026-08-12 08:00:43 on dist-test-slave-49h3"
I20260812 08:00:43.567059  6912 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "bcad432fa2c9426b9be411ad57aac42b"
format_stamp: "Formatted at 2026-08-12 08:00:43 on dist-test-slave-49h3"
I20260812 08:00:43.571277  6912 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:00:43.573874  6929 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:43.575028  6912 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:00:43.575146  6912 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "bcad432fa2c9426b9be411ad57aac42b"
format_stamp: "Formatted at 2026-08-12 08:00:43 on dist-test-slave-49h3"
I20260812 08:00:43.575246  6912 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:43.588955  6912 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:43.589715  6912 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:43.589900  6912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:43.590576  6912 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:43.591823  6912 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:43.591883  6912 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:43.591943  6912 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:43.591972  6912 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:43.603386  6912 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.68:38495
I20260812 08:00:43.603506  7064 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.68:38495 every 8 connection(s)
I20260812 08:00:43.604601  6912 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20260812 08:00:43.612059   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 6912
I20260812 08:00:43.612203   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20260812 08:00:43.616796  7065 heartbeater.cc:344] Connected to a master server at 127.0.186.126:41969
I20260812 08:00:43.617039  7065 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:43.617614  7065 heartbeater.cc:507] Master 127.0.186.126:41969 requested a full tablet report, sending...
I20260812 08:00:43.618719  6374 ts_manager.cc:194] Registered new tserver with Master: bcad432fa2c9426b9be411ad57aac42b (127.0.186.68:38495)
I20260812 08:00:43.619511  6374 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.68:33367
I20260812 08:00:43.626514   745 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20260812 08:00:43.635840  6374 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36536:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
I20260812 08:00:43.658108  6521 tablet_service.cc:1511] Processing CreateTablet for tablet 5bcffa11ab1447feb10888f069357cd6 (DEFAULT_TABLE table=TestTable [id=9e8f5232bf1c439aa93a0cd97dbbb820]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:43.659278  6521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bcffa11ab1447feb10888f069357cd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:43.659128  6829 tablet_service.cc:1511] Processing CreateTablet for tablet 5bcffa11ab1447feb10888f069357cd6 (DEFAULT_TABLE table=TestTable [id=9e8f5232bf1c439aa93a0cd97dbbb820]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:43.659128  6675 tablet_service.cc:1511] Processing CreateTablet for tablet 5bcffa11ab1447feb10888f069357cd6 (DEFAULT_TABLE table=TestTable [id=9e8f5232bf1c439aa93a0cd97dbbb820]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:43.660058  6829 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bcffa11ab1447feb10888f069357cd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:43.660058  6675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bcffa11ab1447feb10888f069357cd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:43.666734  7085 tablet_bootstrap.cc:492] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764: Bootstrap starting.
I20260812 08:00:43.666738  7090 tablet_bootstrap.cc:492] T 5bcffa11ab1447feb10888f069357cd6 P 2c0e5a656f71453f897427789d11f50b: Bootstrap starting.
I20260812 08:00:43.667850  7089 tablet_bootstrap.cc:492] T 5bcffa11ab1447feb10888f069357cd6 P 14c944622556420bb0c8c990ed25b3a8: Bootstrap starting.
I20260812 08:00:43.669162  7090 tablet_bootstrap.cc:654] T 5bcffa11ab1447feb10888f069357cd6 P 2c0e5a656f71453f897427789d11f50b: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:43.669996  7089 tablet_bootstrap.cc:654] T 5bcffa11ab1447feb10888f069357cd6 P 14c944622556420bb0c8c990ed25b3a8: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:43.670022  7090 log.cc:826] T 5bcffa11ab1447feb10888f069357cd6 P 2c0e5a656f71453f897427789d11f50b: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:43.670807  7089 log.cc:826] T 5bcffa11ab1447feb10888f069357cd6 P 14c944622556420bb0c8c990ed25b3a8: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:43.671103  7085 tablet_bootstrap.cc:654] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:43.671905  7090 tablet_bootstrap.cc:492] T 5bcffa11ab1447feb10888f069357cd6 P 2c0e5a656f71453f897427789d11f50b: No bootstrap required, opened a new log
I20260812 08:00:43.672107  7090 ts_tablet_manager.cc:1403] T 5bcffa11ab1447feb10888f069357cd6 P 2c0e5a656f71453f897427789d11f50b: Time spent bootstrapping tablet: real 0.006s	user 0.004s	sys 0.000s
I20260812 08:00:43.672448  7089 tablet_bootstrap.cc:492] T 5bcffa11ab1447feb10888f069357cd6 P 14c944622556420bb0c8c990ed25b3a8: No bootstrap required, opened a new log
I20260812 08:00:43.672608  7089 ts_tablet_manager.cc:1403] T 5bcffa11ab1447feb10888f069357cd6 P 14c944622556420bb0c8c990ed25b3a8: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:00:43.671904  7085 log.cc:826] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:43.674821  7090 raft_consensus.cc:359] T 5bcffa11ab1447feb10888f069357cd6 P 2c0e5a656f71453f897427789d11f50b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14c944622556420bb0c8c990ed25b3a8" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42047 } } peers { permanent_uuid: "2c0e5a656f71453f897427789d11f50b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43239 } } peers { permanent_uuid: "2d2face0e26e42d8806a902c15648764" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45739 } }
I20260812 08:00:43.675001  7090 raft_consensus.cc:385] T 5bcffa11ab1447feb10888f069357cd6 P 2c0e5a656f71453f897427789d11f50b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:43.675057  7090 raft_consensus.cc:740] T 5bcffa11ab1447feb10888f069357cd6 P 2c0e5a656f71453f897427789d11f50b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0e5a656f71453f897427789d11f50b, State: Initialized, Role: FOLLOWER
I20260812 08:00:43.675576  7090 consensus_queue.cc:260] T 5bcffa11ab1447feb10888f069357cd6 P 2c0e5a656f71453f897427789d11f50b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14c944622556420bb0c8c990ed25b3a8" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42047 } } peers { permanent_uuid: "2c0e5a656f71453f897427789d11f50b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43239 } } peers { permanent_uuid: "2d2face0e26e42d8806a902c15648764" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45739 } }
I20260812 08:00:43.675669  7089 raft_consensus.cc:359] T 5bcffa11ab1447feb10888f069357cd6 P 14c944622556420bb0c8c990ed25b3a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14c944622556420bb0c8c990ed25b3a8" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42047 } } peers { permanent_uuid: "2c0e5a656f71453f897427789d11f50b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43239 } } peers { permanent_uuid: "2d2face0e26e42d8806a902c15648764" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45739 } }
I20260812 08:00:43.675863  7089 raft_consensus.cc:385] T 5bcffa11ab1447feb10888f069357cd6 P 14c944622556420bb0c8c990ed25b3a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:43.675907  7089 raft_consensus.cc:740] T 5bcffa11ab1447feb10888f069357cd6 P 14c944622556420bb0c8c990ed25b3a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14c944622556420bb0c8c990ed25b3a8, State: Initialized, Role: FOLLOWER
I20260812 08:00:43.676363  7089 consensus_queue.cc:260] T 5bcffa11ab1447feb10888f069357cd6 P 14c944622556420bb0c8c990ed25b3a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14c944622556420bb0c8c990ed25b3a8" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42047 } } peers { permanent_uuid: "2c0e5a656f71453f897427789d11f50b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43239 } } peers { permanent_uuid: "2d2face0e26e42d8806a902c15648764" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45739 } }
I20260812 08:00:43.676491  7090 ts_tablet_manager.cc:1434] T 5bcffa11ab1447feb10888f069357cd6 P 2c0e5a656f71453f897427789d11f50b: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:00:43.676508  7085 tablet_bootstrap.cc:492] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764: No bootstrap required, opened a new log
I20260812 08:00:43.676676  7085 ts_tablet_manager.cc:1403] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764: Time spent bootstrapping tablet: real 0.010s	user 0.004s	sys 0.000s
I20260812 08:00:43.676867  6752 heartbeater.cc:499] Master 127.0.186.126:41969 was elected leader, sending a full tablet report...
I20260812 08:00:43.677575  6908 heartbeater.cc:499] Master 127.0.186.126:41969 was elected leader, sending a full tablet report...
I20260812 08:00:43.679951  7085 raft_consensus.cc:359] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14c944622556420bb0c8c990ed25b3a8" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42047 } } peers { permanent_uuid: "2c0e5a656f71453f897427789d11f50b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43239 } } peers { permanent_uuid: "2d2face0e26e42d8806a902c15648764" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45739 } }
I20260812 08:00:43.680179  7085 raft_consensus.cc:385] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:43.680248  7085 raft_consensus.cc:740] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d2face0e26e42d8806a902c15648764, State: Initialized, Role: FOLLOWER
I20260812 08:00:43.680661  7089 ts_tablet_manager.cc:1434] T 5bcffa11ab1447feb10888f069357cd6 P 14c944622556420bb0c8c990ed25b3a8: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260812 08:00:43.680646  7085 consensus_queue.cc:260] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14c944622556420bb0c8c990ed25b3a8" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42047 } } peers { permanent_uuid: "2c0e5a656f71453f897427789d11f50b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43239 } } peers { permanent_uuid: "2d2face0e26e42d8806a902c15648764" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45739 } }
I20260812 08:00:43.681710  7085 ts_tablet_manager.cc:1434] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 08:00:43.682032  6595 heartbeater.cc:499] Master 127.0.186.126:41969 was elected leader, sending a full tablet report...
I20260812 08:00:43.755246  7098 raft_consensus.cc:493] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:00:43.755497  7098 raft_consensus.cc:515] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14c944622556420bb0c8c990ed25b3a8" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42047 } } peers { permanent_uuid: "2c0e5a656f71453f897427789d11f50b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43239 } } peers { permanent_uuid: "2d2face0e26e42d8806a902c15648764" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45739 } }
I20260812 08:00:43.756945  7098 leader_election.cc:290] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14c944622556420bb0c8c990ed25b3a8 (127.0.186.67:42047), 2c0e5a656f71453f897427789d11f50b (127.0.186.66:43239)
I20260812 08:00:43.761477  6855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bcffa11ab1447feb10888f069357cd6" candidate_uuid: "2d2face0e26e42d8806a902c15648764" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14c944622556420bb0c8c990ed25b3a8" is_pre_election: true
I20260812 08:00:43.761842  6855 raft_consensus.cc:2468] T 5bcffa11ab1447feb10888f069357cd6 P 14c944622556420bb0c8c990ed25b3a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d2face0e26e42d8806a902c15648764 in term 0.
W20260812 08:00:43.761989  6755 tablet.cc:2406] T 5bcffa11ab1447feb10888f069357cd6 P 2c0e5a656f71453f897427789d11f50b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:00:43.762012  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bcffa11ab1447feb10888f069357cd6" candidate_uuid: "2d2face0e26e42d8806a902c15648764" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0e5a656f71453f897427789d11f50b" is_pre_election: true
I20260812 08:00:43.762382  6463 leader_election.cc:304] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14c944622556420bb0c8c990ed25b3a8, 2d2face0e26e42d8806a902c15648764; no voters: 
I20260812 08:00:43.762548  6701 raft_consensus.cc:2468] T 5bcffa11ab1447feb10888f069357cd6 P 2c0e5a656f71453f897427789d11f50b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d2face0e26e42d8806a902c15648764 in term 0.
I20260812 08:00:43.762634  7098 raft_consensus.cc:2804] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:00:43.762732  7098 raft_consensus.cc:493] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:00:43.762782  7098 raft_consensus.cc:3060] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:43.764040  7098 raft_consensus.cc:515] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14c944622556420bb0c8c990ed25b3a8" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42047 } } peers { permanent_uuid: "2c0e5a656f71453f897427789d11f50b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43239 } } peers { permanent_uuid: "2d2face0e26e42d8806a902c15648764" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45739 } }
I20260812 08:00:43.764467  7098 leader_election.cc:290] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [CANDIDATE]: Term 1 election: Requested vote from peers 14c944622556420bb0c8c990ed25b3a8 (127.0.186.67:42047), 2c0e5a656f71453f897427789d11f50b (127.0.186.66:43239)
I20260812 08:00:43.764873  6855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bcffa11ab1447feb10888f069357cd6" candidate_uuid: "2d2face0e26e42d8806a902c15648764" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14c944622556420bb0c8c990ed25b3a8"
I20260812 08:00:43.764901  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bcffa11ab1447feb10888f069357cd6" candidate_uuid: "2d2face0e26e42d8806a902c15648764" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0e5a656f71453f897427789d11f50b"
I20260812 08:00:43.764982  6855 raft_consensus.cc:3060] T 5bcffa11ab1447feb10888f069357cd6 P 14c944622556420bb0c8c990ed25b3a8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:43.765021  6701 raft_consensus.cc:3060] T 5bcffa11ab1447feb10888f069357cd6 P 2c0e5a656f71453f897427789d11f50b [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:43.766242  6855 raft_consensus.cc:2468] T 5bcffa11ab1447feb10888f069357cd6 P 14c944622556420bb0c8c990ed25b3a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d2face0e26e42d8806a902c15648764 in term 1.
I20260812 08:00:43.766537  6463 leader_election.cc:304] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14c944622556420bb0c8c990ed25b3a8, 2d2face0e26e42d8806a902c15648764; no voters: 
I20260812 08:00:43.766623  6701 raft_consensus.cc:2468] T 5bcffa11ab1447feb10888f069357cd6 P 2c0e5a656f71453f897427789d11f50b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d2face0e26e42d8806a902c15648764 in term 1.
I20260812 08:00:43.766901  7098 raft_consensus.cc:2804] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:00:43.767175  7098 raft_consensus.cc:697] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [term 1 LEADER]: Becoming Leader. State: Replica: 2d2face0e26e42d8806a902c15648764, State: Running, Role: LEADER
I20260812 08:00:43.767539  7098 consensus_queue.cc:237] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14c944622556420bb0c8c990ed25b3a8" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42047 } } peers { permanent_uuid: "2c0e5a656f71453f897427789d11f50b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43239 } } peers { permanent_uuid: "2d2face0e26e42d8806a902c15648764" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45739 } }
I20260812 08:00:43.771468  6372 catalog_manager.cc:5719] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d2face0e26e42d8806a902c15648764 (127.0.186.65). New cstate: current_term: 1 leader_uuid: "2d2face0e26e42d8806a902c15648764" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14c944622556420bb0c8c990ed25b3a8" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0e5a656f71453f897427789d11f50b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d2face0e26e42d8806a902c15648764" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45739 } health_report { overall_health: HEALTHY } } }
I20260812 08:00:43.805223   745 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20260812 08:00:43.806733   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 2d2face0e26e42d8806a902c15648764 to finish bootstrapping
I20260812 08:00:43.812032   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 2c0e5a656f71453f897427789d11f50b to finish bootstrapping
I20260812 08:00:43.816793   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 14c944622556420bb0c8c990ed25b3a8 to finish bootstrapping
W20260812 08:00:43.841770  6596 tablet.cc:2406] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:00:43.930953  6910 tablet.cc:2406] T 5bcffa11ab1447feb10888f069357cd6 P 14c944622556420bb0c8c990ed25b3a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:00:44.195942  7098 consensus_queue.cc:1048] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14c944622556420bb0c8c990ed25b3a8" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:00:44.201331  7108 consensus_queue.cc:1048] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0e5a656f71453f897427789d11f50b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:00:44.207497  6547 consensus_queue.cc:237] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "14c944622556420bb0c8c990ed25b3a8" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42047 } } peers { permanent_uuid: "2c0e5a656f71453f897427789d11f50b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43239 } } peers { permanent_uuid: "2d2face0e26e42d8806a902c15648764" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45739 } } peers { permanent_uuid: "bcad432fa2c9426b9be411ad57aac42b" member_type: NON_VOTER last_known_addr { host: "127.0.186.68" port: 38495 } attrs { } }
I20260812 08:00:44.209210  6855 raft_consensus.cc:1275] T 5bcffa11ab1447feb10888f069357cd6 P 14c944622556420bb0c8c990ed25b3a8 [term 1 FOLLOWER]: Refusing update from remote peer 2d2face0e26e42d8806a902c15648764: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:00:44.209273  6701 raft_consensus.cc:1275] T 5bcffa11ab1447feb10888f069357cd6 P 2c0e5a656f71453f897427789d11f50b [term 1 FOLLOWER]: Refusing update from remote peer 2d2face0e26e42d8806a902c15648764: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:00:44.209650  7101 consensus_queue.cc:1048] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14c944622556420bb0c8c990ed25b3a8" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:00:44.209808  7098 consensus_queue.cc:1048] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0e5a656f71453f897427789d11f50b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:00:44.211675  7101 raft_consensus.cc:2955] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER bcad432fa2c9426b9be411ad57aac42b (127.0.186.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "14c944622556420bb0c8c990ed25b3a8" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42047 } } peers { permanent_uuid: "2c0e5a656f71453f897427789d11f50b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43239 } } peers { permanent_uuid: "2d2face0e26e42d8806a902c15648764" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45739 } } peers { permanent_uuid: "bcad432fa2c9426b9be411ad57aac42b" member_type: NON_VOTER last_known_addr { host: "127.0.186.68" port: 38495 } attrs { } } }
I20260812 08:00:44.212038  6701 raft_consensus.cc:2955] T 5bcffa11ab1447feb10888f069357cd6 P 2c0e5a656f71453f897427789d11f50b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER bcad432fa2c9426b9be411ad57aac42b (127.0.186.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "14c944622556420bb0c8c990ed25b3a8" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42047 } } peers { permanent_uuid: "2c0e5a656f71453f897427789d11f50b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43239 } } peers { permanent_uuid: "2d2face0e26e42d8806a902c15648764" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45739 } } peers { permanent_uuid: "bcad432fa2c9426b9be411ad57aac42b" member_type: NON_VOTER last_known_addr { host: "127.0.186.68" port: 38495 } attrs { } } }
I20260812 08:00:44.212102  6855 raft_consensus.cc:2955] T 5bcffa11ab1447feb10888f069357cd6 P 14c944622556420bb0c8c990ed25b3a8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER bcad432fa2c9426b9be411ad57aac42b (127.0.186.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "14c944622556420bb0c8c990ed25b3a8" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42047 } } peers { permanent_uuid: "2c0e5a656f71453f897427789d11f50b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43239 } } peers { permanent_uuid: "2d2face0e26e42d8806a902c15648764" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45739 } } peers { permanent_uuid: "bcad432fa2c9426b9be411ad57aac42b" member_type: NON_VOTER last_known_addr { host: "127.0.186.68" port: 38495 } attrs { } } }
I20260812 08:00:44.214632  6374 catalog_manager.cc:5719] T 5bcffa11ab1447feb10888f069357cd6 P 2c0e5a656f71453f897427789d11f50b reported cstate change: config changed from index -1 to 2, NON_VOTER bcad432fa2c9426b9be411ad57aac42b (127.0.186.68) added. New cstate: current_term: 1 leader_uuid: "2d2face0e26e42d8806a902c15648764" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "14c944622556420bb0c8c990ed25b3a8" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42047 } } peers { permanent_uuid: "2c0e5a656f71453f897427789d11f50b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43239 } } peers { permanent_uuid: "2d2face0e26e42d8806a902c15648764" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45739 } } peers { permanent_uuid: "bcad432fa2c9426b9be411ad57aac42b" member_type: NON_VOTER last_known_addr { host: "127.0.186.68" port: 38495 } attrs { } } }
W20260812 08:00:44.217128  6463 consensus_peers.cc:597] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 -> Peer bcad432fa2c9426b9be411ad57aac42b (127.0.186.68:38495): Couldn't send request to peer bcad432fa2c9426b9be411ad57aac42b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5bcffa11ab1447feb10888f069357cd6. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:00:44.218530  6407 generic_service.cc:175] {username='slave'} at 127.0.0.1:36530 changed flags via RPC: catalog_manager_evict_excess_replicas from 'false' to 'true'
I20260812 08:00:48.225744  6547 consensus_queue.cc:237] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "14c944622556420bb0c8c990ed25b3a8" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42047 } } peers { permanent_uuid: "2c0e5a656f71453f897427789d11f50b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43239 } } peers { permanent_uuid: "2d2face0e26e42d8806a902c15648764" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45739 } }
I20260812 08:00:48.227227  6701 raft_consensus.cc:1275] T 5bcffa11ab1447feb10888f069357cd6 P 2c0e5a656f71453f897427789d11f50b [term 1 FOLLOWER]: Refusing update from remote peer 2d2face0e26e42d8806a902c15648764: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 08:00:48.227311  6855 raft_consensus.cc:1275] T 5bcffa11ab1447feb10888f069357cd6 P 14c944622556420bb0c8c990ed25b3a8 [term 1 FOLLOWER]: Refusing update from remote peer 2d2face0e26e42d8806a902c15648764: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 08:00:48.227682  7116 consensus_queue.cc:1048] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0e5a656f71453f897427789d11f50b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 08:00:48.227835  7114 consensus_queue.cc:1048] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14c944622556420bb0c8c990ed25b3a8" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 08:00:48.229666  7122 raft_consensus.cc:2955] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER bcad432fa2c9426b9be411ad57aac42b (127.0.186.68) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "14c944622556420bb0c8c990ed25b3a8" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42047 } } peers { permanent_uuid: "2c0e5a656f71453f897427789d11f50b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43239 } } peers { permanent_uuid: "2d2face0e26e42d8806a902c15648764" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45739 } } }
I20260812 08:00:48.230183  6701 raft_consensus.cc:2955] T 5bcffa11ab1447feb10888f069357cd6 P 2c0e5a656f71453f897427789d11f50b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER bcad432fa2c9426b9be411ad57aac42b (127.0.186.68) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "14c944622556420bb0c8c990ed25b3a8" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42047 } } peers { permanent_uuid: "2c0e5a656f71453f897427789d11f50b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43239 } } peers { permanent_uuid: "2d2face0e26e42d8806a902c15648764" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45739 } } }
I20260812 08:00:48.230257  6855 raft_consensus.cc:2955] T 5bcffa11ab1447feb10888f069357cd6 P 14c944622556420bb0c8c990ed25b3a8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER bcad432fa2c9426b9be411ad57aac42b (127.0.186.68) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "14c944622556420bb0c8c990ed25b3a8" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42047 } } peers { permanent_uuid: "2c0e5a656f71453f897427789d11f50b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43239 } } peers { permanent_uuid: "2d2face0e26e42d8806a902c15648764" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45739 } } }
I20260812 08:00:48.231410  6359 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 5bcffa11ab1447feb10888f069357cd6 with cas_config_opid_index 2: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:00:48.232378  6372 catalog_manager.cc:5719] T 5bcffa11ab1447feb10888f069357cd6 P 2d2face0e26e42d8806a902c15648764 reported cstate change: config changed from index 2 to 3, NON_VOTER bcad432fa2c9426b9be411ad57aac42b (127.0.186.68) evicted. New cstate: current_term: 1 leader_uuid: "2d2face0e26e42d8806a902c15648764" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "14c944622556420bb0c8c990ed25b3a8" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c0e5a656f71453f897427789d11f50b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d2face0e26e42d8806a902c15648764" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45739 } health_report { overall_health: HEALTHY } } }
W20260812 08:00:48.379751  6465 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 08:00:48.792950  6465 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 08:00:49.410346  6465 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 08:00:49.630345  6463 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: StartTabletCopy RPC to 127.0.186.68:38495 timed out after 5.000s (SENT)
W20260812 08:00:49.869781  6465 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 08:00:50.265746  6465 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 08:00:50.853055  6465 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 08:00:51.237244  6465 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 08:00:51.490274  7129 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:00:48.235474 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.186.68:38495 (local address 127.0.0.1:44960)
0812 08:00:48.235809 (+   335us) negotiation.cc:107] Waiting for socket to connect
0812 08:00:48.235822 (+    13us) client_negotiation.cc:175] Beginning negotiation
0812 08:00:48.235897 (+    75us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 08:00:51.489740 (+3253843us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.186.68:38495: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":98,"threads_started":1}
W20260812 08:00:51.490558  6357 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: Client connection negotiation failed: client connection to 127.0.186.68:38495: received 0 of 4 requested bytes
W20260812 08:00:51.719702  6465 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 08:00:52.273041  6465 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 08:00:52.873379  6465 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 08:00:53.402652  6465 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 08:00:53.855930  6465 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260812 08:00:54.231034  7065 heartbeater.cc:499] Master 127.0.186.126:41969 was elected leader, sending a full tablet report...
I20260812 08:00:54.231961  7136 ts_tablet_manager.cc:933] T 5bcffa11ab1447feb10888f069357cd6 P bcad432fa2c9426b9be411ad57aac42b: Initiating tablet copy from peer 2d2face0e26e42d8806a902c15648764 (127.0.186.65:45739)
I20260812 08:00:54.232672  7136 tablet_copy_client.cc:323] T 5bcffa11ab1447feb10888f069357cd6 P bcad432fa2c9426b9be411ad57aac42b: tablet copy: Beginning tablet copy session from remote peer at address 127.0.186.65:45739
I20260812 08:00:54.234097   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 6443
W20260812 08:00:54.235944  7138 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:00:54.232010 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:44960 (local address 127.0.186.68:38495)
0812 08:00:54.234774 (+  2764us) server_negotiation.cc:207] Beginning negotiation
0812 08:00:54.234779 (+     5us) server_negotiation.cc:400] Waiting for connection header
0812 08:00:54.234793 (+    14us) server_negotiation.cc:408] Connection header received
0812 08:00:54.234834 (+    41us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 08:00:54.234837 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 08:00:54.234878 (+    41us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 08:00:54.235038 (+   160us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44960: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":2683,"thread_start_us":94,"threads_started":1}
I20260812 08:00:54.242147  6983 tablet_service.cc:1558] Processing DeleteTablet for tablet 5bcffa11ab1447feb10888f069357cd6 with delete_type TABLET_DATA_TOMBSTONED (TS bcad432fa2c9426b9be411ad57aac42b not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:44968
W20260812 08:00:54.244133  6357 catalog_manager.cc:5017] TS bcad432fa2c9426b9be411ad57aac42b (127.0.186.68:38495): delete failed for tablet 5bcffa11ab1447feb10888f069357cd6 because the tablet was not found. No further retry: Not found: Tablet not found: 5bcffa11ab1447feb10888f069357cd6
W20260812 08:00:54.249589  6934 connection.cc:573] server connection from 127.0.186.65:59249 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 08:00:54.249742  7140 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:00:54.233224 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.186.65:45739 (local address 127.0.186.68:51173)
0812 08:00:54.233744 (+   520us) negotiation.cc:107] Waiting for socket to connect
0812 08:00:54.233759 (+    15us) client_negotiation.cc:175] Beginning negotiation
0812 08:00:54.233853 (+    94us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 08:00:54.249626 (+ 15773us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.65:45739: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":87,"threads_started":1}
I20260812 08:00:54.250579   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 6600
W20260812 08:00:54.255376  6934 connection.cc:466] server connection from 127.0.186.65:59249 torn down before Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.186.65:59249 (request call id 1) could send its response
I20260812 08:00:54.262974   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 6757
I20260812 08:00:54.274458   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 6912
I20260812 08:00:54.285687   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 6339
2026-08-12T08:00:54Z chronyd exiting
[       OK ] RaftConsensusNonVoterITest.CatalogManagerEvictsExcessNonVoter (11644 ms)
[ RUN      ] RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter
I20260812 08:00:54.312741   745 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:00:54.312793   745 ts_itest-base.cc:116] --------------
I20260812 08:00:54.312803   745 ts_itest-base.cc:117] 4 tablet servers
I20260812 08:00:54.312812   745 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:00:54.312821   745 ts_itest-base.cc:119] --------------
2026-08-12T08:00:54Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:00:54Z Disabled control of system clock
I20260812 08:00:54.321794   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:45125
--webserver_interface=127.0.186.126
--webserver_port=0
--builtin_ntp_servers=127.0.186.84:35399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:45125
--raft_prepare_replacement_before_eviction=true
--catalog_manager_evict_excess_replicas=false with env {}
W20260812 08:00:54.426760  7152 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:54.427002  7152 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:54.427057  7152 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:54.430771  7152 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:00:54.430843  7152 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:54.430871  7152 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:00:54.430894  7152 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:00:54.434810  7152 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:35399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--catalog_manager_evict_excess_replicas=false
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:45125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:45125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.7152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:54.435950  7152 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:54.437105  7152 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:54.442816  7161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:54.442827  7159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:54.442835  7158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:54.442893  7152 server_base.cc:1061] running on GCE node
I20260812 08:00:54.443590  7152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:54.444486  7152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:54.445658  7152 hybrid_clock.cc:648] HybridClock initialized: now 1786521654445642 us; error 31 us; skew 500 ppm
I20260812 08:00:54.447674  7152 webserver.cc:533] Webserver started at http://127.0.186.126:35167/ using document root <none> and password file <none>
I20260812 08:00:54.448283  7152 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:54.448342  7152 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:54.448575  7152 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:54.450281  7152 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "fdab4a8a35ad4959b4dbfd672fb32d43"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-49h3"
I20260812 08:00:54.450770  7152 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "fdab4a8a35ad4959b4dbfd672fb32d43"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-49h3"
I20260812 08:00:54.454245  7152 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:00:54.456439  7167 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:54.457433  7152 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:00:54.457566  7152 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "fdab4a8a35ad4959b4dbfd672fb32d43"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-49h3"
I20260812 08:00:54.457659  7152 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:54.471014  7152 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:54.471690  7152 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:54.471864  7152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:54.479314  7231 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:45125 every 8 connection(s)
I20260812 08:00:54.479322  7152 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:45125
I20260812 08:00:54.480374  7152 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:00:54.483232  7232 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:54.486759   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 7152
I20260812 08:00:54.486879   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:00:54.488801  7232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdab4a8a35ad4959b4dbfd672fb32d43: Bootstrap starting.
I20260812 08:00:54.490936  7232 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdab4a8a35ad4959b4dbfd672fb32d43: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:54.491628  7232 log.cc:826] T 00000000000000000000000000000000 P fdab4a8a35ad4959b4dbfd672fb32d43: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:54.493364  7232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdab4a8a35ad4959b4dbfd672fb32d43: No bootstrap required, opened a new log
I20260812 08:00:54.495690  7232 raft_consensus.cc:359] T 00000000000000000000000000000000 P fdab4a8a35ad4959b4dbfd672fb32d43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdab4a8a35ad4959b4dbfd672fb32d43" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 45125 } }
I20260812 08:00:54.495873  7232 raft_consensus.cc:385] T 00000000000000000000000000000000 P fdab4a8a35ad4959b4dbfd672fb32d43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:54.495926  7232 raft_consensus.cc:740] T 00000000000000000000000000000000 P fdab4a8a35ad4959b4dbfd672fb32d43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdab4a8a35ad4959b4dbfd672fb32d43, State: Initialized, Role: FOLLOWER
I20260812 08:00:54.496270  7232 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdab4a8a35ad4959b4dbfd672fb32d43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdab4a8a35ad4959b4dbfd672fb32d43" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 45125 } }
I20260812 08:00:54.496372  7232 raft_consensus.cc:399] T 00000000000000000000000000000000 P fdab4a8a35ad4959b4dbfd672fb32d43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:00:54.496407  7232 raft_consensus.cc:493] T 00000000000000000000000000000000 P fdab4a8a35ad4959b4dbfd672fb32d43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:00:54.496480  7232 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fdab4a8a35ad4959b4dbfd672fb32d43 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:54.497315  7232 raft_consensus.cc:515] T 00000000000000000000000000000000 P fdab4a8a35ad4959b4dbfd672fb32d43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdab4a8a35ad4959b4dbfd672fb32d43" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 45125 } }
I20260812 08:00:54.497701  7232 leader_election.cc:304] T 00000000000000000000000000000000 P fdab4a8a35ad4959b4dbfd672fb32d43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdab4a8a35ad4959b4dbfd672fb32d43; no voters: 
I20260812 08:00:54.498023  7232 leader_election.cc:290] T 00000000000000000000000000000000 P fdab4a8a35ad4959b4dbfd672fb32d43 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:00:54.498114  7237 raft_consensus.cc:2804] T 00000000000000000000000000000000 P fdab4a8a35ad4959b4dbfd672fb32d43 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:00:54.498312  7237 raft_consensus.cc:697] T 00000000000000000000000000000000 P fdab4a8a35ad4959b4dbfd672fb32d43 [term 1 LEADER]: Becoming Leader. State: Replica: fdab4a8a35ad4959b4dbfd672fb32d43, State: Running, Role: LEADER
I20260812 08:00:54.498651  7237 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdab4a8a35ad4959b4dbfd672fb32d43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdab4a8a35ad4959b4dbfd672fb32d43" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 45125 } }
I20260812 08:00:54.499286  7232 sys_catalog.cc:565] T 00000000000000000000000000000000 P fdab4a8a35ad4959b4dbfd672fb32d43 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:00:54.499809  7237 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdab4a8a35ad4959b4dbfd672fb32d43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdab4a8a35ad4959b4dbfd672fb32d43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdab4a8a35ad4959b4dbfd672fb32d43" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 45125 } } }
I20260812 08:00:54.499967  7237 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdab4a8a35ad4959b4dbfd672fb32d43 [sys.catalog]: This master's current role is: LEADER
I20260812 08:00:54.500169  7238 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdab4a8a35ad4959b4dbfd672fb32d43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdab4a8a35ad4959b4dbfd672fb32d43. Latest consensus state: current_term: 1 leader_uuid: "fdab4a8a35ad4959b4dbfd672fb32d43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdab4a8a35ad4959b4dbfd672fb32d43" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 45125 } } }
I20260812 08:00:54.500325  7238 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdab4a8a35ad4959b4dbfd672fb32d43 [sys.catalog]: This master's current role is: LEADER
I20260812 08:00:54.500409  7242 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:00:54.503298  7242 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:00:54.508400  7242 catalog_manager.cc:1383] Generated new cluster ID: 0995a184cf004c5a9ba6142e39930b84
I20260812 08:00:54.508471  7242 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:00:54.520072  7242 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:00:54.520888  7242 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:00:54.535414  7242 catalog_manager.cc:6092] T 00000000000000000000000000000000 P fdab4a8a35ad4959b4dbfd672fb32d43: Generated new TSK 0
I20260812 08:00:54.536018  7242 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:00:54.542887   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:0
--local_ip_for_outbound_sockets=127.0.186.65
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:45125
--builtin_ntp_servers=127.0.186.84:35399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=10 with env {}
W20260812 08:00:54.650957  7258 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:54.651178  7258 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:54.651221  7258 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:54.654551  7258 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:54.654661  7258 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:00:54.658505  7258 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:35399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=10
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:45125
--never_fsync=true
--heap_profile_path=/tmp/kudu.7258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:54.659567  7258 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:54.660699  7258 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:54.667045  7264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:54.667088  7265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:54.667044  7267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:54.667353  7258 server_base.cc:1061] running on GCE node
I20260812 08:00:54.667728  7258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:54.668255  7258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:54.669409  7258 hybrid_clock.cc:648] HybridClock initialized: now 1786521654669394 us; error 32 us; skew 500 ppm
I20260812 08:00:54.671247  7258 webserver.cc:533] Webserver started at http://127.0.186.65:39267/ using document root <none> and password file <none>
I20260812 08:00:54.671885  7258 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:54.671962  7258 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:54.672187  7258 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:54.674226  7258 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "2290733bc4724005abcfafd52fe405c5"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-49h3"
I20260812 08:00:54.674746  7258 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "2290733bc4724005abcfafd52fe405c5"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-49h3"
I20260812 08:00:54.678498  7258 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:00:54.680863  7273 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:54.681954  7258 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:00:54.682057  7258 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "2290733bc4724005abcfafd52fe405c5"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-49h3"
I20260812 08:00:54.682143  7258 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:54.697435  7258 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:54.698140  7258 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:54.698294  7258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:54.698837  7258 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:54.699895  7258 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:54.699941  7258 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:54.699982  7258 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:54.700007  7258 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:54.708837  7258 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:41567
I20260812 08:00:54.708868  7412 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:41567 every 8 connection(s)
I20260812 08:00:54.709820  7258 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:00:54.717576  7413 heartbeater.cc:344] Connected to a master server at 127.0.186.126:45125
I20260812 08:00:54.717810  7413 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:54.718287  7413 heartbeater.cc:507] Master 127.0.186.126:45125 requested a full tablet report, sending...
I20260812 08:00:54.719661  7188 ts_manager.cc:194] Registered new tserver with Master: 2290733bc4724005abcfafd52fe405c5 (127.0.186.65:41567)
I20260812 08:00:54.719846   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 7258
I20260812 08:00:54.719957   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:00:54.720983  7188 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.65:42869
I20260812 08:00:54.722360   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.66:0
--local_ip_for_outbound_sockets=127.0.186.66
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:45125
--builtin_ntp_servers=127.0.186.84:35399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=10 with env {}
W20260812 08:00:54.826781  7416 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:54.826995  7416 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:54.827054  7416 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:54.830430  7416 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:54.830550  7416 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.66
I20260812 08:00:54.834251  7416 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:35399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=10
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:45125
--never_fsync=true
--heap_profile_path=/tmp/kudu.7416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:54.835314  7416 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:54.836412  7416 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:54.842854  7422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:54.842839  7423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:54.842998  7425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:54.843161  7416 server_base.cc:1061] running on GCE node
I20260812 08:00:54.843485  7416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:54.843981  7416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:54.845139  7416 hybrid_clock.cc:648] HybridClock initialized: now 1786521654845123 us; error 27 us; skew 500 ppm
I20260812 08:00:54.846833  7416 webserver.cc:533] Webserver started at http://127.0.186.66:41349/ using document root <none> and password file <none>
I20260812 08:00:54.847373  7416 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:54.847424  7416 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:54.847594  7416 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:54.849109  7416 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "f7e522d581c24cebabb4b5319040334b"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-49h3"
I20260812 08:00:54.849601  7416 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "f7e522d581c24cebabb4b5319040334b"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-49h3"
I20260812 08:00:54.852717  7416 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.001s
I20260812 08:00:54.854740  7433 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:54.855650  7416 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:00:54.855739  7416 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "f7e522d581c24cebabb4b5319040334b"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-49h3"
I20260812 08:00:54.855811  7416 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:54.876662  7416 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:54.877272  7416 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:54.877414  7416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:54.877990  7416 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:54.879007  7416 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:54.879053  7416 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:54.879092  7416 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:54.879112  7416 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:54.887784  7416 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.66:34033
I20260812 08:00:54.887807  7568 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.66:34033 every 8 connection(s)
I20260812 08:00:54.888737  7416 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:00:54.896456  7569 heartbeater.cc:344] Connected to a master server at 127.0.186.126:45125
I20260812 08:00:54.896687  7569 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:54.897156  7569 heartbeater.cc:507] Master 127.0.186.126:45125 requested a full tablet report, sending...
I20260812 08:00:54.898217  7188 ts_manager.cc:194] Registered new tserver with Master: f7e522d581c24cebabb4b5319040334b (127.0.186.66:34033)
I20260812 08:00:54.898769  7188 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.66:34231
I20260812 08:00:54.898871   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 7416
I20260812 08:00:54.898990   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:00:54.901283   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.67:0
--local_ip_for_outbound_sockets=127.0.186.67
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:45125
--builtin_ntp_servers=127.0.186.84:35399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=10 with env {}
W20260812 08:00:55.005744  7572 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:55.005945  7572 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:55.005985  7572 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:55.009258  7572 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:55.009367  7572 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.67
I20260812 08:00:55.013470  7572 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:35399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=10
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:45125
--never_fsync=true
--heap_profile_path=/tmp/kudu.7572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:55.014575  7572 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:55.015703  7572 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:55.022029  7578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:55.022068  7581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:55.022068  7579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:55.022310  7572 server_base.cc:1061] running on GCE node
I20260812 08:00:55.022650  7572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:55.023146  7572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:55.024319  7572 hybrid_clock.cc:648] HybridClock initialized: now 1786521655024300 us; error 28 us; skew 500 ppm
I20260812 08:00:55.026007  7572 webserver.cc:533] Webserver started at http://127.0.186.67:34669/ using document root <none> and password file <none>
I20260812 08:00:55.026585  7572 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:55.026643  7572 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:55.026845  7572 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:55.028358  7572 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "325a03b321754804b562068549caf8a3"
format_stamp: "Formatted at 2026-08-12 08:00:55 on dist-test-slave-49h3"
I20260812 08:00:55.028816  7572 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "325a03b321754804b562068549caf8a3"
format_stamp: "Formatted at 2026-08-12 08:00:55 on dist-test-slave-49h3"
I20260812 08:00:55.032191  7572 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:00:55.034240  7589 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:55.035192  7572 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:00:55.035312  7572 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "325a03b321754804b562068549caf8a3"
format_stamp: "Formatted at 2026-08-12 08:00:55 on dist-test-slave-49h3"
I20260812 08:00:55.035396  7572 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:55.047830  7572 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:55.048408  7572 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:55.048564  7572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:55.049089  7572 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:55.050146  7572 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:55.050197  7572 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:55.050258  7572 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:55.050282  7572 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:55.059126  7572 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.67:37129
I20260812 08:00:55.059162  7717 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.67:37129 every 8 connection(s)
I20260812 08:00:55.060079  7572 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:00:55.067641   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 7572
I20260812 08:00:55.067762   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:00:55.068164  7718 heartbeater.cc:344] Connected to a master server at 127.0.186.126:45125
I20260812 08:00:55.068425  7718 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:55.069046  7718 heartbeater.cc:507] Master 127.0.186.126:45125 requested a full tablet report, sending...
I20260812 08:00:55.070035   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.68:0
--local_ip_for_outbound_sockets=127.0.186.68
--webserver_interface=127.0.186.68
--webserver_port=0
--tserver_master_addrs=127.0.186.126:45125
--builtin_ntp_servers=127.0.186.84:35399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=10 with env {}
I20260812 08:00:55.070276  7188 ts_manager.cc:194] Registered new tserver with Master: 325a03b321754804b562068549caf8a3 (127.0.186.67:37129)
I20260812 08:00:55.070873  7188 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.67:52815
W20260812 08:00:55.177770  7721 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:55.177960  7721 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:55.178000  7721 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:55.181180  7721 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:55.181274  7721 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.68
I20260812 08:00:55.184912  7721 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:35399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=10
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.0.186.68
--webserver_port=0
--tserver_master_addrs=127.0.186.126:45125
--never_fsync=true
--heap_profile_path=/tmp/kudu.7721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.68

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:55.185909  7721 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:55.187050  7721 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:55.192905  7728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:55.193055  7731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:55.193223  7721 server_base.cc:1061] running on GCE node
W20260812 08:00:55.193360  7729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:55.193737  7721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:55.194228  7721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:55.195375  7721 hybrid_clock.cc:648] HybridClock initialized: now 1786521655195360 us; error 29 us; skew 500 ppm
I20260812 08:00:55.196978  7721 webserver.cc:533] Webserver started at http://127.0.186.68:43939/ using document root <none> and password file <none>
I20260812 08:00:55.197530  7721 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:55.197597  7721 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:55.197783  7721 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:55.199220  7721 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "8a495179243a41baa4bbd1ab1763317f"
format_stamp: "Formatted at 2026-08-12 08:00:55 on dist-test-slave-49h3"
I20260812 08:00:55.199658  7721 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "8a495179243a41baa4bbd1ab1763317f"
format_stamp: "Formatted at 2026-08-12 08:00:55 on dist-test-slave-49h3"
I20260812 08:00:55.202819  7721 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:00:55.204751  7741 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:55.205703  7721 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:00:55.205798  7721 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "8a495179243a41baa4bbd1ab1763317f"
format_stamp: "Formatted at 2026-08-12 08:00:55 on dist-test-slave-49h3"
I20260812 08:00:55.205874  7721 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:55.234416  7721 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:55.235060  7721 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:55.235224  7721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:55.235788  7721 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:55.236792  7721 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:55.236840  7721 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:55.236893  7721 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:55.236917  7721 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:55.247061  7721 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.68:44643
I20260812 08:00:55.247114  7873 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.68:44643 every 8 connection(s)
I20260812 08:00:55.248126  7721 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20260812 08:00:55.256074  7874 heartbeater.cc:344] Connected to a master server at 127.0.186.126:45125
I20260812 08:00:55.256315  7874 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:55.256822  7874 heartbeater.cc:507] Master 127.0.186.126:45125 requested a full tablet report, sending...
I20260812 08:00:55.257393   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 7721
I20260812 08:00:55.257540   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20260812 08:00:55.258011  7188 ts_manager.cc:194] Registered new tserver with Master: 8a495179243a41baa4bbd1ab1763317f (127.0.186.68:44643)
I20260812 08:00:55.258677  7188 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.68:55579
I20260812 08:00:55.260039   745 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20260812 08:00:55.268570  7188 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45698:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
I20260812 08:00:55.287276  7334 tablet_service.cc:1511] Processing CreateTablet for tablet b65a1b146d324653b10b452e9c54b954 (DEFAULT_TABLE table=TestTable [id=08b6c75d7d914296bbdc896293b8fdbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:55.287292  7799 tablet_service.cc:1511] Processing CreateTablet for tablet b65a1b146d324653b10b452e9c54b954 (DEFAULT_TABLE table=TestTable [id=08b6c75d7d914296bbdc896293b8fdbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:55.287406  7494 tablet_service.cc:1511] Processing CreateTablet for tablet b65a1b146d324653b10b452e9c54b954 (DEFAULT_TABLE table=TestTable [id=08b6c75d7d914296bbdc896293b8fdbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:55.288178  7334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b65a1b146d324653b10b452e9c54b954. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:55.288223  7494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b65a1b146d324653b10b452e9c54b954. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:55.288223  7799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b65a1b146d324653b10b452e9c54b954. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:55.293183  7896 tablet_bootstrap.cc:492] T b65a1b146d324653b10b452e9c54b954 P 2290733bc4724005abcfafd52fe405c5: Bootstrap starting.
I20260812 08:00:55.294312  7898 tablet_bootstrap.cc:492] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f: Bootstrap starting.
I20260812 08:00:55.295543  7896 tablet_bootstrap.cc:654] T b65a1b146d324653b10b452e9c54b954 P 2290733bc4724005abcfafd52fe405c5: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:55.296156  7898 tablet_bootstrap.cc:654] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:55.296238  7896 log.cc:826] T b65a1b146d324653b10b452e9c54b954 P 2290733bc4724005abcfafd52fe405c5: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:55.296788  7898 log.cc:826] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:55.296924  7899 tablet_bootstrap.cc:492] T b65a1b146d324653b10b452e9c54b954 P f7e522d581c24cebabb4b5319040334b: Bootstrap starting.
I20260812 08:00:55.298254  7896 tablet_bootstrap.cc:492] T b65a1b146d324653b10b452e9c54b954 P 2290733bc4724005abcfafd52fe405c5: No bootstrap required, opened a new log
I20260812 08:00:55.298375  7898 tablet_bootstrap.cc:492] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f: No bootstrap required, opened a new log
I20260812 08:00:55.298453  7896 ts_tablet_manager.cc:1403] T b65a1b146d324653b10b452e9c54b954 P 2290733bc4724005abcfafd52fe405c5: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:00:55.298522  7898 ts_tablet_manager.cc:1403] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f: Time spent bootstrapping tablet: real 0.004s	user 0.003s	sys 0.000s
I20260812 08:00:55.298743  7899 tablet_bootstrap.cc:654] T b65a1b146d324653b10b452e9c54b954 P f7e522d581c24cebabb4b5319040334b: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:55.299357  7899 log.cc:826] T b65a1b146d324653b10b452e9c54b954 P f7e522d581c24cebabb4b5319040334b: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:55.300932  7899 tablet_bootstrap.cc:492] T b65a1b146d324653b10b452e9c54b954 P f7e522d581c24cebabb4b5319040334b: No bootstrap required, opened a new log
I20260812 08:00:55.301119  7899 ts_tablet_manager.cc:1403] T b65a1b146d324653b10b452e9c54b954 P f7e522d581c24cebabb4b5319040334b: Time spent bootstrapping tablet: real 0.004s	user 0.003s	sys 0.000s
I20260812 08:00:55.301643  7898 raft_consensus.cc:359] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a495179243a41baa4bbd1ab1763317f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44643 } } peers { permanent_uuid: "f7e522d581c24cebabb4b5319040334b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 34033 } } peers { permanent_uuid: "2290733bc4724005abcfafd52fe405c5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41567 } }
I20260812 08:00:55.301635  7896 raft_consensus.cc:359] T b65a1b146d324653b10b452e9c54b954 P 2290733bc4724005abcfafd52fe405c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a495179243a41baa4bbd1ab1763317f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44643 } } peers { permanent_uuid: "f7e522d581c24cebabb4b5319040334b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 34033 } } peers { permanent_uuid: "2290733bc4724005abcfafd52fe405c5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41567 } }
I20260812 08:00:55.301853  7898 raft_consensus.cc:385] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:55.301853  7896 raft_consensus.cc:385] T b65a1b146d324653b10b452e9c54b954 P 2290733bc4724005abcfafd52fe405c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:55.301911  7898 raft_consensus.cc:740] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a495179243a41baa4bbd1ab1763317f, State: Initialized, Role: FOLLOWER
I20260812 08:00:55.301911  7896 raft_consensus.cc:740] T b65a1b146d324653b10b452e9c54b954 P 2290733bc4724005abcfafd52fe405c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2290733bc4724005abcfafd52fe405c5, State: Initialized, Role: FOLLOWER
I20260812 08:00:55.302361  7898 consensus_queue.cc:260] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a495179243a41baa4bbd1ab1763317f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44643 } } peers { permanent_uuid: "f7e522d581c24cebabb4b5319040334b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 34033 } } peers { permanent_uuid: "2290733bc4724005abcfafd52fe405c5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41567 } }
I20260812 08:00:55.302361  7896 consensus_queue.cc:260] T b65a1b146d324653b10b452e9c54b954 P 2290733bc4724005abcfafd52fe405c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a495179243a41baa4bbd1ab1763317f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44643 } } peers { permanent_uuid: "f7e522d581c24cebabb4b5319040334b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 34033 } } peers { permanent_uuid: "2290733bc4724005abcfafd52fe405c5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41567 } }
I20260812 08:00:55.303244  7896 ts_tablet_manager.cc:1434] T b65a1b146d324653b10b452e9c54b954 P 2290733bc4724005abcfafd52fe405c5: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 08:00:55.303244  7898 ts_tablet_manager.cc:1434] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 08:00:55.303264  7874 heartbeater.cc:499] Master 127.0.186.126:45125 was elected leader, sending a full tablet report...
I20260812 08:00:55.303588  7413 heartbeater.cc:499] Master 127.0.186.126:45125 was elected leader, sending a full tablet report...
I20260812 08:00:55.303942  7899 raft_consensus.cc:359] T b65a1b146d324653b10b452e9c54b954 P f7e522d581c24cebabb4b5319040334b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a495179243a41baa4bbd1ab1763317f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44643 } } peers { permanent_uuid: "f7e522d581c24cebabb4b5319040334b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 34033 } } peers { permanent_uuid: "2290733bc4724005abcfafd52fe405c5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41567 } }
I20260812 08:00:55.304147  7899 raft_consensus.cc:385] T b65a1b146d324653b10b452e9c54b954 P f7e522d581c24cebabb4b5319040334b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:55.304203  7899 raft_consensus.cc:740] T b65a1b146d324653b10b452e9c54b954 P f7e522d581c24cebabb4b5319040334b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7e522d581c24cebabb4b5319040334b, State: Initialized, Role: FOLLOWER
I20260812 08:00:55.304698  7899 consensus_queue.cc:260] T b65a1b146d324653b10b452e9c54b954 P f7e522d581c24cebabb4b5319040334b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a495179243a41baa4bbd1ab1763317f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44643 } } peers { permanent_uuid: "f7e522d581c24cebabb4b5319040334b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 34033 } } peers { permanent_uuid: "2290733bc4724005abcfafd52fe405c5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41567 } }
I20260812 08:00:55.305303  7569 heartbeater.cc:499] Master 127.0.186.126:45125 was elected leader, sending a full tablet report...
I20260812 08:00:55.305542  7899 ts_tablet_manager.cc:1434] T b65a1b146d324653b10b452e9c54b954 P f7e522d581c24cebabb4b5319040334b: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
W20260812 08:00:55.389839  7570 tablet.cc:2406] T b65a1b146d324653b10b452e9c54b954 P f7e522d581c24cebabb4b5319040334b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:00:55.412868  7905 raft_consensus.cc:493] T b65a1b146d324653b10b452e9c54b954 P 2290733bc4724005abcfafd52fe405c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:00:55.413092  7905 raft_consensus.cc:515] T b65a1b146d324653b10b452e9c54b954 P 2290733bc4724005abcfafd52fe405c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a495179243a41baa4bbd1ab1763317f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44643 } } peers { permanent_uuid: "f7e522d581c24cebabb4b5319040334b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 34033 } } peers { permanent_uuid: "2290733bc4724005abcfafd52fe405c5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41567 } }
I20260812 08:00:55.414255  7905 leader_election.cc:290] T b65a1b146d324653b10b452e9c54b954 P 2290733bc4724005abcfafd52fe405c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a495179243a41baa4bbd1ab1763317f (127.0.186.68:44643), f7e522d581c24cebabb4b5319040334b (127.0.186.66:34033)
I20260812 08:00:55.418260  7819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65a1b146d324653b10b452e9c54b954" candidate_uuid: "2290733bc4724005abcfafd52fe405c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a495179243a41baa4bbd1ab1763317f" is_pre_election: true
I20260812 08:00:55.418632  7819 raft_consensus.cc:2468] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2290733bc4724005abcfafd52fe405c5 in term 0.
I20260812 08:00:55.418913  7519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65a1b146d324653b10b452e9c54b954" candidate_uuid: "2290733bc4724005abcfafd52fe405c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e522d581c24cebabb4b5319040334b" is_pre_election: true
I20260812 08:00:55.419169  7519 raft_consensus.cc:2468] T b65a1b146d324653b10b452e9c54b954 P f7e522d581c24cebabb4b5319040334b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2290733bc4724005abcfafd52fe405c5 in term 0.
I20260812 08:00:55.419169  7275 leader_election.cc:304] T b65a1b146d324653b10b452e9c54b954 P 2290733bc4724005abcfafd52fe405c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2290733bc4724005abcfafd52fe405c5, 8a495179243a41baa4bbd1ab1763317f; no voters: 
I20260812 08:00:55.419447  7905 raft_consensus.cc:2804] T b65a1b146d324653b10b452e9c54b954 P 2290733bc4724005abcfafd52fe405c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:00:55.419521  7905 raft_consensus.cc:493] T b65a1b146d324653b10b452e9c54b954 P 2290733bc4724005abcfafd52fe405c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:00:55.419564  7905 raft_consensus.cc:3060] T b65a1b146d324653b10b452e9c54b954 P 2290733bc4724005abcfafd52fe405c5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:55.420492  7905 raft_consensus.cc:515] T b65a1b146d324653b10b452e9c54b954 P 2290733bc4724005abcfafd52fe405c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a495179243a41baa4bbd1ab1763317f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44643 } } peers { permanent_uuid: "f7e522d581c24cebabb4b5319040334b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 34033 } } peers { permanent_uuid: "2290733bc4724005abcfafd52fe405c5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41567 } }
I20260812 08:00:55.420855  7905 leader_election.cc:290] T b65a1b146d324653b10b452e9c54b954 P 2290733bc4724005abcfafd52fe405c5 [CANDIDATE]: Term 1 election: Requested vote from peers 8a495179243a41baa4bbd1ab1763317f (127.0.186.68:44643), f7e522d581c24cebabb4b5319040334b (127.0.186.66:34033)
I20260812 08:00:55.421218  7819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65a1b146d324653b10b452e9c54b954" candidate_uuid: "2290733bc4724005abcfafd52fe405c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a495179243a41baa4bbd1ab1763317f"
I20260812 08:00:55.421280  7519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65a1b146d324653b10b452e9c54b954" candidate_uuid: "2290733bc4724005abcfafd52fe405c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e522d581c24cebabb4b5319040334b"
I20260812 08:00:55.421344  7819 raft_consensus.cc:3060] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:55.421406  7519 raft_consensus.cc:3060] T b65a1b146d324653b10b452e9c54b954 P f7e522d581c24cebabb4b5319040334b [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:55.422583  7819 raft_consensus.cc:2468] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2290733bc4724005abcfafd52fe405c5 in term 1.
I20260812 08:00:55.422596  7519 raft_consensus.cc:2468] T b65a1b146d324653b10b452e9c54b954 P f7e522d581c24cebabb4b5319040334b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2290733bc4724005abcfafd52fe405c5 in term 1.
I20260812 08:00:55.422883  7275 leader_election.cc:304] T b65a1b146d324653b10b452e9c54b954 P 2290733bc4724005abcfafd52fe405c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2290733bc4724005abcfafd52fe405c5, 8a495179243a41baa4bbd1ab1763317f; no voters: 
I20260812 08:00:55.423072  7905 raft_consensus.cc:2804] T b65a1b146d324653b10b452e9c54b954 P 2290733bc4724005abcfafd52fe405c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:00:55.423305  7905 raft_consensus.cc:697] T b65a1b146d324653b10b452e9c54b954 P 2290733bc4724005abcfafd52fe405c5 [term 1 LEADER]: Becoming Leader. State: Replica: 2290733bc4724005abcfafd52fe405c5, State: Running, Role: LEADER
I20260812 08:00:55.423605  7905 consensus_queue.cc:237] T b65a1b146d324653b10b452e9c54b954 P 2290733bc4724005abcfafd52fe405c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a495179243a41baa4bbd1ab1763317f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44643 } } peers { permanent_uuid: "f7e522d581c24cebabb4b5319040334b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 34033 } } peers { permanent_uuid: "2290733bc4724005abcfafd52fe405c5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41567 } }
I20260812 08:00:55.426839  7187 catalog_manager.cc:5719] T b65a1b146d324653b10b452e9c54b954 P 2290733bc4724005abcfafd52fe405c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2290733bc4724005abcfafd52fe405c5 (127.0.186.65). New cstate: current_term: 1 leader_uuid: "2290733bc4724005abcfafd52fe405c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a495179243a41baa4bbd1ab1763317f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e522d581c24cebabb4b5319040334b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 34033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2290733bc4724005abcfafd52fe405c5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41567 } health_report { overall_health: HEALTHY } } }
I20260812 08:00:55.432866   745 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20260812 08:00:55.434437   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 2290733bc4724005abcfafd52fe405c5 to finish bootstrapping
I20260812 08:00:55.440436   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver f7e522d581c24cebabb4b5319040334b to finish bootstrapping
I20260812 08:00:55.445668   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 8a495179243a41baa4bbd1ab1763317f to finish bootstrapping
I20260812 08:00:55.450460   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 7258
W20260812 08:00:55.499054  7875 tablet.cc:2406] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:00:56.073313  7718 heartbeater.cc:499] Master 127.0.186.126:45125 was elected leader, sending a full tablet report...
I20260812 08:00:57.001543  7912 raft_consensus.cc:493] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:00:57.001701  7912 raft_consensus.cc:515] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a495179243a41baa4bbd1ab1763317f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44643 } } peers { permanent_uuid: "f7e522d581c24cebabb4b5319040334b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 34033 } } peers { permanent_uuid: "2290733bc4724005abcfafd52fe405c5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41567 } }
I20260812 08:00:57.002830  7912 leader_election.cc:290] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f7e522d581c24cebabb4b5319040334b (127.0.186.66:34033), 2290733bc4724005abcfafd52fe405c5 (127.0.186.65:41567)
W20260812 08:00:57.003545  7746 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.65:41567: connect: Connection refused (error 111)
W20260812 08:00:57.004336  7746 leader_election.cc:336] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2290733bc4724005abcfafd52fe405c5 (127.0.186.65:41567): Network error: Client connection negotiation failed: client connection to 127.0.186.65:41567: connect: Connection refused (error 111)
I20260812 08:00:57.006388  7519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65a1b146d324653b10b452e9c54b954" candidate_uuid: "8a495179243a41baa4bbd1ab1763317f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e522d581c24cebabb4b5319040334b" is_pre_election: true
I20260812 08:00:57.006543  7519 raft_consensus.cc:2468] T b65a1b146d324653b10b452e9c54b954 P f7e522d581c24cebabb4b5319040334b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a495179243a41baa4bbd1ab1763317f in term 1.
I20260812 08:00:57.006883  7745 leader_election.cc:304] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a495179243a41baa4bbd1ab1763317f, f7e522d581c24cebabb4b5319040334b; no voters: 2290733bc4724005abcfafd52fe405c5
I20260812 08:00:57.007114  7912 raft_consensus.cc:2804] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 08:00:57.007177  7912 raft_consensus.cc:493] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:00:57.007212  7912 raft_consensus.cc:3060] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:00:57.008020  7912 raft_consensus.cc:515] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a495179243a41baa4bbd1ab1763317f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44643 } } peers { permanent_uuid: "f7e522d581c24cebabb4b5319040334b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 34033 } } peers { permanent_uuid: "2290733bc4724005abcfafd52fe405c5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41567 } }
I20260812 08:00:57.008365  7912 leader_election.cc:290] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [CANDIDATE]: Term 2 election: Requested vote from peers f7e522d581c24cebabb4b5319040334b (127.0.186.66:34033), 2290733bc4724005abcfafd52fe405c5 (127.0.186.65:41567)
I20260812 08:00:57.008735  7519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65a1b146d324653b10b452e9c54b954" candidate_uuid: "8a495179243a41baa4bbd1ab1763317f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e522d581c24cebabb4b5319040334b"
I20260812 08:00:57.008867  7519 raft_consensus.cc:3060] T b65a1b146d324653b10b452e9c54b954 P f7e522d581c24cebabb4b5319040334b [term 1 FOLLOWER]: Advancing to term 2
W20260812 08:00:57.009311  7746 leader_election.cc:336] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2290733bc4724005abcfafd52fe405c5 (127.0.186.65:41567): Network error: Client connection negotiation failed: client connection to 127.0.186.65:41567: connect: Connection refused (error 111)
I20260812 08:00:57.009807  7519 raft_consensus.cc:2468] T b65a1b146d324653b10b452e9c54b954 P f7e522d581c24cebabb4b5319040334b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a495179243a41baa4bbd1ab1763317f in term 2.
I20260812 08:00:57.010066  7745 leader_election.cc:304] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a495179243a41baa4bbd1ab1763317f, f7e522d581c24cebabb4b5319040334b; no voters: 2290733bc4724005abcfafd52fe405c5
I20260812 08:00:57.010217  7912 raft_consensus.cc:2804] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:00:57.010402  7912 raft_consensus.cc:697] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [term 2 LEADER]: Becoming Leader. State: Replica: 8a495179243a41baa4bbd1ab1763317f, State: Running, Role: LEADER
I20260812 08:00:57.010710  7912 consensus_queue.cc:237] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a495179243a41baa4bbd1ab1763317f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44643 } } peers { permanent_uuid: "f7e522d581c24cebabb4b5319040334b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 34033 } } peers { permanent_uuid: "2290733bc4724005abcfafd52fe405c5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41567 } }
I20260812 08:00:57.013912  7188 catalog_manager.cc:5719] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f reported cstate change: term changed from 1 to 2, leader changed from 2290733bc4724005abcfafd52fe405c5 (127.0.186.65) to 8a495179243a41baa4bbd1ab1763317f (127.0.186.68). New cstate: current_term: 2 leader_uuid: "8a495179243a41baa4bbd1ab1763317f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a495179243a41baa4bbd1ab1763317f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7e522d581c24cebabb4b5319040334b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 34033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2290733bc4724005abcfafd52fe405c5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41567 } health_report { overall_health: UNKNOWN } } }
I20260812 08:00:57.514925  7519 raft_consensus.cc:1275] T b65a1b146d324653b10b452e9c54b954 P f7e522d581c24cebabb4b5319040334b [term 2 FOLLOWER]: Refusing update from remote peer 8a495179243a41baa4bbd1ab1763317f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260812 08:00:57.515650  7925 consensus_queue.cc:1048] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7e522d581c24cebabb4b5319040334b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 34033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 08:00:57.521414  7746 consensus_peers.cc:597] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f -> Peer 2290733bc4724005abcfafd52fe405c5 (127.0.186.65:41567): Couldn't send request to peer 2290733bc4724005abcfafd52fe405c5. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:41567: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260812 08:01:00.011920  7746 consensus_peers.cc:597] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f -> Peer 2290733bc4724005abcfafd52fe405c5 (127.0.186.65:41567): Couldn't send request to peer 2290733bc4724005abcfafd52fe405c5. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:41567: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260812 08:01:02.116496  7746 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.65:41567: connect: Connection refused (error 111) [suppressed 10 similar messages]
W20260812 08:01:02.579777  7746 consensus_peers.cc:597] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f -> Peer 2290733bc4724005abcfafd52fe405c5 (127.0.186.65:41567): Couldn't send request to peer 2290733bc4724005abcfafd52fe405c5. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:41567: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260812 08:01:05.169122  7746 consensus_peers.cc:597] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f -> Peer 2290733bc4724005abcfafd52fe405c5 (127.0.186.65:41567): Couldn't send request to peer 2290733bc4724005abcfafd52fe405c5. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:41567: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260812 08:01:07.174752  7939 consensus_queue.cc:579] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [LEADER]: Leader has been unable to successfully communicate with peer 2290733bc4724005abcfafd52fe405c5 for more than 10 seconds (10.163s)
W20260812 08:01:07.175802  7746 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.65:41567: connect: Connection refused (error 111) [suppressed 9 similar messages]
I20260812 08:01:07.178277  7819 consensus_queue.cc:237] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8a495179243a41baa4bbd1ab1763317f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44643 } } peers { permanent_uuid: "f7e522d581c24cebabb4b5319040334b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 34033 } } peers { permanent_uuid: "2290733bc4724005abcfafd52fe405c5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41567 } } peers { permanent_uuid: "325a03b321754804b562068549caf8a3" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 37129 } attrs { promote: true } }
I20260812 08:01:07.180159  7519 raft_consensus.cc:1275] T b65a1b146d324653b10b452e9c54b954 P f7e522d581c24cebabb4b5319040334b [term 2 FOLLOWER]: Refusing update from remote peer 8a495179243a41baa4bbd1ab1763317f: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 08:01:07.180629  7961 consensus_queue.cc:1048] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7e522d581c24cebabb4b5319040334b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 34033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260812 08:01:07.181069  7746 consensus_peers.cc:597] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f -> Peer 2290733bc4724005abcfafd52fe405c5 (127.0.186.65:41567): Couldn't send request to peer 2290733bc4724005abcfafd52fe405c5. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:41567: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 08:01:07.182904  7963 raft_consensus.cc:2955] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 325a03b321754804b562068549caf8a3 (127.0.186.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8a495179243a41baa4bbd1ab1763317f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44643 } } peers { permanent_uuid: "f7e522d581c24cebabb4b5319040334b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 34033 } } peers { permanent_uuid: "2290733bc4724005abcfafd52fe405c5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41567 } } peers { permanent_uuid: "325a03b321754804b562068549caf8a3" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 37129 } attrs { promote: true } } }
I20260812 08:01:07.183372  7519 raft_consensus.cc:2955] T b65a1b146d324653b10b452e9c54b954 P f7e522d581c24cebabb4b5319040334b [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 325a03b321754804b562068549caf8a3 (127.0.186.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8a495179243a41baa4bbd1ab1763317f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44643 } } peers { permanent_uuid: "f7e522d581c24cebabb4b5319040334b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 34033 } } peers { permanent_uuid: "2290733bc4724005abcfafd52fe405c5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41567 } } peers { permanent_uuid: "325a03b321754804b562068549caf8a3" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 37129 } attrs { promote: true } } }
I20260812 08:01:07.184871  7169 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet b65a1b146d324653b10b452e9c54b954 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:01:07.185992  7187 catalog_manager.cc:5719] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f reported cstate change: config changed from index -1 to 2, NON_VOTER 325a03b321754804b562068549caf8a3 (127.0.186.67) added. New cstate: current_term: 2 leader_uuid: "8a495179243a41baa4bbd1ab1763317f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8a495179243a41baa4bbd1ab1763317f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7e522d581c24cebabb4b5319040334b" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 34033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2290733bc4724005abcfafd52fe405c5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "325a03b321754804b562068549caf8a3" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 37129 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260812 08:01:07.186518  7742 consensus_peers.cc:597] T b65a1b146d324653b10b452e9c54b954 P 8a495179243a41baa4bbd1ab1763317f -> Peer 325a03b321754804b562068549caf8a3 (127.0.186.67:37129): Couldn't send request to peer 325a03b321754804b562068549caf8a3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b65a1b146d324653b10b452e9c54b954. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:01:07.205292   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 7416
I20260812 08:01:07.218549   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 7572
I20260812 08:01:07.226280   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 7721
I20260812 08:01:07.235342   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 7152
2026-08-12T08:01:07Z chronyd exiting
[       OK ] RaftConsensusNonVoterITest.CatalogManagerAddsNonVoter (12949 ms)
[ RUN      ] RaftConsensusNonVoterITest.TabletServerIsGoneAndBack
I20260812 08:01:07.262121   745 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:01:07.262171   745 ts_itest-base.cc:116] --------------
I20260812 08:01:07.262187   745 ts_itest-base.cc:117] 4 tablet servers
I20260812 08:01:07.262200   745 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:01:07.262213   745 ts_itest-base.cc:119] --------------
2026-08-12T08:01:07Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:01:07Z Disabled control of system clock
I20260812 08:01:07.271196   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:38173
--webserver_interface=127.0.186.126
--webserver_port=0
--builtin_ntp_servers=127.0.186.84:38753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:38173
--raft_prepare_replacement_before_eviction=true with env {}
W20260812 08:01:07.376986  7984 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:07.377235  7984 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:07.377290  7984 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:07.380709  7984 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:01:07.380779  7984 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:07.380806  7984 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:01:07.380828  7984 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:01:07.384672  7984 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:38753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:38173
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:38173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.7984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:07.385800  7984 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:07.386942  7984 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:07.392470  7995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:07.392513  7991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:07.392571  7990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:07.392699  7984 server_base.cc:1061] running on GCE node
I20260812 08:01:07.393190  7984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:07.394084  7984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:07.395239  7984 hybrid_clock.cc:648] HybridClock initialized: now 1786521667395230 us; error 28 us; skew 500 ppm
I20260812 08:01:07.396874  7984 webserver.cc:533] Webserver started at http://127.0.186.126:32775/ using document root <none> and password file <none>
I20260812 08:01:07.397419  7984 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:07.397505  7984 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:07.397706  7984 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:01:07.399195  7984 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "dfdf827d12654081ab005cb08d0609c2"
format_stamp: "Formatted at 2026-08-12 08:01:07 on dist-test-slave-49h3"
I20260812 08:01:07.399655  7984 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "dfdf827d12654081ab005cb08d0609c2"
format_stamp: "Formatted at 2026-08-12 08:01:07 on dist-test-slave-49h3"
I20260812 08:01:07.402969  7984 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:01:07.404954  8001 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:07.405921  7984 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:07.406039  7984 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "dfdf827d12654081ab005cb08d0609c2"
format_stamp: "Formatted at 2026-08-12 08:01:07 on dist-test-slave-49h3"
I20260812 08:01:07.406131  7984 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:07.417943  7984 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:07.418514  7984 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:07.418675  7984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:07.425655  7984 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:38173
I20260812 08:01:07.425663  8064 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:38173 every 8 connection(s)
I20260812 08:01:07.426676  7984 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:01:07.429383  8065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:01:07.434420  8065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfdf827d12654081ab005cb08d0609c2: Bootstrap starting.
I20260812 08:01:07.436105   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 7984
I20260812 08:01:07.436208   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:01:07.436683  8065 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dfdf827d12654081ab005cb08d0609c2: Neither blocks nor log segments found. Creating new log.
I20260812 08:01:07.437530  8065 log.cc:826] T 00000000000000000000000000000000 P dfdf827d12654081ab005cb08d0609c2: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:07.439198  8065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfdf827d12654081ab005cb08d0609c2: No bootstrap required, opened a new log
I20260812 08:01:07.441663  8065 raft_consensus.cc:359] T 00000000000000000000000000000000 P dfdf827d12654081ab005cb08d0609c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfdf827d12654081ab005cb08d0609c2" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 38173 } }
I20260812 08:01:07.441869  8065 raft_consensus.cc:385] T 00000000000000000000000000000000 P dfdf827d12654081ab005cb08d0609c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:01:07.441942  8065 raft_consensus.cc:740] T 00000000000000000000000000000000 P dfdf827d12654081ab005cb08d0609c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfdf827d12654081ab005cb08d0609c2, State: Initialized, Role: FOLLOWER
I20260812 08:01:07.442325  8065 consensus_queue.cc:260] T 00000000000000000000000000000000 P dfdf827d12654081ab005cb08d0609c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfdf827d12654081ab005cb08d0609c2" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 38173 } }
I20260812 08:01:07.442446  8065 raft_consensus.cc:399] T 00000000000000000000000000000000 P dfdf827d12654081ab005cb08d0609c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:01:07.442509  8065 raft_consensus.cc:493] T 00000000000000000000000000000000 P dfdf827d12654081ab005cb08d0609c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:01:07.442616  8065 raft_consensus.cc:3060] T 00000000000000000000000000000000 P dfdf827d12654081ab005cb08d0609c2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:01:07.443354  8065 raft_consensus.cc:515] T 00000000000000000000000000000000 P dfdf827d12654081ab005cb08d0609c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfdf827d12654081ab005cb08d0609c2" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 38173 } }
I20260812 08:01:07.443671  8065 leader_election.cc:304] T 00000000000000000000000000000000 P dfdf827d12654081ab005cb08d0609c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dfdf827d12654081ab005cb08d0609c2; no voters: 
I20260812 08:01:07.443936  8065 leader_election.cc:290] T 00000000000000000000000000000000 P dfdf827d12654081ab005cb08d0609c2 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:01:07.444039  8070 raft_consensus.cc:2804] T 00000000000000000000000000000000 P dfdf827d12654081ab005cb08d0609c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:01:07.444226  8070 raft_consensus.cc:697] T 00000000000000000000000000000000 P dfdf827d12654081ab005cb08d0609c2 [term 1 LEADER]: Becoming Leader. State: Replica: dfdf827d12654081ab005cb08d0609c2, State: Running, Role: LEADER
I20260812 08:01:07.444536  8070 consensus_queue.cc:237] T 00000000000000000000000000000000 P dfdf827d12654081ab005cb08d0609c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfdf827d12654081ab005cb08d0609c2" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 38173 } }
I20260812 08:01:07.444996  8065 sys_catalog.cc:565] T 00000000000000000000000000000000 P dfdf827d12654081ab005cb08d0609c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:01:07.446159  8071 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfdf827d12654081ab005cb08d0609c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dfdf827d12654081ab005cb08d0609c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfdf827d12654081ab005cb08d0609c2" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 38173 } } }
I20260812 08:01:07.446450  8072 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfdf827d12654081ab005cb08d0609c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dfdf827d12654081ab005cb08d0609c2. Latest consensus state: current_term: 1 leader_uuid: "dfdf827d12654081ab005cb08d0609c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfdf827d12654081ab005cb08d0609c2" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 38173 } } }
I20260812 08:01:07.446557  8071 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfdf827d12654081ab005cb08d0609c2 [sys.catalog]: This master's current role is: LEADER
I20260812 08:01:07.446635  8072 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfdf827d12654081ab005cb08d0609c2 [sys.catalog]: This master's current role is: LEADER
I20260812 08:01:07.447525  8080 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:01:07.450048  8080 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:01:07.454942  8080 catalog_manager.cc:1383] Generated new cluster ID: 201765f5af824fc8920ba7b970415d2c
I20260812 08:01:07.455006  8080 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:01:07.466804  8080 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:01:07.467566  8080 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:01:07.477361  8080 catalog_manager.cc:6092] T 00000000000000000000000000000000 P dfdf827d12654081ab005cb08d0609c2: Generated new TSK 0
I20260812 08:01:07.477952  8080 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:01:07.484946   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:0
--local_ip_for_outbound_sockets=127.0.186.65
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:38173
--builtin_ntp_servers=127.0.186.84:38753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--tablet_copy_download_file_inject_latency_ms=180000
--consensus_rpc_timeout_ms=5000 with env {}
W20260812 08:01:07.591204  8090 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:07.591408  8090 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:07.591470  8090 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:07.594703  8090 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:07.594823  8090 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:01:07.598567  8090 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:38753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=5
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=180000
--tserver_master_addrs=127.0.186.126:38173
--never_fsync=true
--heap_profile_path=/tmp/kudu.8090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:07.599598  8090 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:07.600692  8090 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:07.606706  8099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:07.606719  8097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:07.606845  8095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:07.607322  8090 server_base.cc:1061] running on GCE node
I20260812 08:01:07.607689  8090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:07.608217  8090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:07.609375  8090 hybrid_clock.cc:648] HybridClock initialized: now 1786521667609362 us; error 29 us; skew 500 ppm
I20260812 08:01:07.611210  8090 webserver.cc:533] Webserver started at http://127.0.186.65:39279/ using document root <none> and password file <none>
I20260812 08:01:07.611788  8090 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:07.611850  8090 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:07.612027  8090 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:01:07.613569  8090 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "b5ca77770a37434ba93872246814f44a"
format_stamp: "Formatted at 2026-08-12 08:01:07 on dist-test-slave-49h3"
I20260812 08:01:07.614020  8090 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "b5ca77770a37434ba93872246814f44a"
format_stamp: "Formatted at 2026-08-12 08:01:07 on dist-test-slave-49h3"
I20260812 08:01:07.617101  8090 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:01:07.619113  8106 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:07.620028  8090 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:01:07.620131  8090 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "b5ca77770a37434ba93872246814f44a"
format_stamp: "Formatted at 2026-08-12 08:01:07 on dist-test-slave-49h3"
I20260812 08:01:07.620213  8090 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:07.647148  8090 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:07.647827  8090 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:07.648002  8090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:07.648582  8090 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:07.649637  8090 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:01:07.649686  8090 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:07.649740  8090 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:01:07.649770  8090 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:07.658396  8090 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:42451
I20260812 08:01:07.658454  8236 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:42451 every 8 connection(s)
I20260812 08:01:07.659343  8090 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:01:07.661614   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 8090
I20260812 08:01:07.661744   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:01:07.664129   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.66:0
--local_ip_for_outbound_sockets=127.0.186.66
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:38173
--builtin_ntp_servers=127.0.186.84:38753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--tablet_copy_download_file_inject_latency_ms=180000
--consensus_rpc_timeout_ms=5000 with env {}
I20260812 08:01:07.667824  8237 heartbeater.cc:344] Connected to a master server at 127.0.186.126:38173
I20260812 08:01:07.668045  8237 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:07.668572  8237 heartbeater.cc:507] Master 127.0.186.126:38173 requested a full tablet report, sending...
I20260812 08:01:07.670300  8021 ts_manager.cc:194] Registered new tserver with Master: b5ca77770a37434ba93872246814f44a (127.0.186.65:42451)
I20260812 08:01:07.671911  8021 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.65:58089
W20260812 08:01:07.772099  8240 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:07.772310  8240 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:07.772356  8240 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:07.775606  8240 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:07.775714  8240 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.66
I20260812 08:01:07.779531  8240 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:38753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=5
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.186.66
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=180000
--tserver_master_addrs=127.0.186.126:38173
--never_fsync=true
--heap_profile_path=/tmp/kudu.8240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:07.780697  8240 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:07.781826  8240 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:07.788234  8245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:07.788267  8248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:07.788290  8246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:07.788427  8240 server_base.cc:1061] running on GCE node
I20260812 08:01:07.788864  8240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:07.789374  8240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:07.790532  8240 hybrid_clock.cc:648] HybridClock initialized: now 1786521667790519 us; error 30 us; skew 500 ppm
I20260812 08:01:07.792204  8240 webserver.cc:533] Webserver started at http://127.0.186.66:45903/ using document root <none> and password file <none>
I20260812 08:01:07.792759  8240 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:07.792816  8240 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:07.792995  8240 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:01:07.794540  8240 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "26c3805c62084787905d30c43fdf5846"
format_stamp: "Formatted at 2026-08-12 08:01:07 on dist-test-slave-49h3"
I20260812 08:01:07.795011  8240 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "26c3805c62084787905d30c43fdf5846"
format_stamp: "Formatted at 2026-08-12 08:01:07 on dist-test-slave-49h3"
I20260812 08:01:07.798296  8240 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:01:07.800383  8257 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:07.801437  8240 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:07.801574  8240 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "26c3805c62084787905d30c43fdf5846"
format_stamp: "Formatted at 2026-08-12 08:01:07 on dist-test-slave-49h3"
I20260812 08:01:07.801664  8240 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:07.814662  8240 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:07.815308  8240 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:07.815479  8240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:07.816071  8240 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:07.817130  8240 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:01:07.817180  8240 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:07.817233  8240 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:01:07.817260  8240 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:07.826424  8240 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.66:46057
I20260812 08:01:07.826483  8385 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.66:46057 every 8 connection(s)
I20260812 08:01:07.827394  8240 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:01:07.830938   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 8240
I20260812 08:01:07.831069   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:01:07.833632   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.67:0
--local_ip_for_outbound_sockets=127.0.186.67
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:38173
--builtin_ntp_servers=127.0.186.84:38753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--tablet_copy_download_file_inject_latency_ms=180000
--consensus_rpc_timeout_ms=5000 with env {}
I20260812 08:01:07.836386  8386 heartbeater.cc:344] Connected to a master server at 127.0.186.126:38173
I20260812 08:01:07.836640  8386 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:07.837177  8386 heartbeater.cc:507] Master 127.0.186.126:38173 requested a full tablet report, sending...
I20260812 08:01:07.838333  8021 ts_manager.cc:194] Registered new tserver with Master: 26c3805c62084787905d30c43fdf5846 (127.0.186.66:46057)
I20260812 08:01:07.838901  8021 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.66:34519
W20260812 08:01:07.941057  8389 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:07.941337  8389 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:07.941408  8389 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:07.945235  8389 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:07.945362  8389 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.67
I20260812 08:01:07.949440  8389 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:38753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=5
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.186.67
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=180000
--tserver_master_addrs=127.0.186.126:38173
--never_fsync=true
--heap_profile_path=/tmp/kudu.8389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:07.950596  8389 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:07.951706  8389 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:07.958146  8395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:07.958181  8394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:07.958361  8389 server_base.cc:1061] running on GCE node
W20260812 08:01:07.958134  8397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:07.958730  8389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:07.959398  8389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:07.960561  8389 hybrid_clock.cc:648] HybridClock initialized: now 1786521667960546 us; error 27 us; skew 500 ppm
I20260812 08:01:07.962309  8389 webserver.cc:533] Webserver started at http://127.0.186.67:37841/ using document root <none> and password file <none>
I20260812 08:01:07.962852  8389 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:07.962904  8389 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:07.963099  8389 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:01:07.964612  8389 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "1b9f9cecaa3d4db4840827af64e6a055"
format_stamp: "Formatted at 2026-08-12 08:01:07 on dist-test-slave-49h3"
I20260812 08:01:07.965067  8389 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "1b9f9cecaa3d4db4840827af64e6a055"
format_stamp: "Formatted at 2026-08-12 08:01:07 on dist-test-slave-49h3"
I20260812 08:01:07.968372  8389 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:01:07.970677  8405 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:07.971836  8389 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:01:07.971946  8389 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "1b9f9cecaa3d4db4840827af64e6a055"
format_stamp: "Formatted at 2026-08-12 08:01:07 on dist-test-slave-49h3"
I20260812 08:01:07.972051  8389 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:07.986321  8389 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:07.986971  8389 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:07.987157  8389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:07.987723  8389 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:07.988735  8389 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:01:07.988785  8389 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:07.988837  8389 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:01:07.988864  8389 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:07.997642  8389 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.67:38335
I20260812 08:01:07.997679  8534 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.67:38335 every 8 connection(s)
I20260812 08:01:07.998564  8389 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:01:08.000535   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 8389
I20260812 08:01:08.000653   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:01:08.002873   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.68:0
--local_ip_for_outbound_sockets=127.0.186.68
--webserver_interface=127.0.186.68
--webserver_port=0
--tserver_master_addrs=127.0.186.126:38173
--builtin_ntp_servers=127.0.186.84:38753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--tablet_copy_download_file_inject_latency_ms=180000
--consensus_rpc_timeout_ms=5000 with env {}
I20260812 08:01:08.006850  8535 heartbeater.cc:344] Connected to a master server at 127.0.186.126:38173
I20260812 08:01:08.007112  8535 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:08.007731  8535 heartbeater.cc:507] Master 127.0.186.126:38173 requested a full tablet report, sending...
I20260812 08:01:08.009156  8021 ts_manager.cc:194] Registered new tserver with Master: 1b9f9cecaa3d4db4840827af64e6a055 (127.0.186.67:38335)
I20260812 08:01:08.009912  8021 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.67:50139
W20260812 08:01:08.109927  8538 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:08.110138  8538 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:08.110185  8538 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:08.113464  8538 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:08.113579  8538 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.68
I20260812 08:01:08.117342  8538 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:38753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=5
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.0.186.68
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=180000
--tserver_master_addrs=127.0.186.126:38173
--never_fsync=true
--heap_profile_path=/tmp/kudu.8538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.68

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:08.118477  8538 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:08.119696  8538 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:08.126206  8548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:08.126340  8538 server_base.cc:1061] running on GCE node
W20260812 08:01:08.126268  8545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:08.126197  8544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:08.126804  8538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:08.127367  8538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:08.128528  8538 hybrid_clock.cc:648] HybridClock initialized: now 1786521668128514 us; error 33 us; skew 500 ppm
I20260812 08:01:08.130388  8538 webserver.cc:533] Webserver started at http://127.0.186.68:35649/ using document root <none> and password file <none>
I20260812 08:01:08.130959  8538 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:08.131019  8538 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:08.131206  8538 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:01:08.132789  8538 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "2e0a5e5978ec4066a49535da2b5c62df"
format_stamp: "Formatted at 2026-08-12 08:01:08 on dist-test-slave-49h3"
I20260812 08:01:08.133265  8538 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "2e0a5e5978ec4066a49535da2b5c62df"
format_stamp: "Formatted at 2026-08-12 08:01:08 on dist-test-slave-49h3"
I20260812 08:01:08.136583  8538 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.001s	sys 0.001s
I20260812 08:01:08.138744  8556 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:08.139731  8538 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:01:08.139839  8538 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "2e0a5e5978ec4066a49535da2b5c62df"
format_stamp: "Formatted at 2026-08-12 08:01:08 on dist-test-slave-49h3"
I20260812 08:01:08.139926  8538 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:08.175211  8538 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:08.175921  8538 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:08.176189  8538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:08.176826  8538 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:08.177934  8538 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:01:08.177984  8538 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:08.178023  8538 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:01:08.178045  8538 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:08.187217  8538 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.68:44017
I20260812 08:01:08.187250  8685 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.68:44017 every 8 connection(s)
I20260812 08:01:08.188237  8538 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20260812 08:01:08.190086   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 8538
I20260812 08:01:08.190218   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20260812 08:01:08.196841  8686 heartbeater.cc:344] Connected to a master server at 127.0.186.126:38173
I20260812 08:01:08.197072  8686 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:08.197624  8686 heartbeater.cc:507] Master 127.0.186.126:38173 requested a full tablet report, sending...
I20260812 08:01:08.198633  8021 ts_manager.cc:194] Registered new tserver with Master: 2e0a5e5978ec4066a49535da2b5c62df (127.0.186.68:44017)
I20260812 08:01:08.199177  8021 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.68:37195
I20260812 08:01:08.204672   745 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20260812 08:01:08.213671  8021 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44288:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
I20260812 08:01:08.233335  8615 tablet_service.cc:1511] Processing CreateTablet for tablet 43c7002f0c8d4b6b8ea0faadafa56841 (DEFAULT_TABLE table=TestTable [id=9f7dc0f5cb82403f8954be27cfd93d23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:01:08.233199  8310 tablet_service.cc:1511] Processing CreateTablet for tablet 43c7002f0c8d4b6b8ea0faadafa56841 (DEFAULT_TABLE table=TestTable [id=9f7dc0f5cb82403f8954be27cfd93d23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:01:08.234361  8615 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43c7002f0c8d4b6b8ea0faadafa56841. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:01:08.234361  8310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43c7002f0c8d4b6b8ea0faadafa56841. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:01:08.234293  8162 tablet_service.cc:1511] Processing CreateTablet for tablet 43c7002f0c8d4b6b8ea0faadafa56841 (DEFAULT_TABLE table=TestTable [id=9f7dc0f5cb82403f8954be27cfd93d23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:01:08.235106  8162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43c7002f0c8d4b6b8ea0faadafa56841. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:01:08.239861  8707 tablet_bootstrap.cc:492] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a: Bootstrap starting.
I20260812 08:01:08.240922  8709 tablet_bootstrap.cc:492] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846: Bootstrap starting.
I20260812 08:01:08.242501  8707 tablet_bootstrap.cc:654] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a: Neither blocks nor log segments found. Creating new log.
I20260812 08:01:08.242822  8709 tablet_bootstrap.cc:654] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846: Neither blocks nor log segments found. Creating new log.
I20260812 08:01:08.243245  8707 log.cc:826] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:08.243446  8709 log.cc:826] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:08.244086  8708 tablet_bootstrap.cc:492] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df: Bootstrap starting.
I20260812 08:01:08.245040  8709 tablet_bootstrap.cc:492] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846: No bootstrap required, opened a new log
I20260812 08:01:08.245121  8707 tablet_bootstrap.cc:492] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a: No bootstrap required, opened a new log
I20260812 08:01:08.245235  8709 ts_tablet_manager.cc:1403] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846: Time spent bootstrapping tablet: real 0.004s	user 0.003s	sys 0.000s
I20260812 08:01:08.245314  8707 ts_tablet_manager.cc:1403] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a: Time spent bootstrapping tablet: real 0.006s	user 0.004s	sys 0.000s
I20260812 08:01:08.246248  8708 tablet_bootstrap.cc:654] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df: Neither blocks nor log segments found. Creating new log.
I20260812 08:01:08.247181  8708 log.cc:826] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:08.248054  8709 raft_consensus.cc:359] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } }
I20260812 08:01:08.248273  8709 raft_consensus.cc:385] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:01:08.248332  8709 raft_consensus.cc:740] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26c3805c62084787905d30c43fdf5846, State: Initialized, Role: FOLLOWER
I20260812 08:01:08.248440  8707 raft_consensus.cc:359] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } }
I20260812 08:01:08.248615  8707 raft_consensus.cc:385] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:01:08.248708  8707 raft_consensus.cc:740] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5ca77770a37434ba93872246814f44a, State: Initialized, Role: FOLLOWER
I20260812 08:01:08.248775  8709 consensus_queue.cc:260] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } }
I20260812 08:01:08.249014  8708 tablet_bootstrap.cc:492] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df: No bootstrap required, opened a new log
I20260812 08:01:08.249207  8708 ts_tablet_manager.cc:1403] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:01:08.249123  8707 consensus_queue.cc:260] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } }
I20260812 08:01:08.249812  8709 ts_tablet_manager.cc:1434] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 08:01:08.250046  8707 ts_tablet_manager.cc:1434] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 08:01:08.251605  8708 raft_consensus.cc:359] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } }
I20260812 08:01:08.251757  8708 raft_consensus.cc:385] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:01:08.251787  8708 raft_consensus.cc:740] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e0a5e5978ec4066a49535da2b5c62df, State: Initialized, Role: FOLLOWER
I20260812 08:01:08.252103  8708 consensus_queue.cc:260] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } }
I20260812 08:01:08.252206  8237 heartbeater.cc:499] Master 127.0.186.126:38173 was elected leader, sending a full tablet report...
I20260812 08:01:08.252715  8686 heartbeater.cc:499] Master 127.0.186.126:38173 was elected leader, sending a full tablet report...
I20260812 08:01:08.252810  8708 ts_tablet_manager.cc:1434] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df: Time spent starting tablet: real 0.004s	user 0.003s	sys 0.000s
I20260812 08:01:08.253230  8386 heartbeater.cc:499] Master 127.0.186.126:38173 was elected leader, sending a full tablet report...
W20260812 08:01:08.328451  8387 tablet.cc:2406] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:01:08.410663  8238 tablet.cc:2406] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:01:08.439165  8687 tablet.cc:2406] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:01:08.479475  8714 raft_consensus.cc:493] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:01:08.479704  8714 raft_consensus.cc:515] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } }
I20260812 08:01:08.480924  8714 leader_election.cc:290] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5ca77770a37434ba93872246814f44a (127.0.186.65:42451), 2e0a5e5978ec4066a49535da2b5c62df (127.0.186.68:44017)
I20260812 08:01:08.485205  8187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c7002f0c8d4b6b8ea0faadafa56841" candidate_uuid: "26c3805c62084787905d30c43fdf5846" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5ca77770a37434ba93872246814f44a" is_pre_election: true
I20260812 08:01:08.485553  8187 raft_consensus.cc:2468] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26c3805c62084787905d30c43fdf5846 in term 0.
I20260812 08:01:08.485792  8637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c7002f0c8d4b6b8ea0faadafa56841" candidate_uuid: "26c3805c62084787905d30c43fdf5846" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e0a5e5978ec4066a49535da2b5c62df" is_pre_election: true
I20260812 08:01:08.486081  8637 raft_consensus.cc:2468] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26c3805c62084787905d30c43fdf5846 in term 0.
I20260812 08:01:08.486078  8260 leader_election.cc:304] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26c3805c62084787905d30c43fdf5846, b5ca77770a37434ba93872246814f44a; no voters: 
I20260812 08:01:08.486332  8714 raft_consensus.cc:2804] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:01:08.486406  8714 raft_consensus.cc:493] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:01:08.486451  8714 raft_consensus.cc:3060] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:01:08.487360  8714 raft_consensus.cc:515] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } }
I20260812 08:01:08.487713  8714 leader_election.cc:290] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [CANDIDATE]: Term 1 election: Requested vote from peers b5ca77770a37434ba93872246814f44a (127.0.186.65:42451), 2e0a5e5978ec4066a49535da2b5c62df (127.0.186.68:44017)
I20260812 08:01:08.488076  8187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c7002f0c8d4b6b8ea0faadafa56841" candidate_uuid: "26c3805c62084787905d30c43fdf5846" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5ca77770a37434ba93872246814f44a"
I20260812 08:01:08.488170  8637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c7002f0c8d4b6b8ea0faadafa56841" candidate_uuid: "26c3805c62084787905d30c43fdf5846" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e0a5e5978ec4066a49535da2b5c62df"
I20260812 08:01:08.488202  8187 raft_consensus.cc:3060] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:01:08.488314  8637 raft_consensus.cc:3060] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:01:08.489068  8187 raft_consensus.cc:2468] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26c3805c62084787905d30c43fdf5846 in term 1.
I20260812 08:01:08.489158  8637 raft_consensus.cc:2468] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26c3805c62084787905d30c43fdf5846 in term 1.
I20260812 08:01:08.489344  8260 leader_election.cc:304] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26c3805c62084787905d30c43fdf5846, b5ca77770a37434ba93872246814f44a; no voters: 
I20260812 08:01:08.489523  8714 raft_consensus.cc:2804] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:01:08.489737  8714 raft_consensus.cc:697] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [term 1 LEADER]: Becoming Leader. State: Replica: 26c3805c62084787905d30c43fdf5846, State: Running, Role: LEADER
I20260812 08:01:08.490056  8714 consensus_queue.cc:237] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } }
I20260812 08:01:08.493115  8018 catalog_manager.cc:5719] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 reported cstate change: term changed from 0 to 1, leader changed from <none> to 26c3805c62084787905d30c43fdf5846 (127.0.186.66). New cstate: current_term: 1 leader_uuid: "26c3805c62084787905d30c43fdf5846" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } health_report { overall_health: UNKNOWN } } }
I20260812 08:01:08.519958   745 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20260812 08:01:08.521507   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver b5ca77770a37434ba93872246814f44a to finish bootstrapping
I20260812 08:01:08.527460   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 26c3805c62084787905d30c43fdf5846 to finish bootstrapping
I20260812 08:01:08.532192   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 2e0a5e5978ec4066a49535da2b5c62df to finish bootstrapping
I20260812 08:01:08.544899   745 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
I20260812 08:01:08.572811  8187 raft_consensus.cc:1275] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [term 1 FOLLOWER]: Refusing update from remote peer 26c3805c62084787905d30c43fdf5846: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:01:08.572813  8637 raft_consensus.cc:1275] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [term 1 FOLLOWER]: Refusing update from remote peer 26c3805c62084787905d30c43fdf5846: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:01:08.573506  8714 consensus_queue.cc:1048] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:01:08.573727  8721 consensus_queue.cc:1048] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:01:08.607879   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 8240
W20260812 08:01:08.621572  8109 connection.cc:573] server connection from 127.0.186.66:45041 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 08:01:08.621649  8559 connection.cc:573] server connection from 127.0.186.66:40347 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 08:01:09.012202  8535 heartbeater.cc:499] Master 127.0.186.126:38173 was elected leader, sending a full tablet report...
I20260812 08:01:10.108639  8760 raft_consensus.cc:493] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 26c3805c62084787905d30c43fdf5846)
I20260812 08:01:10.108789  8760 raft_consensus.cc:515] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } }
I20260812 08:01:10.109596  8762 raft_consensus.cc:493] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 26c3805c62084787905d30c43fdf5846)
I20260812 08:01:10.109711  8762 raft_consensus.cc:515] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } }
I20260812 08:01:10.109957  8760 leader_election.cc:290] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 26c3805c62084787905d30c43fdf5846 (127.0.186.66:46057), 2e0a5e5978ec4066a49535da2b5c62df (127.0.186.68:44017)
W20260812 08:01:10.110795  8108 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.66:46057: connect: Connection refused (error 111)
I20260812 08:01:10.110843  8762 leader_election.cc:290] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5ca77770a37434ba93872246814f44a (127.0.186.65:42451), 26c3805c62084787905d30c43fdf5846 (127.0.186.66:46057)
W20260812 08:01:10.111343  8557 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.66:46057: connect: Connection refused (error 111)
W20260812 08:01:10.111899  8108 leader_election.cc:336] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 26c3805c62084787905d30c43fdf5846 (127.0.186.66:46057): Network error: Client connection negotiation failed: client connection to 127.0.186.66:46057: connect: Connection refused (error 111)
W20260812 08:01:10.112687  8557 leader_election.cc:336] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 26c3805c62084787905d30c43fdf5846 (127.0.186.66:46057): Network error: Client connection negotiation failed: client connection to 127.0.186.66:46057: connect: Connection refused (error 111)
I20260812 08:01:10.114527  8637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c7002f0c8d4b6b8ea0faadafa56841" candidate_uuid: "b5ca77770a37434ba93872246814f44a" candidate_term: 2 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: false dest_uuid: "2e0a5e5978ec4066a49535da2b5c62df" is_pre_election: true
I20260812 08:01:10.114619  8187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c7002f0c8d4b6b8ea0faadafa56841" candidate_uuid: "2e0a5e5978ec4066a49535da2b5c62df" candidate_term: 2 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: false dest_uuid: "b5ca77770a37434ba93872246814f44a" is_pre_election: true
I20260812 08:01:10.114676  8637 raft_consensus.cc:2468] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5ca77770a37434ba93872246814f44a in term 1.
I20260812 08:01:10.114760  8187 raft_consensus.cc:2468] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e0a5e5978ec4066a49535da2b5c62df in term 1.
I20260812 08:01:10.115020  8110 leader_election.cc:304] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2e0a5e5978ec4066a49535da2b5c62df, b5ca77770a37434ba93872246814f44a; no voters: 26c3805c62084787905d30c43fdf5846
I20260812 08:01:10.115053  8559 leader_election.cc:304] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2e0a5e5978ec4066a49535da2b5c62df, b5ca77770a37434ba93872246814f44a; no voters: 26c3805c62084787905d30c43fdf5846
I20260812 08:01:10.115247  8760 raft_consensus.cc:2804] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 08:01:10.115302  8760 raft_consensus.cc:493] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 26c3805c62084787905d30c43fdf5846)
I20260812 08:01:10.115338  8760 raft_consensus.cc:3060] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:01:10.116103  8760 raft_consensus.cc:515] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } }
I20260812 08:01:10.116405  8760 leader_election.cc:290] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [CANDIDATE]: Term 2 election: Requested vote from peers 26c3805c62084787905d30c43fdf5846 (127.0.186.66:46057), 2e0a5e5978ec4066a49535da2b5c62df (127.0.186.68:44017)
I20260812 08:01:10.116559  8762 raft_consensus.cc:2804] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 08:01:10.116616  8762 raft_consensus.cc:493] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [term 1 FOLLOWER]: Starting leader election (detected failure of leader 26c3805c62084787905d30c43fdf5846)
I20260812 08:01:10.116657  8762 raft_consensus.cc:3060] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:01:10.116848  8637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c7002f0c8d4b6b8ea0faadafa56841" candidate_uuid: "b5ca77770a37434ba93872246814f44a" candidate_term: 2 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: false dest_uuid: "2e0a5e5978ec4066a49535da2b5c62df"
W20260812 08:01:10.117389  8108 leader_election.cc:336] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 26c3805c62084787905d30c43fdf5846 (127.0.186.66:46057): Network error: Client connection negotiation failed: client connection to 127.0.186.66:46057: connect: Connection refused (error 111)
I20260812 08:01:10.117635  8762 raft_consensus.cc:515] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } }
I20260812 08:01:10.117862  8637 raft_consensus.cc:2393] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b5ca77770a37434ba93872246814f44a in current term 2: Already voted for candidate 2e0a5e5978ec4066a49535da2b5c62df in this term.
I20260812 08:01:10.117980  8762 leader_election.cc:290] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [CANDIDATE]: Term 2 election: Requested vote from peers b5ca77770a37434ba93872246814f44a (127.0.186.65:42451), 26c3805c62084787905d30c43fdf5846 (127.0.186.66:46057)
I20260812 08:01:10.118244  8110 leader_election.cc:304] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5ca77770a37434ba93872246814f44a; no voters: 26c3805c62084787905d30c43fdf5846, 2e0a5e5978ec4066a49535da2b5c62df
I20260812 08:01:10.118315  8187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c7002f0c8d4b6b8ea0faadafa56841" candidate_uuid: "2e0a5e5978ec4066a49535da2b5c62df" candidate_term: 2 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: false dest_uuid: "b5ca77770a37434ba93872246814f44a"
I20260812 08:01:10.118449  8187 raft_consensus.cc:2393] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 2e0a5e5978ec4066a49535da2b5c62df in current term 2: Already voted for candidate b5ca77770a37434ba93872246814f44a in this term.
I20260812 08:01:10.118620  8760 raft_consensus.cc:2749] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260812 08:01:10.118971  8557 leader_election.cc:336] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 26c3805c62084787905d30c43fdf5846 (127.0.186.66:46057): Network error: Client connection negotiation failed: client connection to 127.0.186.66:46057: connect: Connection refused (error 111)
I20260812 08:01:10.119031  8557 leader_election.cc:304] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2e0a5e5978ec4066a49535da2b5c62df; no voters: 26c3805c62084787905d30c43fdf5846, b5ca77770a37434ba93872246814f44a
I20260812 08:01:10.119179  8762 raft_consensus.cc:2749] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 08:01:11.659403  8772 raft_consensus.cc:493] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:01:11.659541  8772 raft_consensus.cc:515] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } }
I20260812 08:01:11.660018  8772 leader_election.cc:290] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b5ca77770a37434ba93872246814f44a (127.0.186.65:42451), 26c3805c62084787905d30c43fdf5846 (127.0.186.66:46057)
I20260812 08:01:11.660486  8187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c7002f0c8d4b6b8ea0faadafa56841" candidate_uuid: "2e0a5e5978ec4066a49535da2b5c62df" candidate_term: 3 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: false dest_uuid: "b5ca77770a37434ba93872246814f44a" is_pre_election: true
I20260812 08:01:11.660647  8187 raft_consensus.cc:2468] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e0a5e5978ec4066a49535da2b5c62df in term 2.
I20260812 08:01:11.661008  8559 leader_election.cc:304] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e0a5e5978ec4066a49535da2b5c62df, b5ca77770a37434ba93872246814f44a; no voters: 
I20260812 08:01:11.661202  8772 raft_consensus.cc:2804] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 08:01:11.661270  8772 raft_consensus.cc:493] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:01:11.661310  8772 raft_consensus.cc:3060] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [term 2 FOLLOWER]: Advancing to term 3
W20260812 08:01:11.661496  8557 leader_election.cc:336] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 26c3805c62084787905d30c43fdf5846 (127.0.186.66:46057): Network error: Client connection negotiation failed: client connection to 127.0.186.66:46057: connect: Connection refused (error 111)
I20260812 08:01:11.662258  8772 raft_consensus.cc:515] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } }
I20260812 08:01:11.662585  8772 leader_election.cc:290] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [CANDIDATE]: Term 3 election: Requested vote from peers b5ca77770a37434ba93872246814f44a (127.0.186.65:42451), 26c3805c62084787905d30c43fdf5846 (127.0.186.66:46057)
I20260812 08:01:11.662950  8187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c7002f0c8d4b6b8ea0faadafa56841" candidate_uuid: "2e0a5e5978ec4066a49535da2b5c62df" candidate_term: 3 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: false dest_uuid: "b5ca77770a37434ba93872246814f44a"
I20260812 08:01:11.663079  8187 raft_consensus.cc:3060] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [term 2 FOLLOWER]: Advancing to term 3
W20260812 08:01:11.663575  8557 leader_election.cc:336] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 26c3805c62084787905d30c43fdf5846 (127.0.186.66:46057): Network error: Client connection negotiation failed: client connection to 127.0.186.66:46057: connect: Connection refused (error 111)
I20260812 08:01:11.664048  8187 raft_consensus.cc:2468] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e0a5e5978ec4066a49535da2b5c62df in term 3.
I20260812 08:01:11.664363  8559 leader_election.cc:304] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2e0a5e5978ec4066a49535da2b5c62df, b5ca77770a37434ba93872246814f44a; no voters: 26c3805c62084787905d30c43fdf5846
I20260812 08:01:11.664561  8772 raft_consensus.cc:2804] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [term 3 FOLLOWER]: Leader election won for term 3
I20260812 08:01:11.664745  8772 raft_consensus.cc:697] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [term 3 LEADER]: Becoming Leader. State: Replica: 2e0a5e5978ec4066a49535da2b5c62df, State: Running, Role: LEADER
I20260812 08:01:11.665117  8772 consensus_queue.cc:237] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 10, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } }
I20260812 08:01:11.667460  8019 catalog_manager.cc:5719] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df reported cstate change: term changed from 1 to 3, leader changed from 26c3805c62084787905d30c43fdf5846 (127.0.186.66) to 2e0a5e5978ec4066a49535da2b5c62df (127.0.186.68). New cstate: current_term: 3 leader_uuid: "2e0a5e5978ec4066a49535da2b5c62df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } health_report { overall_health: HEALTHY } } }
W20260812 08:01:12.068181  8557 consensus_peers.cc:597] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df -> Peer 26c3805c62084787905d30c43fdf5846 (127.0.186.66:46057): Couldn't send request to peer 26c3805c62084787905d30c43fdf5846. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.66:46057: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 08:01:12.158102  8187 raft_consensus.cc:1275] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [term 3 FOLLOWER]: Refusing update from remote peer 2e0a5e5978ec4066a49535da2b5c62df: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 3 index: 11. (index mismatch)
I20260812 08:01:12.158650  8772 consensus_queue.cc:1048] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260812 08:01:14.832468  8557 consensus_peers.cc:597] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df -> Peer 26c3805c62084787905d30c43fdf5846 (127.0.186.66:46057): Couldn't send request to peer 26c3805c62084787905d30c43fdf5846. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.66:46057: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260812 08:01:15.313918  8557 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.66:46057: connect: Connection refused (error 111) [suppressed 9 similar messages]
I20260812 08:01:16.731771  8772 consensus_queue.cc:579] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [LEADER]: Leader has been unable to successfully communicate with peer 26c3805c62084787905d30c43fdf5846 for more than 5 seconds (5.066s)
I20260812 08:01:16.735320  8637 consensus_queue.cc:237] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 3.11, Last appended by leader: 10, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } } peers { permanent_uuid: "1b9f9cecaa3d4db4840827af64e6a055" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 38335 } attrs { promote: true } }
I20260812 08:01:16.736943  8187 raft_consensus.cc:1275] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [term 3 FOLLOWER]: Refusing update from remote peer 2e0a5e5978ec4066a49535da2b5c62df: Log matching property violated. Preceding OpId in replica: term: 3 index: 11. Preceding OpId from leader: term: 3 index: 12. (index mismatch)
I20260812 08:01:16.737736  8795 consensus_queue.cc:1048] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260812 08:01:16.738016  8557 consensus_peers.cc:597] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df -> Peer 26c3805c62084787905d30c43fdf5846 (127.0.186.66:46057): Couldn't send request to peer 26c3805c62084787905d30c43fdf5846. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.66:46057: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 08:01:16.739863  8794 raft_consensus.cc:2955] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [term 3 LEADER]: Committing config change with OpId 3.12: config changed from index -1 to 12, NON_VOTER 1b9f9cecaa3d4db4840827af64e6a055 (127.0.186.67) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } } peers { permanent_uuid: "1b9f9cecaa3d4db4840827af64e6a055" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 38335 } attrs { promote: true } } }
I20260812 08:01:16.740406  8187 raft_consensus.cc:2955] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [term 3 FOLLOWER]: Committing config change with OpId 3.12: config changed from index -1 to 12, NON_VOTER 1b9f9cecaa3d4db4840827af64e6a055 (127.0.186.67) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } } peers { permanent_uuid: "1b9f9cecaa3d4db4840827af64e6a055" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 38335 } attrs { promote: true } } }
I20260812 08:01:16.741616  8005 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 43c7002f0c8d4b6b8ea0faadafa56841 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:01:16.743690  8020 catalog_manager.cc:5719] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df reported cstate change: config changed from index -1 to 12, NON_VOTER 1b9f9cecaa3d4db4840827af64e6a055 (127.0.186.67) added. New cstate: current_term: 3 leader_uuid: "2e0a5e5978ec4066a49535da2b5c62df" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b9f9cecaa3d4db4840827af64e6a055" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 38335 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260812 08:01:16.744642  8557 consensus_peers.cc:597] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df -> Peer 1b9f9cecaa3d4db4840827af64e6a055 (127.0.186.67:38335): Couldn't send request to peer 1b9f9cecaa3d4db4840827af64e6a055. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43c7002f0c8d4b6b8ea0faadafa56841. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:01:16.768253   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.66:46057
--local_ip_for_outbound_sockets=127.0.186.66
--tserver_master_addrs=127.0.186.126:38173
--webserver_port=45903
--webserver_interface=127.0.186.66
--builtin_ntp_servers=127.0.186.84:38753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--tablet_copy_download_file_inject_latency_ms=180000
--consensus_rpc_timeout_ms=5000 with env {}
W20260812 08:01:16.872833  8807 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:16.873049  8807 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:16.873103  8807 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:16.876389  8807 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:16.876502  8807 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.66
I20260812 08:01:16.880270  8807 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:38753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=5
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.66:46057
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.186.66
--webserver_port=45903
--tablet_copy_download_file_inject_latency_ms=180000
--tserver_master_addrs=127.0.186.126:38173
--never_fsync=true
--heap_profile_path=/tmp/kudu.8807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:16.881353  8807 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:16.882515  8807 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:16.888803  8813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:16.889015  8807 server_base.cc:1061] running on GCE node
W20260812 08:01:16.889029  8815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:16.889066  8812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:16.889500  8807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:16.890051  8807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:16.891206  8807 hybrid_clock.cc:648] HybridClock initialized: now 1786521676891187 us; error 31 us; skew 500 ppm
I20260812 08:01:16.892963  8807 webserver.cc:533] Webserver started at http://127.0.186.66:45903/ using document root <none> and password file <none>
I20260812 08:01:16.893684  8807 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:16.893810  8807 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:16.901856  8807 fs_manager.cc:714] Time spent opening directory manager: real 0.007s	user 0.003s	sys 0.000s
I20260812 08:01:16.904318  8822 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:16.905588  8807 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:01:16.905751  8807 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "26c3805c62084787905d30c43fdf5846"
format_stamp: "Formatted at 2026-08-12 08:01:07 on dist-test-slave-49h3"
I20260812 08:01:16.906324  8807 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:16.920408  8807 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:16.921088  8807 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:16.921265  8807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:16.921872  8807 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:16.923125  8830 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:01:16.926860  8807 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:01:16.926923  8807 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:01:16.926986  8807 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:01:16.928519  8807 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:01:16.928566  8807 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:01:16.928934  8830 tablet_bootstrap.cc:492] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846: Bootstrap starting.
I20260812 08:01:16.938900  8807 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.66:46057
I20260812 08:01:16.939062  8954 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.66:46057 every 8 connection(s)
I20260812 08:01:16.939908  8807 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.TabletServerIsGoneAndBack.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:01:16.944854   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 8807
I20260812 08:01:16.946158  8830 log.cc:826] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:16.956718  8955 heartbeater.cc:344] Connected to a master server at 127.0.186.126:38173
I20260812 08:01:16.956988  8955 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:16.957659  8955 heartbeater.cc:507] Master 127.0.186.126:38173 requested a full tablet report, sending...
I20260812 08:01:16.959877  8019 ts_manager.cc:194] Re-registered known tserver with Master: 26c3805c62084787905d30c43fdf5846 (127.0.186.66:46057)
I20260812 08:01:16.960891  8019 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.66:36363
I20260812 08:01:16.967868  8830 tablet_bootstrap.cc:492] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=450 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:01:16.968230  8830 tablet_bootstrap.cc:492] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846: Bootstrap complete.
I20260812 08:01:16.969101  8830 ts_tablet_manager.cc:1403] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846: Time spent bootstrapping tablet: real 0.040s	user 0.032s	sys 0.004s
I20260812 08:01:16.971184  8830 raft_consensus.cc:359] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } }
I20260812 08:01:16.971493  8830 raft_consensus.cc:740] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26c3805c62084787905d30c43fdf5846, State: Initialized, Role: FOLLOWER
I20260812 08:01:16.971931  8830 consensus_queue.cc:260] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 1.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } }
I20260812 08:01:16.972560  8955 heartbeater.cc:499] Master 127.0.186.126:38173 was elected leader, sending a full tablet report...
I20260812 08:01:16.972800  8830 ts_tablet_manager.cc:1434] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846: Time spent starting tablet: real 0.004s	user 0.007s	sys 0.000s
I20260812 08:01:17.345022  8962 ts_tablet_manager.cc:933] T 43c7002f0c8d4b6b8ea0faadafa56841 P 1b9f9cecaa3d4db4840827af64e6a055: Initiating tablet copy from peer 2e0a5e5978ec4066a49535da2b5c62df (127.0.186.68:44017)
I20260812 08:01:17.345698  8962 tablet_copy_client.cc:323] T 43c7002f0c8d4b6b8ea0faadafa56841 P 1b9f9cecaa3d4db4840827af64e6a055: tablet copy: Beginning tablet copy session from remote peer at address 127.0.186.68:44017
I20260812 08:01:17.350291  8659 tablet_copy_service.cc:140] P 2e0a5e5978ec4066a49535da2b5c62df: Received BeginTabletCopySession request for tablet 43c7002f0c8d4b6b8ea0faadafa56841 from peer 1b9f9cecaa3d4db4840827af64e6a055 ({username='slave'} at 127.0.186.67:52705)
I20260812 08:01:17.350459  8659 tablet_copy_service.cc:161] P 2e0a5e5978ec4066a49535da2b5c62df: Beginning new tablet copy session on tablet 43c7002f0c8d4b6b8ea0faadafa56841 from peer 1b9f9cecaa3d4db4840827af64e6a055 at {username='slave'} at 127.0.186.67:52705: session id = 1b9f9cecaa3d4db4840827af64e6a055-43c7002f0c8d4b6b8ea0faadafa56841
I20260812 08:01:17.351577  8659 tablet_copy_source_session.cc:215] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 08:01:17.351635  8903 raft_consensus.cc:3060] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [term 1 FOLLOWER]: Advancing to term 3
I20260812 08:01:17.353932  8903 raft_consensus.cc:1275] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [term 3 FOLLOWER]: Refusing update from remote peer 2e0a5e5978ec4066a49535da2b5c62df: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 3 index: 11. (index mismatch)
I20260812 08:01:17.354955  8961 consensus_queue.cc:1050] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [LEADER]: Got LMP mismatch error from peer: Peer: permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260812 08:01:17.355439  8962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43c7002f0c8d4b6b8ea0faadafa56841. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:01:17.357220  8903 raft_consensus.cc:2955] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [term 3 FOLLOWER]: Committing config change with OpId 3.12: config changed from index -1 to 12, NON_VOTER 1b9f9cecaa3d4db4840827af64e6a055 (127.0.186.67) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } } peers { permanent_uuid: "1b9f9cecaa3d4db4840827af64e6a055" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 38335 } attrs { promote: true } } }
I20260812 08:01:17.360944  8962 tablet_copy_client.cc:806] T 43c7002f0c8d4b6b8ea0faadafa56841 P 1b9f9cecaa3d4db4840827af64e6a055: tablet copy: Starting download of 0 data blocks...
I20260812 08:01:17.361217  8962 tablet_copy_client.cc:670] T 43c7002f0c8d4b6b8ea0faadafa56841 P 1b9f9cecaa3d4db4840827af64e6a055: tablet copy: Starting download of 1 WAL segments...
I20260812 08:01:17.362897  8963 tablet_copy_client.cc:1025] T 43c7002f0c8d4b6b8ea0faadafa56841 P 1b9f9cecaa3d4db4840827af64e6a055: tablet copy: Injecting latency into file download: 180000
I20260812 08:01:17.786885  8637 consensus_queue.cc:237] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 3.12, Last appended by leader: 10, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } }
I20260812 08:01:17.788295  8187 raft_consensus.cc:1275] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [term 3 FOLLOWER]: Refusing update from remote peer 2e0a5e5978ec4066a49535da2b5c62df: Log matching property violated. Preceding OpId in replica: term: 3 index: 12. Preceding OpId from leader: term: 3 index: 13. (index mismatch)
I20260812 08:01:17.788295  8903 raft_consensus.cc:1275] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [term 3 FOLLOWER]: Refusing update from remote peer 2e0a5e5978ec4066a49535da2b5c62df: Log matching property violated. Preceding OpId in replica: term: 3 index: 12. Preceding OpId from leader: term: 3 index: 13. (index mismatch)
I20260812 08:01:17.788769  8971 consensus_queue.cc:1048] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260812 08:01:17.788921  8972 consensus_queue.cc:1048] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [LEADER]: Connected to new peer: Peer: permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260812 08:01:17.790516  8972 raft_consensus.cc:2955] T 43c7002f0c8d4b6b8ea0faadafa56841 P 2e0a5e5978ec4066a49535da2b5c62df [term 3 LEADER]: Committing config change with OpId 3.13: config changed from index 12 to 13, NON_VOTER 1b9f9cecaa3d4db4840827af64e6a055 (127.0.186.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } } }
I20260812 08:01:17.791097  8903 raft_consensus.cc:2955] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 [term 3 FOLLOWER]: Committing config change with OpId 3.13: config changed from index 12 to 13, NON_VOTER 1b9f9cecaa3d4db4840827af64e6a055 (127.0.186.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } } }
I20260812 08:01:17.791172  8187 raft_consensus.cc:2955] T 43c7002f0c8d4b6b8ea0faadafa56841 P b5ca77770a37434ba93872246814f44a [term 3 FOLLOWER]: Committing config change with OpId 3.13: config changed from index 12 to 13, NON_VOTER 1b9f9cecaa3d4db4840827af64e6a055 (127.0.186.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } } }
I20260812 08:01:17.792065  8005 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 43c7002f0c8d4b6b8ea0faadafa56841 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:01:17.793478  8019 catalog_manager.cc:5719] T 43c7002f0c8d4b6b8ea0faadafa56841 P 26c3805c62084787905d30c43fdf5846 reported cstate change: config changed from index 12 to 13, NON_VOTER 1b9f9cecaa3d4db4840827af64e6a055 (127.0.186.67) evicted. New cstate: current_term: 3 leader_uuid: "2e0a5e5978ec4066a49535da2b5c62df" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b5ca77770a37434ba93872246814f44a" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 42451 } } peers { permanent_uuid: "26c3805c62084787905d30c43fdf5846" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46057 } } peers { permanent_uuid: "2e0a5e5978ec4066a49535da2b5c62df" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 44017 } } }
I20260812 08:01:17.799978  8460 tablet_service.cc:1558] Processing DeleteTablet for tablet 43c7002f0c8d4b6b8ea0faadafa56841 with delete_type TABLET_DATA_TOMBSTONED (TS 1b9f9cecaa3d4db4840827af64e6a055 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:56514
W20260812 08:01:17.801112  8003 catalog_manager.cc:5030] TS 1b9f9cecaa3d4db4840827af64e6a055 (127.0.186.67:38335): delete failed for tablet 43c7002f0c8d4b6b8ea0faadafa56841 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 43c7002f0c8d4b6b8ea0faadafa56841 already in progress: copying tablet
I20260812 08:01:17.807152   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 8090
I20260812 08:01:17.825330   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 8807
I20260812 08:01:17.833590   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 8389
I20260812 08:01:17.842753   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 8538
I20260812 08:01:17.850924   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 7984
2026-08-12T08:01:17Z chronyd exiting
[       OK ] RaftConsensusNonVoterITest.TabletServerIsGoneAndBack (10616 ms)
[ RUN      ] RaftConsensusNonVoterITest.FailedTabletCopy
I20260812 08:01:17.878438   745 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:01:17.878494   745 ts_itest-base.cc:116] --------------
I20260812 08:01:17.878504   745 ts_itest-base.cc:117] 5 tablet servers
I20260812 08:01:17.878515   745 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:01:17.878523   745 ts_itest-base.cc:119] --------------
2026-08-12T08:01:17Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:01:17Z Disabled control of system clock
I20260812 08:01:17.887406   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:45183
--webserver_interface=127.0.186.126
--webserver_port=0
--builtin_ntp_servers=127.0.186.84:44209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:45183
--raft_prepare_replacement_before_eviction=true
--tserver_unresponsive_timeout_ms=2500 with env {}
W20260812 08:01:17.994029  8994 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:17.994266  8994 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:17.994321  8994 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:17.997599  8994 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:01:17.997663  8994 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:17.997686  8994 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:01:17.997711  8994 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:01:18.001663  8994 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:44209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:45183
--tserver_unresponsive_timeout_ms=2500
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:45183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.8994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:18.002760  8994 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:18.003863  8994 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:18.009147  9002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:18.009050  9005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:18.009369  8994 server_base.cc:1061] running on GCE node
W20260812 08:01:18.009052  9000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:18.009968  8994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:18.010829  8994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:18.011972  8994 hybrid_clock.cc:648] HybridClock initialized: now 1786521678011954 us; error 35 us; skew 500 ppm
I20260812 08:01:18.013847  8994 webserver.cc:533] Webserver started at http://127.0.186.126:43337/ using document root <none> and password file <none>
I20260812 08:01:18.014407  8994 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:18.014464  8994 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:18.014652  8994 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:01:18.016156  8994 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "90eaf32f1b3c4b22ae893fdf146194ce"
format_stamp: "Formatted at 2026-08-12 08:01:18 on dist-test-slave-49h3"
I20260812 08:01:18.016618  8994 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "90eaf32f1b3c4b22ae893fdf146194ce"
format_stamp: "Formatted at 2026-08-12 08:01:18 on dist-test-slave-49h3"
I20260812 08:01:18.019783  8994 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:01:18.021811  9011 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:18.022755  8994 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:18.022848  8994 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "90eaf32f1b3c4b22ae893fdf146194ce"
format_stamp: "Formatted at 2026-08-12 08:01:18 on dist-test-slave-49h3"
I20260812 08:01:18.022919  8994 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:18.034466  8994 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:18.035158  8994 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:18.035326  8994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:18.042555  9071 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:45183 every 8 connection(s)
I20260812 08:01:18.042567  8994 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:45183
I20260812 08:01:18.043556  8994 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:01:18.046397  9073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:01:18.051174  9073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90eaf32f1b3c4b22ae893fdf146194ce: Bootstrap starting.
I20260812 08:01:18.052634   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 8994
I20260812 08:01:18.052734   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:01:18.053406  9073 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90eaf32f1b3c4b22ae893fdf146194ce: Neither blocks nor log segments found. Creating new log.
I20260812 08:01:18.054239  9073 log.cc:826] T 00000000000000000000000000000000 P 90eaf32f1b3c4b22ae893fdf146194ce: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:18.056113  9073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90eaf32f1b3c4b22ae893fdf146194ce: No bootstrap required, opened a new log
I20260812 08:01:18.059000  9073 raft_consensus.cc:359] T 00000000000000000000000000000000 P 90eaf32f1b3c4b22ae893fdf146194ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90eaf32f1b3c4b22ae893fdf146194ce" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 45183 } }
I20260812 08:01:18.059219  9073 raft_consensus.cc:385] T 00000000000000000000000000000000 P 90eaf32f1b3c4b22ae893fdf146194ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:01:18.059278  9073 raft_consensus.cc:740] T 00000000000000000000000000000000 P 90eaf32f1b3c4b22ae893fdf146194ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90eaf32f1b3c4b22ae893fdf146194ce, State: Initialized, Role: FOLLOWER
I20260812 08:01:18.059731  9073 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90eaf32f1b3c4b22ae893fdf146194ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90eaf32f1b3c4b22ae893fdf146194ce" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 45183 } }
I20260812 08:01:18.059885  9073 raft_consensus.cc:399] T 00000000000000000000000000000000 P 90eaf32f1b3c4b22ae893fdf146194ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:01:18.059948  9073 raft_consensus.cc:493] T 00000000000000000000000000000000 P 90eaf32f1b3c4b22ae893fdf146194ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:01:18.060060  9073 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 90eaf32f1b3c4b22ae893fdf146194ce [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:01:18.060786  9073 raft_consensus.cc:515] T 00000000000000000000000000000000 P 90eaf32f1b3c4b22ae893fdf146194ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90eaf32f1b3c4b22ae893fdf146194ce" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 45183 } }
I20260812 08:01:18.061096  9073 leader_election.cc:304] T 00000000000000000000000000000000 P 90eaf32f1b3c4b22ae893fdf146194ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90eaf32f1b3c4b22ae893fdf146194ce; no voters: 
I20260812 08:01:18.061363  9073 leader_election.cc:290] T 00000000000000000000000000000000 P 90eaf32f1b3c4b22ae893fdf146194ce [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:01:18.061479  9078 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 90eaf32f1b3c4b22ae893fdf146194ce [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:01:18.061648  9078 raft_consensus.cc:697] T 00000000000000000000000000000000 P 90eaf32f1b3c4b22ae893fdf146194ce [term 1 LEADER]: Becoming Leader. State: Replica: 90eaf32f1b3c4b22ae893fdf146194ce, State: Running, Role: LEADER
I20260812 08:01:18.061965  9078 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90eaf32f1b3c4b22ae893fdf146194ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90eaf32f1b3c4b22ae893fdf146194ce" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 45183 } }
I20260812 08:01:18.062506  9073 sys_catalog.cc:565] T 00000000000000000000000000000000 P 90eaf32f1b3c4b22ae893fdf146194ce [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:01:18.063598  9079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90eaf32f1b3c4b22ae893fdf146194ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90eaf32f1b3c4b22ae893fdf146194ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90eaf32f1b3c4b22ae893fdf146194ce" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 45183 } } }
I20260812 08:01:18.063623  9080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90eaf32f1b3c4b22ae893fdf146194ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90eaf32f1b3c4b22ae893fdf146194ce. Latest consensus state: current_term: 1 leader_uuid: "90eaf32f1b3c4b22ae893fdf146194ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90eaf32f1b3c4b22ae893fdf146194ce" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 45183 } } }
I20260812 08:01:18.063786  9079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90eaf32f1b3c4b22ae893fdf146194ce [sys.catalog]: This master's current role is: LEADER
I20260812 08:01:18.063793  9080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90eaf32f1b3c4b22ae893fdf146194ce [sys.catalog]: This master's current role is: LEADER
I20260812 08:01:18.064672  9088 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:01:18.067088  9088 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:01:18.071951  9088 catalog_manager.cc:1383] Generated new cluster ID: 294a8958ad644fc0ba2e561dc01df889
I20260812 08:01:18.072016  9088 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:01:18.080436  9088 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:01:18.081228  9088 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:01:18.091950  9088 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 90eaf32f1b3c4b22ae893fdf146194ce: Generated new TSK 0
I20260812 08:01:18.092499  9088 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:01:18.099316   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:0
--local_ip_for_outbound_sockets=127.0.186.65
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:45183
--builtin_ntp_servers=127.0.186.84:44209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=10
--consensus_rpc_timeout_ms=5000
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20260812 08:01:18.209234  9101 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:01:18.209532  9101 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:18.209584  9101 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:18.209640  9101 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:18.212930  9101 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:01:18.213050  9101 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:01:18.213155  9101 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:18.213249  9101 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:01:18.213285  9101 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:01:18.217388  9101 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:44209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=10
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:45183
--never_fsync=true
--heap_profile_path=/tmp/kudu.9101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:18.218633  9101 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:18.219823  9101 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:18.226519  9107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:18.226544  9110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:18.226528  9108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:18.226785  9101 server_base.cc:1061] running on GCE node
I20260812 08:01:18.227301  9101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:18.227916  9101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:18.229069  9101 hybrid_clock.cc:648] HybridClock initialized: now 1786521678229061 us; error 39 us; skew 500 ppm
I20260812 08:01:18.230835  9101 webserver.cc:533] Webserver started at http://127.0.186.65:46529/ using document root <none> and password file <none>
I20260812 08:01:18.231386  9101 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:18.231451  9101 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:18.231647  9101 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:01:18.233207  9101 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "c62ae7d1dcce4b2bab792fad5dbb4fa1"
format_stamp: "Formatted at 2026-08-12 08:01:18 on dist-test-slave-49h3"
I20260812 08:01:18.233706  9101 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "c62ae7d1dcce4b2bab792fad5dbb4fa1"
format_stamp: "Formatted at 2026-08-12 08:01:18 on dist-test-slave-49h3"
I20260812 08:01:18.237068  9101 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.001s
I20260812 08:01:18.239283  9116 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:18.240278  9101 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:18.240387  9101 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "c62ae7d1dcce4b2bab792fad5dbb4fa1"
format_stamp: "Formatted at 2026-08-12 08:01:18 on dist-test-slave-49h3"
I20260812 08:01:18.240473  9101 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:18.260831  9101 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:18.261418  9101 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:18.261616  9101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:18.262172  9101 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:18.263206  9101 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:01:18.263257  9101 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:18.263310  9101 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:01:18.263337  9101 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:18.272181  9101 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:42355
I20260812 08:01:18.272233  9244 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:42355 every 8 connection(s)
I20260812 08:01:18.273146  9101 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:01:18.276193   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 9101
I20260812 08:01:18.276316   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:01:18.278546   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.66:0
--local_ip_for_outbound_sockets=127.0.186.66
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:45183
--builtin_ntp_servers=127.0.186.84:44209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=10
--consensus_rpc_timeout_ms=5000
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20260812 08:01:18.281816  9245 heartbeater.cc:344] Connected to a master server at 127.0.186.126:45183
I20260812 08:01:18.282076  9245 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:18.282586  9245 heartbeater.cc:507] Master 127.0.186.126:45183 requested a full tablet report, sending...
I20260812 08:01:18.284091  9025 ts_manager.cc:194] Registered new tserver with Master: c62ae7d1dcce4b2bab792fad5dbb4fa1 (127.0.186.65:42355)
I20260812 08:01:18.285247  9025 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.65:44827
W20260812 08:01:18.383666  9249 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:01:18.383915  9249 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:18.383952  9249 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:18.384014  9249 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:18.387302  9249 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:01:18.387358  9249 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:01:18.387393  9249 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:18.387478  9249 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:01:18.387511  9249 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.66
I20260812 08:01:18.391358  9249 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:44209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=10
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:45183
--never_fsync=true
--heap_profile_path=/tmp/kudu.9249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:18.392373  9249 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:18.393517  9249 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:18.399780  9255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:18.399791  9254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:18.399958  9249 server_base.cc:1061] running on GCE node
W20260812 08:01:18.400089  9257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:18.400441  9249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:18.400942  9249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:18.402088  9249 hybrid_clock.cc:648] HybridClock initialized: now 1786521678402070 us; error 28 us; skew 500 ppm
I20260812 08:01:18.403836  9249 webserver.cc:533] Webserver started at http://127.0.186.66:43929/ using document root <none> and password file <none>
I20260812 08:01:18.404363  9249 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:18.404424  9249 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:18.404600  9249 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:01:18.406121  9249 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "5dd1fae6db3c40b3bcb5230241c55bda"
format_stamp: "Formatted at 2026-08-12 08:01:18 on dist-test-slave-49h3"
I20260812 08:01:18.406563  9249 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "5dd1fae6db3c40b3bcb5230241c55bda"
format_stamp: "Formatted at 2026-08-12 08:01:18 on dist-test-slave-49h3"
I20260812 08:01:18.409744  9249 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:01:18.411787  9265 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:18.412689  9249 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:18.412791  9249 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "5dd1fae6db3c40b3bcb5230241c55bda"
format_stamp: "Formatted at 2026-08-12 08:01:18 on dist-test-slave-49h3"
I20260812 08:01:18.412873  9249 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:18.431311  9249 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:18.431960  9249 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:18.432132  9249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:18.432713  9249 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:18.433789  9249 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:01:18.433840  9249 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:18.433894  9249 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:01:18.433919  9249 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:18.442833  9249 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.66:43543
I20260812 08:01:18.442863  9391 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.66:43543 every 8 connection(s)
I20260812 08:01:18.443822  9249 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:01:18.445281   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 9249
I20260812 08:01:18.445402   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:01:18.447744   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.67:0
--local_ip_for_outbound_sockets=127.0.186.67
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:45183
--builtin_ntp_servers=127.0.186.84:44209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=10
--consensus_rpc_timeout_ms=5000
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20260812 08:01:18.452276  9392 heartbeater.cc:344] Connected to a master server at 127.0.186.126:45183
I20260812 08:01:18.452564  9392 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:18.453241  9392 heartbeater.cc:507] Master 127.0.186.126:45183 requested a full tablet report, sending...
I20260812 08:01:18.454622  9025 ts_manager.cc:194] Registered new tserver with Master: 5dd1fae6db3c40b3bcb5230241c55bda (127.0.186.66:43543)
I20260812 08:01:18.455193  9025 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.66:52715
W20260812 08:01:18.554803  9395 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:01:18.555099  9395 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:18.555143  9395 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:18.555204  9395 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:18.558537  9395 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:01:18.558602  9395 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:01:18.558646  9395 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:18.558724  9395 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:01:18.558759  9395 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.67
I20260812 08:01:18.562811  9395 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:44209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=10
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:45183
--never_fsync=true
--heap_profile_path=/tmp/kudu.9395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:18.564137  9395 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:18.565362  9395 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:18.571612  9405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:18.571631  9402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:18.571611  9403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:18.571862  9395 server_base.cc:1061] running on GCE node
I20260812 08:01:18.572260  9395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:18.572801  9395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:18.573954  9395 hybrid_clock.cc:648] HybridClock initialized: now 1786521678573942 us; error 23 us; skew 500 ppm
I20260812 08:01:18.575683  9395 webserver.cc:533] Webserver started at http://127.0.186.67:39655/ using document root <none> and password file <none>
I20260812 08:01:18.576251  9395 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:18.576310  9395 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:18.576493  9395 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:01:18.578022  9395 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "830db3153c9f430683f0e061910b65cf"
format_stamp: "Formatted at 2026-08-12 08:01:18 on dist-test-slave-49h3"
I20260812 08:01:18.578480  9395 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "830db3153c9f430683f0e061910b65cf"
format_stamp: "Formatted at 2026-08-12 08:01:18 on dist-test-slave-49h3"
I20260812 08:01:18.581682  9395 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.000s
I20260812 08:01:18.583739  9412 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:18.584640  9395 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:18.584749  9395 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "830db3153c9f430683f0e061910b65cf"
format_stamp: "Formatted at 2026-08-12 08:01:18 on dist-test-slave-49h3"
I20260812 08:01:18.584833  9395 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:18.610443  9395 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:18.611050  9395 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:18.611213  9395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:18.611750  9395 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:18.612759  9395 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:01:18.612807  9395 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:18.612862  9395 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:01:18.612885  9395 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:18.621815  9395 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.67:34687
I20260812 08:01:18.621846  9547 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.67:34687 every 8 connection(s)
I20260812 08:01:18.622993  9395 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:01:18.624389   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 9395
I20260812 08:01:18.624509   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:01:18.627105   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.68:0
--local_ip_for_outbound_sockets=127.0.186.68
--webserver_interface=127.0.186.68
--webserver_port=0
--tserver_master_addrs=127.0.186.126:45183
--builtin_ntp_servers=127.0.186.84:44209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=10
--consensus_rpc_timeout_ms=5000
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20260812 08:01:18.631803  9548 heartbeater.cc:344] Connected to a master server at 127.0.186.126:45183
I20260812 08:01:18.632077  9548 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:18.632709  9548 heartbeater.cc:507] Master 127.0.186.126:45183 requested a full tablet report, sending...
I20260812 08:01:18.633909  9025 ts_manager.cc:194] Registered new tserver with Master: 830db3153c9f430683f0e061910b65cf (127.0.186.67:34687)
I20260812 08:01:18.634611  9025 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.67:33469
W20260812 08:01:18.748000  9551 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:01:18.748317  9551 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:18.748339  9551 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:18.748387  9551 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:18.751916  9551 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:01:18.751989  9551 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:01:18.752023  9551 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:18.752086  9551 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:01:18.752113  9551 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.68
I20260812 08:01:18.756306  9551 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:44209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=10
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.0.186.68
--webserver_port=0
--tserver_master_addrs=127.0.186.126:45183
--never_fsync=true
--heap_profile_path=/tmp/kudu.9551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.68

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:18.757539  9551 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:18.758749  9551 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:18.765271  9557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:18.765265  9556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:18.765265  9562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:18.765466  9551 server_base.cc:1061] running on GCE node
I20260812 08:01:18.765887  9551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:18.766414  9551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:18.767562  9551 hybrid_clock.cc:648] HybridClock initialized: now 1786521678767548 us; error 24 us; skew 500 ppm
I20260812 08:01:18.769258  9551 webserver.cc:533] Webserver started at http://127.0.186.68:35617/ using document root <none> and password file <none>
I20260812 08:01:18.769829  9551 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:18.769889  9551 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:18.770088  9551 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:01:18.771612  9551 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "e82880c1240c4279a85a104f294d4b91"
format_stamp: "Formatted at 2026-08-12 08:01:18 on dist-test-slave-49h3"
I20260812 08:01:18.772084  9551 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "e82880c1240c4279a85a104f294d4b91"
format_stamp: "Formatted at 2026-08-12 08:01:18 on dist-test-slave-49h3"
I20260812 08:01:18.775465  9551 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:01:18.777787  9572 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:18.778934  9551 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 08:01:18.779059  9551 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "e82880c1240c4279a85a104f294d4b91"
format_stamp: "Formatted at 2026-08-12 08:01:18 on dist-test-slave-49h3"
I20260812 08:01:18.779160  9551 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:18.790727  9551 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:18.791281  9551 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:18.791436  9551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:18.791942  9551 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:18.792923  9551 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:01:18.792971  9551 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:18.793021  9551 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:01:18.793049  9551 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:18.801532  9551 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.68:37125
I20260812 08:01:18.801553  9703 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.68:37125 every 8 connection(s)
I20260812 08:01:18.802407  9551 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20260812 08:01:18.804234   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 9551
I20260812 08:01:18.804358   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20260812 08:01:18.806663   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.69:0
--local_ip_for_outbound_sockets=127.0.186.69
--webserver_interface=127.0.186.69
--webserver_port=0
--tserver_master_addrs=127.0.186.126:45183
--builtin_ntp_servers=127.0.186.84:44209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=10
--consensus_rpc_timeout_ms=5000
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20260812 08:01:18.810722  9704 heartbeater.cc:344] Connected to a master server at 127.0.186.126:45183
I20260812 08:01:18.810983  9704 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:18.811514  9704 heartbeater.cc:507] Master 127.0.186.126:45183 requested a full tablet report, sending...
I20260812 08:01:18.812587  9025 ts_manager.cc:194] Registered new tserver with Master: e82880c1240c4279a85a104f294d4b91 (127.0.186.68:37125)
I20260812 08:01:18.813134  9025 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.68:60973
W20260812 08:01:18.917501  9707 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:01:18.917798  9707 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:18.917833  9707 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:18.917887  9707 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:18.921250  9707 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:01:18.921320  9707 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:01:18.921363  9707 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:18.921480  9707 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:01:18.921523  9707 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.69
I20260812 08:01:18.925767  9707 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:44209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=10
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.69:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.0.186.69
--webserver_port=0
--tserver_master_addrs=127.0.186.126:45183
--never_fsync=true
--heap_profile_path=/tmp/kudu.9707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.69

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:18.926996  9707 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:18.928234  9707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:18.934897  9714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:18.934944  9716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:18.934988  9713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:18.935197  9707 server_base.cc:1061] running on GCE node
I20260812 08:01:18.935657  9707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:18.936192  9707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:18.937350  9707 hybrid_clock.cc:648] HybridClock initialized: now 1786521678937332 us; error 33 us; skew 500 ppm
I20260812 08:01:18.939339  9707 webserver.cc:533] Webserver started at http://127.0.186.69:33765/ using document root <none> and password file <none>
I20260812 08:01:18.939942  9707 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:18.940006  9707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:18.940233  9707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:01:18.941944  9707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c"
format_stamp: "Formatted at 2026-08-12 08:01:18 on dist-test-slave-49h3"
I20260812 08:01:18.942442  9707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c"
format_stamp: "Formatted at 2026-08-12 08:01:18 on dist-test-slave-49h3"
I20260812 08:01:18.946039  9707 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:01:18.948324  9723 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:18.949321  9707 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:01:18.949429  9707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c"
format_stamp: "Formatted at 2026-08-12 08:01:18 on dist-test-slave-49h3"
I20260812 08:01:18.949553  9707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:18.972317  9707 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:18.973148  9707 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:18.973356  9707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:18.974037  9707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:18.975277  9707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:01:18.975332  9707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:18.975389  9707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:01:18.975414  9707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:18.984570  9707 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.69:42173
I20260812 08:01:18.984606  9858 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.69:42173 every 8 connection(s)
I20260812 08:01:18.985618  9707 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20260812 08:01:18.993494  9859 heartbeater.cc:344] Connected to a master server at 127.0.186.126:45183
I20260812 08:01:18.993731  9859 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:18.994169   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 9707
I20260812 08:01:18.994308   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20260812 08:01:18.994344  9859 heartbeater.cc:507] Master 127.0.186.126:45183 requested a full tablet report, sending...
I20260812 08:01:18.995515  9028 ts_manager.cc:194] Registered new tserver with Master: e62f5ac41b1e4fdd8fc4f24179cffa4c (127.0.186.69:42173)
I20260812 08:01:18.996239  9028 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.69:41639
I20260812 08:01:18.996840   745 external_mini_cluster.cc:955] 5 TS(s) registered with all masters
I20260812 08:01:19.005391  9025 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45050:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
I20260812 08:01:19.024271  9627 tablet_service.cc:1511] Processing CreateTablet for tablet 6a6928a22d924847b068b4979c93b78f (DEFAULT_TABLE table=TestTable [id=8a2c50878e1446b4a61f7b39d4a84565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:01:19.024725  9317 tablet_service.cc:1511] Processing CreateTablet for tablet 6a6928a22d924847b068b4979c93b78f (DEFAULT_TABLE table=TestTable [id=8a2c50878e1446b4a61f7b39d4a84565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:01:19.025506  9627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a6928a22d924847b068b4979c93b78f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:01:19.025676  9317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a6928a22d924847b068b4979c93b78f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:01:19.029393  9779 tablet_service.cc:1511] Processing CreateTablet for tablet 6a6928a22d924847b068b4979c93b78f (DEFAULT_TABLE table=TestTable [id=8a2c50878e1446b4a61f7b39d4a84565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:01:19.030601  9779 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a6928a22d924847b068b4979c93b78f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:01:19.033206  9880 tablet_bootstrap.cc:492] T 6a6928a22d924847b068b4979c93b78f P e82880c1240c4279a85a104f294d4b91: Bootstrap starting.
I20260812 08:01:19.033999  9881 tablet_bootstrap.cc:492] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda: Bootstrap starting.
I20260812 08:01:19.035447  9880 tablet_bootstrap.cc:654] T 6a6928a22d924847b068b4979c93b78f P e82880c1240c4279a85a104f294d4b91: Neither blocks nor log segments found. Creating new log.
I20260812 08:01:19.036180  9881 tablet_bootstrap.cc:654] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda: Neither blocks nor log segments found. Creating new log.
I20260812 08:01:19.036170  9882 tablet_bootstrap.cc:492] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c: Bootstrap starting.
I20260812 08:01:19.036264  9880 log.cc:826] T 6a6928a22d924847b068b4979c93b78f P e82880c1240c4279a85a104f294d4b91: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:19.036928  9881 log.cc:826] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:19.038125  9880 tablet_bootstrap.cc:492] T 6a6928a22d924847b068b4979c93b78f P e82880c1240c4279a85a104f294d4b91: No bootstrap required, opened a new log
I20260812 08:01:19.038317  9880 ts_tablet_manager.cc:1403] T 6a6928a22d924847b068b4979c93b78f P e82880c1240c4279a85a104f294d4b91: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:01:19.038404  9882 tablet_bootstrap.cc:654] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c: Neither blocks nor log segments found. Creating new log.
I20260812 08:01:19.038535  9881 tablet_bootstrap.cc:492] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda: No bootstrap required, opened a new log
I20260812 08:01:19.038707  9881 ts_tablet_manager.cc:1403] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:01:19.039105  9882 log.cc:826] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:19.040517  9882 tablet_bootstrap.cc:492] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c: No bootstrap required, opened a new log
I20260812 08:01:19.040697  9882 ts_tablet_manager.cc:1403] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.000s
I20260812 08:01:19.041425  9880 raft_consensus.cc:359] T 6a6928a22d924847b068b4979c93b78f P e82880c1240c4279a85a104f294d4b91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } } peers { permanent_uuid: "5dd1fae6db3c40b3bcb5230241c55bda" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43543 } }
I20260812 08:01:19.041534  9881 raft_consensus.cc:359] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } } peers { permanent_uuid: "5dd1fae6db3c40b3bcb5230241c55bda" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43543 } }
I20260812 08:01:19.041684  9881 raft_consensus.cc:385] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:01:19.041703  9880 raft_consensus.cc:385] T 6a6928a22d924847b068b4979c93b78f P e82880c1240c4279a85a104f294d4b91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:01:19.041741  9881 raft_consensus.cc:740] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dd1fae6db3c40b3bcb5230241c55bda, State: Initialized, Role: FOLLOWER
I20260812 08:01:19.041764  9880 raft_consensus.cc:740] T 6a6928a22d924847b068b4979c93b78f P e82880c1240c4279a85a104f294d4b91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82880c1240c4279a85a104f294d4b91, State: Initialized, Role: FOLLOWER
I20260812 08:01:19.042207  9881 consensus_queue.cc:260] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } } peers { permanent_uuid: "5dd1fae6db3c40b3bcb5230241c55bda" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43543 } }
I20260812 08:01:19.042207  9880 consensus_queue.cc:260] T 6a6928a22d924847b068b4979c93b78f P e82880c1240c4279a85a104f294d4b91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } } peers { permanent_uuid: "5dd1fae6db3c40b3bcb5230241c55bda" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43543 } }
I20260812 08:01:19.043120  9881 ts_tablet_manager.cc:1434] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:01:19.043120  9880 ts_tablet_manager.cc:1434] T 6a6928a22d924847b068b4979c93b78f P e82880c1240c4279a85a104f294d4b91: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.003s
I20260812 08:01:19.043533  9704 heartbeater.cc:499] Master 127.0.186.126:45183 was elected leader, sending a full tablet report...
I20260812 08:01:19.043535  9392 heartbeater.cc:499] Master 127.0.186.126:45183 was elected leader, sending a full tablet report...
I20260812 08:01:19.044927  9882 raft_consensus.cc:359] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } } peers { permanent_uuid: "5dd1fae6db3c40b3bcb5230241c55bda" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43543 } }
I20260812 08:01:19.045163  9882 raft_consensus.cc:385] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:01:19.045209  9882 raft_consensus.cc:740] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e62f5ac41b1e4fdd8fc4f24179cffa4c, State: Initialized, Role: FOLLOWER
W20260812 08:01:19.045746  9393 tablet.cc:2406] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:01:19.045743  9882 consensus_queue.cc:260] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } } peers { permanent_uuid: "5dd1fae6db3c40b3bcb5230241c55bda" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43543 } }
I20260812 08:01:19.046839  9882 ts_tablet_manager.cc:1434] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260812 08:01:19.047217  9859 heartbeater.cc:499] Master 127.0.186.126:45183 was elected leader, sending a full tablet report...
W20260812 08:01:19.086504  9860 tablet.cc:2406] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:01:19.103693  9705 tablet.cc:2406] T 6a6928a22d924847b068b4979c93b78f P e82880c1240c4279a85a104f294d4b91: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:01:19.287560  9245 heartbeater.cc:499] Master 127.0.186.126:45183 was elected leader, sending a full tablet report...
I20260812 08:01:19.329321  9888 raft_consensus.cc:493] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:01:19.329581  9888 raft_consensus.cc:515] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } } peers { permanent_uuid: "5dd1fae6db3c40b3bcb5230241c55bda" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43543 } }
I20260812 08:01:19.330708  9888 leader_election.cc:290] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e82880c1240c4279a85a104f294d4b91 (127.0.186.68:37125), 5dd1fae6db3c40b3bcb5230241c55bda (127.0.186.66:43543)
I20260812 08:01:19.334558  9650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6928a22d924847b068b4979c93b78f" candidate_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82880c1240c4279a85a104f294d4b91" is_pre_election: true
I20260812 08:01:19.334812  9338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6928a22d924847b068b4979c93b78f" candidate_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd1fae6db3c40b3bcb5230241c55bda" is_pre_election: true
I20260812 08:01:19.334882  9650 raft_consensus.cc:2468] T 6a6928a22d924847b068b4979c93b78f P e82880c1240c4279a85a104f294d4b91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e62f5ac41b1e4fdd8fc4f24179cffa4c in term 0.
I20260812 08:01:19.335053  9338 raft_consensus.cc:2468] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e62f5ac41b1e4fdd8fc4f24179cffa4c in term 0.
I20260812 08:01:19.335371  9728 leader_election.cc:304] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e62f5ac41b1e4fdd8fc4f24179cffa4c, e82880c1240c4279a85a104f294d4b91; no voters: 
I20260812 08:01:19.335593  9888 raft_consensus.cc:2804] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:01:19.335664  9888 raft_consensus.cc:493] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:01:19.335693  9888 raft_consensus.cc:3060] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:01:19.336527  9888 raft_consensus.cc:515] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } } peers { permanent_uuid: "5dd1fae6db3c40b3bcb5230241c55bda" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43543 } }
I20260812 08:01:19.336869  9888 leader_election.cc:290] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [CANDIDATE]: Term 1 election: Requested vote from peers e82880c1240c4279a85a104f294d4b91 (127.0.186.68:37125), 5dd1fae6db3c40b3bcb5230241c55bda (127.0.186.66:43543)
I20260812 08:01:19.337248  9338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6928a22d924847b068b4979c93b78f" candidate_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd1fae6db3c40b3bcb5230241c55bda"
I20260812 08:01:19.337275  9650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6928a22d924847b068b4979c93b78f" candidate_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82880c1240c4279a85a104f294d4b91"
I20260812 08:01:19.337355  9338 raft_consensus.cc:3060] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:01:19.337380  9650 raft_consensus.cc:3060] T 6a6928a22d924847b068b4979c93b78f P e82880c1240c4279a85a104f294d4b91 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:01:19.338316  9338 raft_consensus.cc:2468] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e62f5ac41b1e4fdd8fc4f24179cffa4c in term 1.
I20260812 08:01:19.338599  9728 leader_election.cc:304] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dd1fae6db3c40b3bcb5230241c55bda, e62f5ac41b1e4fdd8fc4f24179cffa4c; no voters: 
I20260812 08:01:19.338620  9650 raft_consensus.cc:2468] T 6a6928a22d924847b068b4979c93b78f P e82880c1240c4279a85a104f294d4b91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e62f5ac41b1e4fdd8fc4f24179cffa4c in term 1.
I20260812 08:01:19.338821  9888 raft_consensus.cc:2804] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:01:19.339008  9888 raft_consensus.cc:697] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [term 1 LEADER]: Becoming Leader. State: Replica: e62f5ac41b1e4fdd8fc4f24179cffa4c, State: Running, Role: LEADER
I20260812 08:01:19.339318  9888 consensus_queue.cc:237] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } } peers { permanent_uuid: "5dd1fae6db3c40b3bcb5230241c55bda" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43543 } }
I20260812 08:01:19.342362  9027 catalog_manager.cc:5719] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c reported cstate change: term changed from 0 to 1, leader changed from <none> to e62f5ac41b1e4fdd8fc4f24179cffa4c (127.0.186.69). New cstate: current_term: 1 leader_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dd1fae6db3c40b3bcb5230241c55bda" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43543 } health_report { overall_health: UNKNOWN } } }
I20260812 08:01:19.384183   745 external_mini_cluster.cc:955] 5 TS(s) registered with all masters
I20260812 08:01:19.385704   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 5dd1fae6db3c40b3bcb5230241c55bda to finish bootstrapping
I20260812 08:01:19.391832   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver e82880c1240c4279a85a104f294d4b91 to finish bootstrapping
I20260812 08:01:19.396263   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver e62f5ac41b1e4fdd8fc4f24179cffa4c to finish bootstrapping
I20260812 08:01:19.405684  9541 generic_service.cc:175] {username='slave'} at 127.0.0.1:54054 changed flags via RPC: tablet_copy_fault_crash_on_fetch_all from '0' to '1.0'
I20260812 08:01:19.406477   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 9101
I20260812 08:01:19.636898  9548 heartbeater.cc:499] Master 127.0.186.126:45183 was elected leader, sending a full tablet report...
I20260812 08:01:19.772324  9888 consensus_queue.cc:1048] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd1fae6db3c40b3bcb5230241c55bda" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:01:19.778290  9890 consensus_queue.cc:1048] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:01:24.454218   745 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
I20260812 08:01:24.454514   745 raft_consensus-itest-base.cc:268] Waiting until we've written at least 4MB...
I20260812 08:01:24.505683  9860 maintenance_manager.cc:419] P e62f5ac41b1e4fdd8fc4f24179cffa4c: Scheduling FlushMRSOp(6a6928a22d924847b068b4979c93b78f): 1840834 bytes log retention, and flush 2998537 bytes memory
I20260812 08:01:24.523931  9705 maintenance_manager.cc:419] P e82880c1240c4279a85a104f294d4b91: Scheduling FlushMRSOp(6a6928a22d924847b068b4979c93b78f): 2498334 bytes log retention, and flush 4038921 bytes memory
I20260812 08:01:24.566700   745 raft_consensus-itest-base.cc:274] Waiting for log GC on e62f5ac41b1e4fdd8fc4f24179cffa4c
I20260812 08:01:24.579823  9729 maintenance_manager.cc:643] P e62f5ac41b1e4fdd8fc4f24179cffa4c: FlushMRSOp(6a6928a22d924847b068b4979c93b78f) complete. Timing: real 0.074s	user 0.039s	sys 0.000s Metrics: {"bytes_written":135869,"cfile_init":1,"compiler_manager_pool.queue_time_us":247,"drs_written":1,"lbm_read_time_us":73,"lbm_reads_lt_1ms":4,"lbm_write_time_us":627,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":19,"thread_start_us":250,"threads_started":2}
I20260812 08:01:24.580925  9860 maintenance_manager.cc:419] P e62f5ac41b1e4fdd8fc4f24179cffa4c: Scheduling LogGCOp(6a6928a22d924847b068b4979c93b78f): free 1840834 bytes of WAL
I20260812 08:01:24.581223  9729 log_reader.cc:385] T 6a6928a22d924847b068b4979c93b78f: removed 2 log segments from log reader
I20260812 08:01:24.581326  9729 log.cc:1079] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/wal/wals/6a6928a22d924847b068b4979c93b78f/wal-000000001 (ops 1-8)
I20260812 08:01:24.581431  9729 log.cc:1079] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/wal/wals/6a6928a22d924847b068b4979c93b78f/wal-000000002 (ops 9-15)
I20260812 08:01:24.582149  9729 maintenance_manager.cc:643] P e62f5ac41b1e4fdd8fc4f24179cffa4c: LogGCOp(6a6928a22d924847b068b4979c93b78f) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 08:01:24.582463  9860 maintenance_manager.cc:419] P e62f5ac41b1e4fdd8fc4f24179cffa4c: Scheduling FlushMRSOp(6a6928a22d924847b068b4979c93b78f): 2761368 bytes log retention, and flush 4038921 bytes memory
I20260812 08:01:24.587369   745 raft_consensus-itest-base.cc:279] Log GC complete on e62f5ac41b1e4fdd8fc4f24179cffa4c
I20260812 08:01:24.595513  9579 maintenance_manager.cc:643] P e82880c1240c4279a85a104f294d4b91: FlushMRSOp(6a6928a22d924847b068b4979c93b78f) complete. Timing: real 0.071s	user 0.044s	sys 0.000s Metrics: {"bytes_written":135937,"cfile_init":1,"compiler_manager_pool.queue_time_us":2829,"drs_written":1,"lbm_read_time_us":58,"lbm_reads_lt_1ms":4,"lbm_write_time_us":596,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":25,"thread_start_us":255,"threads_started":2}
I20260812 08:01:24.596539  9705 maintenance_manager.cc:419] P e82880c1240c4279a85a104f294d4b91: Scheduling LogGCOp(6a6928a22d924847b068b4979c93b78f): free 2498334 bytes of WAL
I20260812 08:01:24.596812  9579 log_reader.cc:385] T 6a6928a22d924847b068b4979c93b78f: removed 3 log segments from log reader
I20260812 08:01:24.596901  9579 log.cc:1079] T 6a6928a22d924847b068b4979c93b78f P e82880c1240c4279a85a104f294d4b91: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal/wals/6a6928a22d924847b068b4979c93b78f/wal-000000001 (ops 1-8)
I20260812 08:01:24.596992  9579 log.cc:1079] T 6a6928a22d924847b068b4979c93b78f P e82880c1240c4279a85a104f294d4b91: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal/wals/6a6928a22d924847b068b4979c93b78f/wal-000000002 (ops 9-14)
I20260812 08:01:24.597057  9579 log.cc:1079] T 6a6928a22d924847b068b4979c93b78f P e82880c1240c4279a85a104f294d4b91: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal/wals/6a6928a22d924847b068b4979c93b78f/wal-000000003 (ops 15-20)
I20260812 08:01:24.597990  9579 maintenance_manager.cc:643] P e82880c1240c4279a85a104f294d4b91: LogGCOp(6a6928a22d924847b068b4979c93b78f) complete. Timing: real 0.001s	user 0.000s	sys 0.001s Metrics: {}
I20260812 08:01:24.598340  9705 maintenance_manager.cc:419] P e82880c1240c4279a85a104f294d4b91: Scheduling FlushMRSOp(6a6928a22d924847b068b4979c93b78f): 1840794 bytes log retention, and flush 2998537 bytes memory
I20260812 08:01:24.610512  9729 maintenance_manager.cc:643] P e62f5ac41b1e4fdd8fc4f24179cffa4c: FlushMRSOp(6a6928a22d924847b068b4979c93b78f) complete. Timing: real 0.028s	user 0.023s	sys 0.003s Metrics: {"bytes_written":135869,"cfile_init":1,"dirs.queue_time_us":41,"dirs.run_cpu_time_us":124,"dirs.run_wall_time_us":887,"drs_written":1,"lbm_read_time_us":57,"lbm_reads_lt_1ms":4,"lbm_write_time_us":557,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":20}
I20260812 08:01:24.611584  9860 maintenance_manager.cc:419] P e62f5ac41b1e4fdd8fc4f24179cffa4c: Scheduling LogGCOp(6a6928a22d924847b068b4979c93b78f): free 920570 bytes of WAL
I20260812 08:01:24.611827  9729 log_reader.cc:385] T 6a6928a22d924847b068b4979c93b78f: removed 1 log segments from log reader
I20260812 08:01:24.611902  9729 log.cc:1079] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/wal/wals/6a6928a22d924847b068b4979c93b78f/wal-000000003 (ops 16-22)
I20260812 08:01:24.612308  9729 maintenance_manager.cc:643] P e62f5ac41b1e4fdd8fc4f24179cffa4c: LogGCOp(6a6928a22d924847b068b4979c93b78f) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 08:01:24.612562  9860 maintenance_manager.cc:419] P e62f5ac41b1e4fdd8fc4f24179cffa4c: Scheduling UndoDeltaBlockGCOp(6a6928a22d924847b068b4979c93b78f): 702 bytes on disk
I20260812 08:01:24.613238  9729 maintenance_manager.cc:643] P e62f5ac41b1e4fdd8fc4f24179cffa4c: UndoDeltaBlockGCOp(6a6928a22d924847b068b4979c93b78f) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":95,"lbm_reads_lt_1ms":8}
I20260812 08:01:24.613720  9860 maintenance_manager.cc:419] P e62f5ac41b1e4fdd8fc4f24179cffa4c: Scheduling CompactRowSetsOp(6a6928a22d924847b068b4979c93b78f): perf score=0.992235
I20260812 08:01:24.619648  9579 maintenance_manager.cc:643] P e82880c1240c4279a85a104f294d4b91: FlushMRSOp(6a6928a22d924847b068b4979c93b78f) complete. Timing: real 0.021s	user 0.016s	sys 0.003s Metrics: {"bytes_written":135803,"cfile_init":1,"dirs.queue_time_us":42,"dirs.run_cpu_time_us":141,"dirs.run_wall_time_us":913,"drs_written":1,"lbm_read_time_us":50,"lbm_reads_lt_1ms":4,"lbm_write_time_us":613,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":14}
I20260812 08:01:24.620551  9705 maintenance_manager.cc:419] P e82880c1240c4279a85a104f294d4b91: Scheduling UndoDeltaBlockGCOp(6a6928a22d924847b068b4979c93b78f): 712 bytes on disk
I20260812 08:01:24.621248  9579 maintenance_manager.cc:643] P e82880c1240c4279a85a104f294d4b91: UndoDeltaBlockGCOp(6a6928a22d924847b068b4979c93b78f) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":107,"lbm_reads_lt_1ms":8}
I20260812 08:01:24.621712  9705 maintenance_manager.cc:419] P e82880c1240c4279a85a104f294d4b91: Scheduling CompactRowSetsOp(6a6928a22d924847b068b4979c93b78f): perf score=0.994203
I20260812 08:01:24.670635  9729 maintenance_manager.cc:643] P e62f5ac41b1e4fdd8fc4f24179cffa4c: CompactRowSetsOp(6a6928a22d924847b068b4979c93b78f) complete. Timing: real 0.057s	user 0.054s	sys 0.001s Metrics: {"bytes_written":136001,"cfile_cache_hit":10,"cfile_cache_hit_bytes":262392,"cfile_cache_miss":26,"cfile_cache_miss_bytes":264472,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":128,"dirs.run_cpu_time_us":165,"dirs.run_wall_time_us":1208,"drs_written":1,"lbm_read_time_us":750,"lbm_reads_lt_1ms":46,"lbm_write_time_us":601,"lbm_writes_lt_1ms":23,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":39,"thread_start_us":386,"threads_started":5}
I20260812 08:01:24.671496  9860 maintenance_manager.cc:419] P e62f5ac41b1e4fdd8fc4f24179cffa4c: Scheduling UndoDeltaBlockGCOp(6a6928a22d924847b068b4979c93b78f): 569 bytes on disk
I20260812 08:01:24.672067  9729 maintenance_manager.cc:643] P e62f5ac41b1e4fdd8fc4f24179cffa4c: UndoDeltaBlockGCOp(6a6928a22d924847b068b4979c93b78f) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":49,"lbm_reads_lt_1ms":4}
I20260812 08:01:24.677048  9579 maintenance_manager.cc:643] P e82880c1240c4279a85a104f294d4b91: CompactRowSetsOp(6a6928a22d924847b068b4979c93b78f) complete. Timing: real 0.055s	user 0.052s	sys 0.000s Metrics: {"bytes_written":136001,"cfile_cache_hit":10,"cfile_cache_hit_bytes":262394,"cfile_cache_miss":26,"cfile_cache_miss_bytes":264496,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":861,"dirs.run_cpu_time_us":133,"dirs.run_wall_time_us":782,"drs_written":1,"lbm_read_time_us":659,"lbm_reads_lt_1ms":46,"lbm_write_time_us":598,"lbm_writes_lt_1ms":23,"mutex_wait_us":30,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":39,"thread_start_us":342,"threads_started":5}
I20260812 08:01:24.677632  9705 maintenance_manager.cc:419] P e82880c1240c4279a85a104f294d4b91: Scheduling UndoDeltaBlockGCOp(6a6928a22d924847b068b4979c93b78f): 569 bytes on disk
I20260812 08:01:24.677997  9579 maintenance_manager.cc:643] P e82880c1240c4279a85a104f294d4b91: UndoDeltaBlockGCOp(6a6928a22d924847b068b4979c93b78f) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":56,"lbm_reads_lt_1ms":4}
I20260812 08:01:26.588655   745 raft_consensus-itest-base.cc:295] Servers converged with original term 1
I20260812 08:01:26.588723   745 raft_consensus-itest-base.cc:300] Resuming 5dd1fae6db3c40b3bcb5230241c55bda
I20260812 08:01:26.589835  9957 raft_consensus.cc:493] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e62f5ac41b1e4fdd8fc4f24179cffa4c)
I20260812 08:01:26.590009  9957 raft_consensus.cc:515] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } } peers { permanent_uuid: "5dd1fae6db3c40b3bcb5230241c55bda" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 43543 } }
I20260812 08:01:26.591287  9957 leader_election.cc:290] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e62f5ac41b1e4fdd8fc4f24179cffa4c (127.0.186.69:42173), e82880c1240c4279a85a104f294d4b91 (127.0.186.68:37125)
I20260812 08:01:26.592200  9385 generic_service.cc:175] {username='slave'} at 127.0.0.1:40338 changed flags via RPC: tablet_copy_fault_crash_on_fetch_all from '0' to '1.0'
I20260812 08:01:26.594655  9956 consensus_queue.cc:753] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [LEADER]: The logs necessary to catch up peer 5dd1fae6db3c40b3bcb5230241c55bda have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 3..34: Segment 1 which contained index 3 has been GCed)
I20260812 08:01:26.594743  9956 consensus_queue.cc:579] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [LEADER]: The logs necessary to catch up peer 5dd1fae6db3c40b3bcb5230241c55bda have been garbage collected. The replica will never be able to catch up
I20260812 08:01:26.597718  9805 consensus_queue.cc:237] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 1.40, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } }
I20260812 08:01:26.599010  9649 raft_consensus.cc:1275] T 6a6928a22d924847b068b4979c93b78f P e82880c1240c4279a85a104f294d4b91 [term 1 FOLLOWER]: Refusing update from remote peer e62f5ac41b1e4fdd8fc4f24179cffa4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 40. Preceding OpId from leader: term: 1 index: 41. (index mismatch)
I20260812 08:01:26.599429  9805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6928a22d924847b068b4979c93b78f" candidate_uuid: "5dd1fae6db3c40b3bcb5230241c55bda" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" is_pre_election: true
I20260812 08:01:26.599507  9965 consensus_queue.cc:1048] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 41, Last known committed idx: 40, Time since last communication: 0.000s
I20260812 08:01:26.599890  9649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6928a22d924847b068b4979c93b78f" candidate_uuid: "5dd1fae6db3c40b3bcb5230241c55bda" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e82880c1240c4279a85a104f294d4b91" is_pre_election: true
I20260812 08:01:26.600304  9269 leader_election.cc:304] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5dd1fae6db3c40b3bcb5230241c55bda; no voters: e62f5ac41b1e4fdd8fc4f24179cffa4c, e82880c1240c4279a85a104f294d4b91
I20260812 08:01:26.600522  9957 raft_consensus.cc:2749] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 08:01:26.601428  9956 raft_consensus.cc:2955] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [term 1 LEADER]: Committing config change with OpId 1.41: config changed from index -1 to 41, VOTER 5dd1fae6db3c40b3bcb5230241c55bda (127.0.186.66) evicted. New config: { opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } } }
I20260812 08:01:26.601897  9650 raft_consensus.cc:2955] T 6a6928a22d924847b068b4979c93b78f P e82880c1240c4279a85a104f294d4b91 [term 1 FOLLOWER]: Committing config change with OpId 1.41: config changed from index -1 to 41, VOTER 5dd1fae6db3c40b3bcb5230241c55bda (127.0.186.66) evicted. New config: { opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } } }
I20260812 08:01:26.602998  9012 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 6a6928a22d924847b068b4979c93b78f with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:01:26.603842  9028 catalog_manager.cc:5719] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c reported cstate change: config changed from index -1 to 41, VOTER 5dd1fae6db3c40b3bcb5230241c55bda (127.0.186.66) evicted. New cstate: current_term: 1 leader_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" committed_config { opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } health_report { overall_health: HEALTHY } } }
I20260812 08:01:26.606715  9317 tablet_service.cc:1558] Processing DeleteTablet for tablet 6a6928a22d924847b068b4979c93b78f with delete_type TABLET_DATA_TOMBSTONED (TS 5dd1fae6db3c40b3bcb5230241c55bda not found in new config with opid_index 41) from {username='slave'} at 127.0.0.1:40334
I20260812 08:01:26.607237  9974 tablet_replica.cc:333] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda: stopping tablet replica
I20260812 08:01:26.607520  9974 raft_consensus.cc:2243] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 08:01:26.607546  9805 consensus_queue.cc:237] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 41, Committed index: 41, Last appended: 1.41, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } } peers { permanent_uuid: "830db3153c9f430683f0e061910b65cf" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 34687 } attrs { promote: true } }
I20260812 08:01:26.607702  9974 pending_rounds.cc:70] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda: Trying to abort 1 pending ops.
I20260812 08:01:26.609081  9650 raft_consensus.cc:1275] T 6a6928a22d924847b068b4979c93b78f P e82880c1240c4279a85a104f294d4b91 [term 1 FOLLOWER]: Refusing update from remote peer e62f5ac41b1e4fdd8fc4f24179cffa4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 41. Preceding OpId from leader: term: 1 index: 42. (index mismatch)
I20260812 08:01:26.609432  9956 consensus_queue.cc:1048] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 42, Last known committed idx: 41, Time since last communication: 0.000s
I20260812 08:01:26.607760  9974 pending_rounds.cc:77] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7317592819571499008 op_type: WRITE_OP write_request { tablet_id: "6a6928a22d924847b068b4979c93b78f" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\0000\237\300w]\376\230y\000\000\000\000\000\000\000\000\000\000\002\000\000\000\000\000" indirect_data: "<redacted>""000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
I20260812 08:01:26.610968  9965 raft_consensus.cc:2955] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [term 1 LEADER]: Committing config change with OpId 1.42: config changed from index 41 to 42, NON_VOTER 830db3153c9f430683f0e061910b65cf (127.0.186.67) added. New config: { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } } peers { permanent_uuid: "830db3153c9f430683f0e061910b65cf" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 34687 } attrs { promote: true } } }
I20260812 08:01:26.611388  9974 raft_consensus.cc:2272] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 08:01:26.612468  9012 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 6a6928a22d924847b068b4979c93b78f with cas_config_opid_index 41: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:01:26.612520  9974 ts_tablet_manager.cc:1916] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 08:01:26.612576  9650 raft_consensus.cc:2955] T 6a6928a22d924847b068b4979c93b78f P e82880c1240c4279a85a104f294d4b91 [term 1 FOLLOWER]: Committing config change with OpId 1.42: config changed from index 41 to 42, NON_VOTER 830db3153c9f430683f0e061910b65cf (127.0.186.67) added. New config: { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } } peers { permanent_uuid: "830db3153c9f430683f0e061910b65cf" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 34687 } attrs { promote: true } } }
I20260812 08:01:26.613080  9028 catalog_manager.cc:5719] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c reported cstate change: config changed from index 41 to 42, NON_VOTER 830db3153c9f430683f0e061910b65cf (127.0.186.67) added. New cstate: current_term: 1 leader_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" committed_config { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "830db3153c9f430683f0e061910b65cf" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 34687 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260812 08:01:26.614795  9974 ts_tablet_manager.cc:1929] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20260812 08:01:26.615245  9974 log.cc:1199] T 6a6928a22d924847b068b4979c93b78f P 5dd1fae6db3c40b3bcb5230241c55bda: Deleting WAL directory at /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/wals/6a6928a22d924847b068b4979c93b78f
W20260812 08:01:26.615734  9726 consensus_peers.cc:597] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c -> Peer 830db3153c9f430683f0e061910b65cf (127.0.186.67:34687): Couldn't send request to peer 830db3153c9f430683f0e061910b65cf. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6a6928a22d924847b068b4979c93b78f. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:01:26.616267  9015 catalog_manager.cc:5050] TS 5dd1fae6db3c40b3bcb5230241c55bda (127.0.186.66:43543): tablet 6a6928a22d924847b068b4979c93b78f (table TestTable [id=8a2c50878e1446b4a61f7b39d4a84565]) successfully deleted
I20260812 08:01:26.620337   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:42355
--local_ip_for_outbound_sockets=127.0.186.65
--tserver_master_addrs=127.0.186.126:45183
--webserver_port=46529
--webserver_interface=127.0.186.65
--builtin_ntp_servers=127.0.186.84:44209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=10
--consensus_rpc_timeout_ms=5000
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20260812 08:01:26.726833  9977 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:01:26.727099  9977 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:26.727141  9977 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:26.727205  9977 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:26.730453  9977 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:01:26.730508  9977 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:01:26.730559  9977 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:26.730643  9977 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:01:26.730676  9977 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:01:26.734567  9977 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:44209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=10
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:42355
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=46529
--tserver_master_addrs=127.0.186.126:45183
--never_fsync=true
--heap_profile_path=/tmp/kudu.9977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:26.735591  9977 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:26.736716  9977 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:26.743353  9985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:26.743491  9977 server_base.cc:1061] running on GCE node
W20260812 08:01:26.743503  9982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:26.743379  9983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:26.744065  9977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:26.744639  9977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:26.745792  9977 hybrid_clock.cc:648] HybridClock initialized: now 1786521686745773 us; error 29 us; skew 500 ppm
I20260812 08:01:26.747704  9977 webserver.cc:533] Webserver started at http://127.0.186.65:46529/ using document root <none> and password file <none>
I20260812 08:01:26.748278  9977 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:26.748358  9977 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:26.751827  9977 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.000s	sys 0.001s
I20260812 08:01:26.753741  9991 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:26.754776  9977 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260812 08:01:26.754880  9977 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "c62ae7d1dcce4b2bab792fad5dbb4fa1"
format_stamp: "Formatted at 2026-08-12 08:01:18 on dist-test-slave-49h3"
I20260812 08:01:26.755273  9977 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:26.771663  9977 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:26.772280  9977 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:26.772444  9977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:26.772994  9977 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:26.774066  9977 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:01:26.774119  9977 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:26.774173  9977 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:01:26.774202  9977 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:26.783326  9977 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:42355
I20260812 08:01:26.783356 10124 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:42355 every 8 connection(s)
I20260812 08:01:26.784364  9977 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.FailedTabletCopy.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:01:26.786679   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 9977
I20260812 08:01:26.793109 10125 heartbeater.cc:344] Connected to a master server at 127.0.186.126:45183
I20260812 08:01:26.793335 10125 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:26.793912 10125 heartbeater.cc:507] Master 127.0.186.126:45183 requested a full tablet report, sending...
I20260812 08:01:26.795135  9028 ts_manager.cc:194] Re-registered known tserver with Master: c62ae7d1dcce4b2bab792fad5dbb4fa1 (127.0.186.65:42355)
I20260812 08:01:26.795784  9028 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.65:60367
I20260812 08:01:27.065420 10130 ts_tablet_manager.cc:933] T 6a6928a22d924847b068b4979c93b78f P 830db3153c9f430683f0e061910b65cf: Initiating tablet copy from peer e62f5ac41b1e4fdd8fc4f24179cffa4c (127.0.186.69:42173)
I20260812 08:01:27.066040 10130 tablet_copy_client.cc:323] T 6a6928a22d924847b068b4979c93b78f P 830db3153c9f430683f0e061910b65cf: tablet copy: Beginning tablet copy session from remote peer at address 127.0.186.69:42173
I20260812 08:01:27.069698  9825 tablet_copy_service.cc:140] P e62f5ac41b1e4fdd8fc4f24179cffa4c: Received BeginTabletCopySession request for tablet 6a6928a22d924847b068b4979c93b78f from peer 830db3153c9f430683f0e061910b65cf ({username='slave'} at 127.0.186.67:40481)
I20260812 08:01:27.069844  9825 tablet_copy_service.cc:161] P e62f5ac41b1e4fdd8fc4f24179cffa4c: Beginning new tablet copy session on tablet 6a6928a22d924847b068b4979c93b78f from peer 830db3153c9f430683f0e061910b65cf at {username='slave'} at 127.0.186.67:40481: session id = 830db3153c9f430683f0e061910b65cf-6a6928a22d924847b068b4979c93b78f
I20260812 08:01:27.071293  9825 tablet_copy_source_session.cc:215] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c: Tablet Copy: opened 5 blocks and 3 log segments
I20260812 08:01:27.075064 10130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a6928a22d924847b068b4979c93b78f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260812 08:01:27.080392 10130 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260812 08:01:27.080440 10130 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260812 08:01:27.080463 10130 fault_injection.cc:59] injecting fault for kudu (pid 9395): FLAGS_tablet_copy_fault_crash_on_fetch_all (process will exit)
W20260812 08:01:27.596926  9726 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.67:34687: connect: Connection refused (error 111)
I20260812 08:01:27.797861 10125 heartbeater.cc:499] Master 127.0.186.126:45183 was elected leader, sending a full tablet report...
W20260812 08:01:29.835510  9726 consensus_peers.cc:597] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c -> Peer 830db3153c9f430683f0e061910b65cf (127.0.186.67:34687): Couldn't send request to peer 830db3153c9f430683f0e061910b65cf. Error code: TABLET_NOT_FOUND (6). Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:34687: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260812 08:01:32.652667  9726 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.67:34687: connect: Connection refused (error 111) [suppressed 8 similar messages]
W20260812 08:01:32.653528  9726 consensus_peers.cc:597] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c -> Peer 830db3153c9f430683f0e061910b65cf (127.0.186.67:34687): Couldn't send request to peer 830db3153c9f430683f0e061910b65cf. Error code: TABLET_NOT_FOUND (6). Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:34687: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260812 08:01:35.204239  9726 consensus_peers.cc:597] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c -> Peer 830db3153c9f430683f0e061910b65cf (127.0.186.67:34687): Couldn't send request to peer 830db3153c9f430683f0e061910b65cf. Error code: TABLET_NOT_FOUND (6). Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:34687: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260812 08:01:37.177732 10156 consensus_queue.cc:579] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [LEADER]: Leader has been unable to successfully communicate with peer 830db3153c9f430683f0e061910b65cf for more than 10 seconds (10.097s)
I20260812 08:01:37.180534  9805 consensus_queue.cc:237] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 42, Committed index: 42, Last appended: 1.42, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } }
I20260812 08:01:37.181836  9650 raft_consensus.cc:1275] T 6a6928a22d924847b068b4979c93b78f P e82880c1240c4279a85a104f294d4b91 [term 1 FOLLOWER]: Refusing update from remote peer e62f5ac41b1e4fdd8fc4f24179cffa4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 42. Preceding OpId from leader: term: 1 index: 43. (index mismatch)
I20260812 08:01:37.182255 10185 consensus_queue.cc:1048] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 43, Last known committed idx: 42, Time since last communication: 0.000s
I20260812 08:01:37.183979 10156 raft_consensus.cc:2955] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [term 1 LEADER]: Committing config change with OpId 1.43: config changed from index 42 to 43, NON_VOTER 830db3153c9f430683f0e061910b65cf (127.0.186.67) evicted. New config: { opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } } }
I20260812 08:01:37.184432  9650 raft_consensus.cc:2955] T 6a6928a22d924847b068b4979c93b78f P e82880c1240c4279a85a104f294d4b91 [term 1 FOLLOWER]: Committing config change with OpId 1.43: config changed from index 42 to 43, NON_VOTER 830db3153c9f430683f0e061910b65cf (127.0.186.67) evicted. New config: { opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } } }
I20260812 08:01:37.185745  9012 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 6a6928a22d924847b068b4979c93b78f with cas_config_opid_index 42: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:01:37.186496  9028 catalog_manager.cc:5719] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c reported cstate change: config changed from index 42 to 43, NON_VOTER 830db3153c9f430683f0e061910b65cf (127.0.186.67) evicted. New cstate: current_term: 1 leader_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" committed_config { opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } health_report { overall_health: HEALTHY } } }
W20260812 08:01:37.189980  9013 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.67:34687: connect: Connection refused (error 111)
I20260812 08:01:37.190310  9805 consensus_queue.cc:237] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 43, Committed index: 43, Last appended: 1.43, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } } peers { permanent_uuid: "c62ae7d1dcce4b2bab792fad5dbb4fa1" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 42355 } attrs { promote: true } }
W20260812 08:01:37.190944  9013 catalog_manager.cc:4777] TS 830db3153c9f430683f0e061910b65cf (127.0.186.67:34687): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 6a6928a22d924847b068b4979c93b78f: Network error: Client connection negotiation failed: client connection to 127.0.186.67:34687: connect: Connection refused (error 111)
I20260812 08:01:37.191767  9650 raft_consensus.cc:1275] T 6a6928a22d924847b068b4979c93b78f P e82880c1240c4279a85a104f294d4b91 [term 1 FOLLOWER]: Refusing update from remote peer e62f5ac41b1e4fdd8fc4f24179cffa4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 43. Preceding OpId from leader: term: 1 index: 44. (index mismatch)
I20260812 08:01:37.192130 10185 consensus_queue.cc:1048] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 44, Last known committed idx: 43, Time since last communication: 0.000s
I20260812 08:01:37.193738 10156 raft_consensus.cc:2955] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c [term 1 LEADER]: Committing config change with OpId 1.44: config changed from index 43 to 44, NON_VOTER c62ae7d1dcce4b2bab792fad5dbb4fa1 (127.0.186.65) added. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } } peers { permanent_uuid: "c62ae7d1dcce4b2bab792fad5dbb4fa1" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 42355 } attrs { promote: true } } }
I20260812 08:01:37.194242  9650 raft_consensus.cc:2955] T 6a6928a22d924847b068b4979c93b78f P e82880c1240c4279a85a104f294d4b91 [term 1 FOLLOWER]: Committing config change with OpId 1.44: config changed from index 43 to 44, NON_VOTER c62ae7d1dcce4b2bab792fad5dbb4fa1 (127.0.186.65) added. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } } peers { permanent_uuid: "c62ae7d1dcce4b2bab792fad5dbb4fa1" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 42355 } attrs { promote: true } } }
I20260812 08:01:37.195294  9012 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 6a6928a22d924847b068b4979c93b78f with cas_config_opid_index 43: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:01:37.196416  9028 catalog_manager.cc:5719] T 6a6928a22d924847b068b4979c93b78f P e82880c1240c4279a85a104f294d4b91 reported cstate change: config changed from index 43 to 44, NON_VOTER c62ae7d1dcce4b2bab792fad5dbb4fa1 (127.0.186.65) added. New cstate: current_term: 1 leader_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" committed_config { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } } peers { permanent_uuid: "c62ae7d1dcce4b2bab792fad5dbb4fa1" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 42355 } attrs { promote: true } } }
W20260812 08:01:37.198336  9727 consensus_peers.cc:597] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c -> Peer c62ae7d1dcce4b2bab792fad5dbb4fa1 (127.0.186.65:42355): Couldn't send request to peer c62ae7d1dcce4b2bab792fad5dbb4fa1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6a6928a22d924847b068b4979c93b78f. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:01:37.646068 10200 ts_tablet_manager.cc:933] T 6a6928a22d924847b068b4979c93b78f P c62ae7d1dcce4b2bab792fad5dbb4fa1: Initiating tablet copy from peer e62f5ac41b1e4fdd8fc4f24179cffa4c (127.0.186.69:42173)
I20260812 08:01:37.646677 10200 tablet_copy_client.cc:323] T 6a6928a22d924847b068b4979c93b78f P c62ae7d1dcce4b2bab792fad5dbb4fa1: tablet copy: Beginning tablet copy session from remote peer at address 127.0.186.69:42173
I20260812 08:01:37.650534  9825 tablet_copy_service.cc:140] P e62f5ac41b1e4fdd8fc4f24179cffa4c: Received BeginTabletCopySession request for tablet 6a6928a22d924847b068b4979c93b78f from peer c62ae7d1dcce4b2bab792fad5dbb4fa1 ({username='slave'} at 127.0.186.65:46743)
I20260812 08:01:37.650646  9825 tablet_copy_service.cc:161] P e62f5ac41b1e4fdd8fc4f24179cffa4c: Beginning new tablet copy session on tablet 6a6928a22d924847b068b4979c93b78f from peer c62ae7d1dcce4b2bab792fad5dbb4fa1 at {username='slave'} at 127.0.186.65:46743: session id = c62ae7d1dcce4b2bab792fad5dbb4fa1-6a6928a22d924847b068b4979c93b78f
I20260812 08:01:37.651528  9825 tablet_copy_source_session.cc:215] T 6a6928a22d924847b068b4979c93b78f P e62f5ac41b1e4fdd8fc4f24179cffa4c: Tablet Copy: opened 5 blocks and 3 log segments
I20260812 08:01:37.655105 10200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a6928a22d924847b068b4979c93b78f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:01:37.660215 10200 tablet_copy_client.cc:806] T 6a6928a22d924847b068b4979c93b78f P c62ae7d1dcce4b2bab792fad5dbb4fa1: tablet copy: Starting download of 5 data blocks...
I20260812 08:01:37.667426 10200 tablet_copy_client.cc:670] T 6a6928a22d924847b068b4979c93b78f P c62ae7d1dcce4b2bab792fad5dbb4fa1: tablet copy: Starting download of 3 WAL segments...
I20260812 08:01:37.675001 10200 tablet_copy_client.cc:538] T 6a6928a22d924847b068b4979c93b78f P c62ae7d1dcce4b2bab792fad5dbb4fa1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 08:01:37.677143 10200 tablet_bootstrap.cc:492] T 6a6928a22d924847b068b4979c93b78f P c62ae7d1dcce4b2bab792fad5dbb4fa1: Bootstrap starting.
I20260812 08:01:37.684574 10200 log.cc:826] T 6a6928a22d924847b068b4979c93b78f P c62ae7d1dcce4b2bab792fad5dbb4fa1: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:37.691205 10200 tablet_bootstrap.cc:492] T 6a6928a22d924847b068b4979c93b78f P c62ae7d1dcce4b2bab792fad5dbb4fa1: Bootstrap replayed 1/3 log segments. Stats: ops{read=7 overwritten=0 applied=6 ignored=6} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260812 08:01:37.694871 10200 tablet_bootstrap.cc:492] T 6a6928a22d924847b068b4979c93b78f P c62ae7d1dcce4b2bab792fad5dbb4fa1: Bootstrap replayed 2/3 log segments. Stats: ops{read=14 overwritten=0 applied=11 ignored=11} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 3 replicates
I20260812 08:01:37.699067 10200 tablet_bootstrap.cc:492] T 6a6928a22d924847b068b4979c93b78f P c62ae7d1dcce4b2bab792fad5dbb4fa1: Bootstrap replayed 3/3 log segments. Stats: ops{read=22 overwritten=0 applied=22 ignored=18} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:01:37.699422 10200 tablet_bootstrap.cc:492] T 6a6928a22d924847b068b4979c93b78f P c62ae7d1dcce4b2bab792fad5dbb4fa1: Bootstrap complete.
I20260812 08:01:37.700116 10200 ts_tablet_manager.cc:1403] T 6a6928a22d924847b068b4979c93b78f P c62ae7d1dcce4b2bab792fad5dbb4fa1: Time spent bootstrapping tablet: real 0.023s	user 0.023s	sys 0.000s
I20260812 08:01:37.702265 10200 raft_consensus.cc:359] T 6a6928a22d924847b068b4979c93b78f P c62ae7d1dcce4b2bab792fad5dbb4fa1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } } peers { permanent_uuid: "c62ae7d1dcce4b2bab792fad5dbb4fa1" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 42355 } attrs { promote: true } }
I20260812 08:01:37.702479 10200 raft_consensus.cc:740] T 6a6928a22d924847b068b4979c93b78f P c62ae7d1dcce4b2bab792fad5dbb4fa1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c62ae7d1dcce4b2bab792fad5dbb4fa1, State: Initialized, Role: LEARNER
I20260812 08:01:37.702818 10200 consensus_queue.cc:260] T 6a6928a22d924847b068b4979c93b78f P c62ae7d1dcce4b2bab792fad5dbb4fa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 44, Last appended: 1.44, Last appended by leader: 44, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "e62f5ac41b1e4fdd8fc4f24179cffa4c" member_type: VOTER last_known_addr { host: "127.0.186.69" port: 42173 } } peers { permanent_uuid: "e82880c1240c4279a85a104f294d4b91" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 37125 } } peers { permanent_uuid: "c62ae7d1dcce4b2bab792fad5dbb4fa1" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 42355 } attrs { promote: true } }
I20260812 08:01:37.703670 10200 ts_tablet_manager.cc:1434] T 6a6928a22d924847b068b4979c93b78f P c62ae7d1dcce4b2bab792fad5dbb4fa1: Time spent starting tablet: real 0.003s	user 0.006s	sys 0.000s
I20260812 08:01:37.704638  9824 tablet_copy_service.cc:342] P e62f5ac41b1e4fdd8fc4f24179cffa4c: Request end of tablet copy session c62ae7d1dcce4b2bab792fad5dbb4fa1-6a6928a22d924847b068b4979c93b78f received from {username='slave'} at 127.0.186.65:46743
I20260812 08:01:37.704766  9824 tablet_copy_service.cc:434] P e62f5ac41b1e4fdd8fc4f24179cffa4c: ending tablet copy session c62ae7d1dcce4b2bab792fad5dbb4fa1-6a6928a22d924847b068b4979c93b78f on tablet 6a6928a22d924847b068b4979c93b78f with peer c62ae7d1dcce4b2bab792fad5dbb4fa1
I20260812 08:01:37.707121 10127 maintenance_manager.cc:419] P c62ae7d1dcce4b2bab792fad5dbb4fa1: Scheduling UndoDeltaBlockGCOp(6a6928a22d924847b068b4979c93b78f): 569 bytes on disk
I20260812 08:01:37.707917  9997 maintenance_manager.cc:643] P c62ae7d1dcce4b2bab792fad5dbb4fa1: UndoDeltaBlockGCOp(6a6928a22d924847b068b4979c93b78f) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":53,"lbm_reads_lt_1ms":4}
I20260812 08:01:37.903386   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 9977
I20260812 08:01:37.914328   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 9249
I20260812 08:01:37.923766   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 9551
I20260812 08:01:37.933271   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 9707
I20260812 08:01:37.943980   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 8994
2026-08-12T08:01:37Z chronyd exiting
[       OK ] RaftConsensusNonVoterITest.FailedTabletCopy (20095 ms)
[ RUN      ] RaftConsensusNonVoterITest.RestartClusterWithNonVoter
I20260812 08:01:37.973495   745 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:01:37.973549   745 ts_itest-base.cc:116] --------------
I20260812 08:01:37.973560   745 ts_itest-base.cc:117] 7 tablet servers
I20260812 08:01:37.973570   745 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:01:37.973580   745 ts_itest-base.cc:119] --------------
2026-08-12T08:01:37Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:01:37Z Disabled control of system clock
I20260812 08:01:37.982825   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:39483
--webserver_interface=127.0.186.126
--webserver_port=0
--builtin_ntp_servers=127.0.186.84:46389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:39483
--raft_prepare_replacement_before_eviction=true with env {}
W20260812 08:01:38.091979 10217 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:38.092229 10217 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:38.092284 10217 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:38.095713 10217 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:01:38.095777 10217 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:38.095804 10217 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:01:38.095827 10217 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:01:38.099766 10217 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:46389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:39483
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:39483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.10217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:38.100955 10217 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:38.102207 10217 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:38.108328 10222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:38.108323 10224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:38.108392 10217 server_base.cc:1061] running on GCE node
W20260812 08:01:38.108320 10226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:38.108978 10217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:38.110057 10217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:38.111238 10217 hybrid_clock.cc:648] HybridClock initialized: now 1786521698111226 us; error 28 us; skew 500 ppm
I20260812 08:01:38.113022 10217 webserver.cc:533] Webserver started at http://127.0.186.126:37805/ using document root <none> and password file <none>
I20260812 08:01:38.113651 10217 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:38.113713 10217 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:38.113912 10217 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:01:38.115522 10217 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "717537b7ea584eafbbe1e6a991ab9e2f"
format_stamp: "Formatted at 2026-08-12 08:01:38 on dist-test-slave-49h3"
I20260812 08:01:38.116007 10217 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "717537b7ea584eafbbe1e6a991ab9e2f"
format_stamp: "Formatted at 2026-08-12 08:01:38 on dist-test-slave-49h3"
I20260812 08:01:38.119333 10217 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:01:38.121538 10232 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:38.122591 10217 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:38.122713 10217 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "717537b7ea584eafbbe1e6a991ab9e2f"
format_stamp: "Formatted at 2026-08-12 08:01:38 on dist-test-slave-49h3"
I20260812 08:01:38.122802 10217 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:38.138806 10217 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:38.139528 10217 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:38.139707 10217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:38.147240 10217 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:39483
I20260812 08:01:38.147290 10293 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:39483 every 8 connection(s)
I20260812 08:01:38.148304 10217 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:01:38.151073   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 10217
I20260812 08:01:38.151180   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:01:38.151275 10294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:01:38.156997 10294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f: Bootstrap starting.
I20260812 08:01:38.159485 10294 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f: Neither blocks nor log segments found. Creating new log.
I20260812 08:01:38.160161 10294 log.cc:826] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:38.161773 10294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f: No bootstrap required, opened a new log
I20260812 08:01:38.164148 10294 raft_consensus.cc:359] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "717537b7ea584eafbbe1e6a991ab9e2f" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 39483 } }
I20260812 08:01:38.164336 10294 raft_consensus.cc:385] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:01:38.164408 10294 raft_consensus.cc:740] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 717537b7ea584eafbbe1e6a991ab9e2f, State: Initialized, Role: FOLLOWER
I20260812 08:01:38.164789 10294 consensus_queue.cc:260] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "717537b7ea584eafbbe1e6a991ab9e2f" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 39483 } }
I20260812 08:01:38.164911 10294 raft_consensus.cc:399] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:01:38.164971 10294 raft_consensus.cc:493] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:01:38.165072 10294 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:01:38.165854 10294 raft_consensus.cc:515] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "717537b7ea584eafbbe1e6a991ab9e2f" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 39483 } }
I20260812 08:01:38.166195 10294 leader_election.cc:304] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 717537b7ea584eafbbe1e6a991ab9e2f; no voters: 
I20260812 08:01:38.166596 10294 leader_election.cc:290] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:01:38.166735 10299 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:01:38.166939 10299 raft_consensus.cc:697] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [term 1 LEADER]: Becoming Leader. State: Replica: 717537b7ea584eafbbe1e6a991ab9e2f, State: Running, Role: LEADER
I20260812 08:01:38.167301 10299 consensus_queue.cc:237] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "717537b7ea584eafbbe1e6a991ab9e2f" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 39483 } }
I20260812 08:01:38.167810 10294 sys_catalog.cc:565] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:01:38.168426 10301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 717537b7ea584eafbbe1e6a991ab9e2f. Latest consensus state: current_term: 1 leader_uuid: "717537b7ea584eafbbe1e6a991ab9e2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "717537b7ea584eafbbe1e6a991ab9e2f" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 39483 } } }
I20260812 08:01:38.168606 10301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [sys.catalog]: This master's current role is: LEADER
I20260812 08:01:38.168879 10302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "717537b7ea584eafbbe1e6a991ab9e2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "717537b7ea584eafbbe1e6a991ab9e2f" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 39483 } } }
I20260812 08:01:38.169013 10302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [sys.catalog]: This master's current role is: LEADER
I20260812 08:01:38.169123 10307 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:01:38.171912 10307 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:01:38.177495 10307 catalog_manager.cc:1383] Generated new cluster ID: 87fd517f85154b10ac39f25b50bd7d8f
I20260812 08:01:38.177570 10307 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:01:38.196209 10307 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:01:38.197062 10307 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:01:38.206293 10307 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f: Generated new TSK 0
I20260812 08:01:38.206874 10307 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:01:38.209872   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:0
--local_ip_for_outbound_sockets=127.0.186.65
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:39483
--builtin_ntp_servers=127.0.186.84:46389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=10000
--follower_unavailable_considered_failed_sec=2
--consensus_rpc_timeout_ms=5000 with env {}
W20260812 08:01:38.316205 10323 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:38.316414 10323 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:38.316464 10323 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:38.319820 10323 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:38.319936 10323 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:01:38.323799 10323 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:46389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=2
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=10000
--tserver_master_addrs=127.0.186.126:39483
--never_fsync=true
--heap_profile_path=/tmp/kudu.10323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:38.324852 10323 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:38.326046 10323 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:38.332263 10334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:38.332266 10330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:38.332422 10323 server_base.cc:1061] running on GCE node
W20260812 08:01:38.332526 10332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:38.332865 10323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:38.333391 10323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:38.334553 10323 hybrid_clock.cc:648] HybridClock initialized: now 1786521698334530 us; error 34 us; skew 500 ppm
I20260812 08:01:38.336170 10323 webserver.cc:533] Webserver started at http://127.0.186.65:44845/ using document root <none> and password file <none>
I20260812 08:01:38.336746 10323 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:38.336807 10323 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:38.336985 10323 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:01:38.338531 10323 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "2ef089e13bd1432e9fa75814933f3bec"
format_stamp: "Formatted at 2026-08-12 08:01:38 on dist-test-slave-49h3"
I20260812 08:01:38.338996 10323 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "2ef089e13bd1432e9fa75814933f3bec"
format_stamp: "Formatted at 2026-08-12 08:01:38 on dist-test-slave-49h3"
I20260812 08:01:38.342172 10323 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:01:38.344218 10341 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:38.345119 10323 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:38.345222 10323 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "2ef089e13bd1432e9fa75814933f3bec"
format_stamp: "Formatted at 2026-08-12 08:01:38 on dist-test-slave-49h3"
I20260812 08:01:38.345304 10323 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:38.371594 10323 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:38.372313 10323 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:38.372521 10323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:38.373313 10323 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:38.374652 10323 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:01:38.374723 10323 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:38.374778 10323 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:01:38.374807 10323 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:38.383546 10323 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:35839
I20260812 08:01:38.383574 10474 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:35839 every 8 connection(s)
I20260812 08:01:38.384521 10323 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:01:38.386559   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 10323
I20260812 08:01:38.386667   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:01:38.389071   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.66:0
--local_ip_for_outbound_sockets=127.0.186.66
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:39483
--builtin_ntp_servers=127.0.186.84:46389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=10000
--follower_unavailable_considered_failed_sec=2
--consensus_rpc_timeout_ms=5000 with env {}
I20260812 08:01:38.392992 10475 heartbeater.cc:344] Connected to a master server at 127.0.186.126:39483
I20260812 08:01:38.393227 10475 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:38.393760 10475 heartbeater.cc:507] Master 127.0.186.126:39483 requested a full tablet report, sending...
I20260812 08:01:38.395367 10251 ts_manager.cc:194] Registered new tserver with Master: 2ef089e13bd1432e9fa75814933f3bec (127.0.186.65:35839)
I20260812 08:01:38.396958 10251 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.65:42573
W20260812 08:01:38.497651 10481 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:38.497969 10481 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:38.498045 10481 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:38.502813 10481 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:38.502969 10481 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.66
I20260812 08:01:38.508673 10481 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:46389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=2
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.186.66
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=10000
--tserver_master_addrs=127.0.186.126:39483
--never_fsync=true
--heap_profile_path=/tmp/kudu.10481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:38.510236 10481 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:38.511821 10481 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:38.519938 10489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:38.519995 10481 server_base.cc:1061] running on GCE node
W20260812 08:01:38.519948 10493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:38.520275 10490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:38.520663 10481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:38.521276 10481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:38.522431 10481 hybrid_clock.cc:648] HybridClock initialized: now 1786521698522410 us; error 39 us; skew 500 ppm
I20260812 08:01:38.524161 10481 webserver.cc:533] Webserver started at http://127.0.186.66:36465/ using document root <none> and password file <none>
I20260812 08:01:38.524731 10481 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:38.524791 10481 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:38.524973 10481 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:01:38.526567 10481 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "c5c99765f674430db87f0bd401e7a193"
format_stamp: "Formatted at 2026-08-12 08:01:38 on dist-test-slave-49h3"
I20260812 08:01:38.527081 10481 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "c5c99765f674430db87f0bd401e7a193"
format_stamp: "Formatted at 2026-08-12 08:01:38 on dist-test-slave-49h3"
I20260812 08:01:38.530563 10481 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:01:38.532876 10499 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:38.533954 10481 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:01:38.534065 10481 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "c5c99765f674430db87f0bd401e7a193"
format_stamp: "Formatted at 2026-08-12 08:01:38 on dist-test-slave-49h3"
I20260812 08:01:38.534152 10481 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:38.548842 10481 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:38.549723 10481 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:38.549983 10481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:38.550694 10481 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:38.551972 10481 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:01:38.552026 10481 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:38.552080 10481 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:01:38.552105 10481 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:38.562037 10481 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.66:43103
I20260812 08:01:38.562088 10631 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.66:43103 every 8 connection(s)
I20260812 08:01:38.563045 10481 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:01:38.566190   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 10481
I20260812 08:01:38.566298   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:01:38.568511   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.67:0
--local_ip_for_outbound_sockets=127.0.186.67
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:39483
--builtin_ntp_servers=127.0.186.84:46389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=10000
--follower_unavailable_considered_failed_sec=2
--consensus_rpc_timeout_ms=5000 with env {}
I20260812 08:01:38.572945 10632 heartbeater.cc:344] Connected to a master server at 127.0.186.126:39483
I20260812 08:01:38.573216 10632 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:38.573841 10632 heartbeater.cc:507] Master 127.0.186.126:39483 requested a full tablet report, sending...
I20260812 08:01:38.574920 10251 ts_manager.cc:194] Registered new tserver with Master: c5c99765f674430db87f0bd401e7a193 (127.0.186.66:43103)
I20260812 08:01:38.575484 10251 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.66:52081
W20260812 08:01:38.685587 10635 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:38.685791 10635 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:38.685839 10635 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:38.689049 10635 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:38.689154 10635 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.67
I20260812 08:01:38.692797 10635 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:46389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=2
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.186.67
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=10000
--tserver_master_addrs=127.0.186.126:39483
--never_fsync=true
--heap_profile_path=/tmp/kudu.10635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:38.693847 10635 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:38.694939 10635 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:38.701167 10642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:38.701210 10645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:38.701182 10643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:38.701292 10635 server_base.cc:1061] running on GCE node
I20260812 08:01:38.701717 10635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:38.702287 10635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:38.703457 10635 hybrid_clock.cc:648] HybridClock initialized: now 1786521698703438 us; error 27 us; skew 500 ppm
I20260812 08:01:38.705284 10635 webserver.cc:533] Webserver started at http://127.0.186.67:36731/ using document root <none> and password file <none>
I20260812 08:01:38.705901 10635 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:38.705967 10635 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:38.706238 10635 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:01:38.707870 10635 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "0fc4212fb0854d5d8b405156a6f8c4bc"
format_stamp: "Formatted at 2026-08-12 08:01:38 on dist-test-slave-49h3"
I20260812 08:01:38.708351 10635 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "0fc4212fb0854d5d8b405156a6f8c4bc"
format_stamp: "Formatted at 2026-08-12 08:01:38 on dist-test-slave-49h3"
I20260812 08:01:38.711593 10635 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:01:38.713657 10652 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:38.714617 10635 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:38.714720 10635 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "0fc4212fb0854d5d8b405156a6f8c4bc"
format_stamp: "Formatted at 2026-08-12 08:01:38 on dist-test-slave-49h3"
I20260812 08:01:38.714799 10635 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:38.782328 10635 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:38.783085 10635 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:38.783269 10635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:38.783892 10635 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:38.784979 10635 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:01:38.785030 10635 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:38.785095 10635 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:01:38.785120 10635 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:38.794898 10635 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.67:36027
I20260812 08:01:38.794924 10784 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.67:36027 every 8 connection(s)
I20260812 08:01:38.796482 10635 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:01:38.796792   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 10635
I20260812 08:01:38.796906   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:01:38.801198   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.68:0
--local_ip_for_outbound_sockets=127.0.186.68
--webserver_interface=127.0.186.68
--webserver_port=0
--tserver_master_addrs=127.0.186.126:39483
--builtin_ntp_servers=127.0.186.84:46389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=10000
--follower_unavailable_considered_failed_sec=2
--consensus_rpc_timeout_ms=5000 with env {}
I20260812 08:01:38.808615 10785 heartbeater.cc:344] Connected to a master server at 127.0.186.126:39483
I20260812 08:01:38.808877 10785 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:38.809485 10785 heartbeater.cc:507] Master 127.0.186.126:39483 requested a full tablet report, sending...
I20260812 08:01:38.810603 10251 ts_manager.cc:194] Registered new tserver with Master: 0fc4212fb0854d5d8b405156a6f8c4bc (127.0.186.67:36027)
I20260812 08:01:38.811295 10251 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.67:43361
W20260812 08:01:38.915825 10788 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:38.916013 10788 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:38.916054 10788 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:38.919261 10788 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:38.919358 10788 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.68
I20260812 08:01:38.923022 10788 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:46389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=2
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.0.186.68
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=10000
--tserver_master_addrs=127.0.186.126:39483
--never_fsync=true
--heap_profile_path=/tmp/kudu.10788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.68

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:38.924028 10788 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:38.925129 10788 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:38.931427 10794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:38.931486 10793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:38.931636 10788 server_base.cc:1061] running on GCE node
W20260812 08:01:38.931542 10797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:38.932137 10788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:38.932674 10788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:38.933820 10788 hybrid_clock.cc:648] HybridClock initialized: now 1786521698933809 us; error 27 us; skew 500 ppm
I20260812 08:01:38.935547 10788 webserver.cc:533] Webserver started at http://127.0.186.68:38503/ using document root <none> and password file <none>
I20260812 08:01:38.936105 10788 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:38.936161 10788 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:38.936344 10788 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:01:38.937928 10788 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "5f0292e14d114be2be61f1b49041540f"
format_stamp: "Formatted at 2026-08-12 08:01:38 on dist-test-slave-49h3"
I20260812 08:01:38.938478 10788 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "5f0292e14d114be2be61f1b49041540f"
format_stamp: "Formatted at 2026-08-12 08:01:38 on dist-test-slave-49h3"
I20260812 08:01:38.941790 10788 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:01:38.943904 10804 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:38.944888 10788 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:38.945007 10788 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "5f0292e14d114be2be61f1b49041540f"
format_stamp: "Formatted at 2026-08-12 08:01:38 on dist-test-slave-49h3"
I20260812 08:01:38.945107 10788 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:38.971830 10788 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:38.972512 10788 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:38.972694 10788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:38.973278 10788 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:38.974422 10788 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:01:38.974470 10788 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:38.974525 10788 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:01:38.974555 10788 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:38.983246 10788 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.68:43825
I20260812 08:01:38.983304 10940 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.68:43825 every 8 connection(s)
I20260812 08:01:38.984242 10788 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20260812 08:01:38.988291   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 10788
I20260812 08:01:38.988410   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20260812 08:01:38.990873   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.69:0
--local_ip_for_outbound_sockets=127.0.186.69
--webserver_interface=127.0.186.69
--webserver_port=0
--tserver_master_addrs=127.0.186.126:39483
--builtin_ntp_servers=127.0.186.84:46389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=10000
--follower_unavailable_considered_failed_sec=2
--consensus_rpc_timeout_ms=5000 with env {}
I20260812 08:01:38.997090 10941 heartbeater.cc:344] Connected to a master server at 127.0.186.126:39483
I20260812 08:01:38.997345 10941 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:38.997921 10941 heartbeater.cc:507] Master 127.0.186.126:39483 requested a full tablet report, sending...
I20260812 08:01:38.999119 10251 ts_manager.cc:194] Registered new tserver with Master: 5f0292e14d114be2be61f1b49041540f (127.0.186.68:43825)
I20260812 08:01:38.999918 10251 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.68:40093
W20260812 08:01:39.100888 10945 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:39.101177 10945 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:39.101248 10945 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:39.105734 10945 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:39.105901 10945 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.69
I20260812 08:01:39.110983 10945 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:46389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=2
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.69:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.0.186.69
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=10000
--tserver_master_addrs=127.0.186.126:39483
--never_fsync=true
--heap_profile_path=/tmp/kudu.10945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.69

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:39.112527 10945 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:39.114205 10945 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:39.123391 10952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:39.123406 10950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:39.123412 10955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:39.123512 10945 server_base.cc:1061] running on GCE node
I20260812 08:01:39.124156 10945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:39.124900 10945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:39.126080 10945 hybrid_clock.cc:648] HybridClock initialized: now 1786521699126060 us; error 47 us; skew 500 ppm
I20260812 08:01:39.128687 10945 webserver.cc:533] Webserver started at http://127.0.186.69:40787/ using document root <none> and password file <none>
I20260812 08:01:39.129515 10945 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:39.129611 10945 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:39.129904 10945 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:01:39.131986 10945 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "b8e3b126cd934c6e85b1c5f9442ac7f5"
format_stamp: "Formatted at 2026-08-12 08:01:39 on dist-test-slave-49h3"
I20260812 08:01:39.132651 10945 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "b8e3b126cd934c6e85b1c5f9442ac7f5"
format_stamp: "Formatted at 2026-08-12 08:01:39 on dist-test-slave-49h3"
I20260812 08:01:39.137676 10945 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260812 08:01:39.140895 10962 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:39.142510 10945 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260812 08:01:39.142668 10945 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "b8e3b126cd934c6e85b1c5f9442ac7f5"
format_stamp: "Formatted at 2026-08-12 08:01:39 on dist-test-slave-49h3"
I20260812 08:01:39.142810 10945 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:39.165895 10945 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:39.166747 10945 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:39.166961 10945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:39.167738 10945 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:39.169181 10945 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:01:39.169255 10945 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:39.169317 10945 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:01:39.169348 10945 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:39.182394 10945 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.69:46205
I20260812 08:01:39.182453 11096 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.69:46205 every 8 connection(s)
I20260812 08:01:39.183713 10945 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20260812 08:01:39.188155   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 10945
I20260812 08:01:39.188298   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20260812 08:01:39.191673   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.70:0
--local_ip_for_outbound_sockets=127.0.186.70
--webserver_interface=127.0.186.70
--webserver_port=0
--tserver_master_addrs=127.0.186.126:39483
--builtin_ntp_servers=127.0.186.84:46389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=10000
--follower_unavailable_considered_failed_sec=2
--consensus_rpc_timeout_ms=5000 with env {}
I20260812 08:01:39.195526 11097 heartbeater.cc:344] Connected to a master server at 127.0.186.126:39483
I20260812 08:01:39.195886 11097 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:39.196791 11097 heartbeater.cc:507] Master 127.0.186.126:39483 requested a full tablet report, sending...
I20260812 08:01:39.198398 10251 ts_manager.cc:194] Registered new tserver with Master: b8e3b126cd934c6e85b1c5f9442ac7f5 (127.0.186.69:46205)
I20260812 08:01:39.199359 10251 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.69:50593
W20260812 08:01:39.303647 11100 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:39.303844 11100 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:39.303887 11100 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:39.307155 11100 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:39.307256 11100 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.70
I20260812 08:01:39.311105 11100 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:46389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=2
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.70:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.0.186.70
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=10000
--tserver_master_addrs=127.0.186.126:39483
--never_fsync=true
--heap_profile_path=/tmp/kudu.11100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.70

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:39.312160 11100 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:39.313234 11100 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:39.319608 11106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:39.319675 11109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:39.319799 11107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:39.319867 11100 server_base.cc:1061] running on GCE node
I20260812 08:01:39.320230 11100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:39.320712 11100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:39.321870 11100 hybrid_clock.cc:648] HybridClock initialized: now 1786521699321849 us; error 31 us; skew 500 ppm
I20260812 08:01:39.323527 11100 webserver.cc:533] Webserver started at http://127.0.186.70:33971/ using document root <none> and password file <none>
I20260812 08:01:39.324038 11100 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:39.324096 11100 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:39.324250 11100 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:01:39.325865 11100 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "df29c7b34b2e4b8d9eddef7c5f166e06"
format_stamp: "Formatted at 2026-08-12 08:01:39 on dist-test-slave-49h3"
I20260812 08:01:39.326373 11100 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "df29c7b34b2e4b8d9eddef7c5f166e06"
format_stamp: "Formatted at 2026-08-12 08:01:39 on dist-test-slave-49h3"
I20260812 08:01:39.329737 11100 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:01:39.331753 11116 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:39.332702 11100 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:39.332798 11100 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "df29c7b34b2e4b8d9eddef7c5f166e06"
format_stamp: "Formatted at 2026-08-12 08:01:39 on dist-test-slave-49h3"
I20260812 08:01:39.332888 11100 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:39.346791 11100 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:39.347422 11100 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:39.347602 11100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:39.348158 11100 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:39.349280 11100 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:01:39.349342 11100 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:39.349395 11100 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:01:39.349424 11100 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:39.358222 11100 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.70:45261
I20260812 08:01:39.358261 11247 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.70:45261 every 8 connection(s)
I20260812 08:01:39.359146 11100 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20260812 08:01:39.360042   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 11100
I20260812 08:01:39.360140   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20260812 08:01:39.362444   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.71:0
--local_ip_for_outbound_sockets=127.0.186.71
--webserver_interface=127.0.186.71
--webserver_port=0
--tserver_master_addrs=127.0.186.126:39483
--builtin_ntp_servers=127.0.186.84:46389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=10000
--follower_unavailable_considered_failed_sec=2
--consensus_rpc_timeout_ms=5000 with env {}
I20260812 08:01:39.370654 11248 heartbeater.cc:344] Connected to a master server at 127.0.186.126:39483
I20260812 08:01:39.370918 11248 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:39.371451 11248 heartbeater.cc:507] Master 127.0.186.126:39483 requested a full tablet report, sending...
I20260812 08:01:39.372583 10251 ts_manager.cc:194] Registered new tserver with Master: df29c7b34b2e4b8d9eddef7c5f166e06 (127.0.186.70:45261)
I20260812 08:01:39.373139 10251 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.70:53389
I20260812 08:01:39.399060 10475 heartbeater.cc:499] Master 127.0.186.126:39483 was elected leader, sending a full tablet report...
W20260812 08:01:39.471697 11251 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:39.471901 11251 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:39.471944 11251 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:39.475219 11251 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:39.475329 11251 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.71
I20260812 08:01:39.479138 11251 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:46389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=2
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.71:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.0.186.71
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=10000
--tserver_master_addrs=127.0.186.126:39483
--never_fsync=true
--heap_profile_path=/tmp/kudu.11251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.71

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:39.480201 11251 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:39.481279 11251 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:39.487677 11258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:39.487689 11257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:39.487685 11260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:39.487900 11251 server_base.cc:1061] running on GCE node
I20260812 08:01:39.488300 11251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:39.488824 11251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:39.489956 11251 hybrid_clock.cc:648] HybridClock initialized: now 1786521699489931 us; error 37 us; skew 500 ppm
I20260812 08:01:39.491628 11251 webserver.cc:533] Webserver started at http://127.0.186.71:42403/ using document root <none> and password file <none>
I20260812 08:01:39.492169 11251 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:39.492224 11251 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:39.492422 11251 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:01:39.493965 11251 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "0daf92242f824102a4d9d0feaa7604dc"
format_stamp: "Formatted at 2026-08-12 08:01:39 on dist-test-slave-49h3"
I20260812 08:01:39.494421 11251 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "0daf92242f824102a4d9d0feaa7604dc"
format_stamp: "Formatted at 2026-08-12 08:01:39 on dist-test-slave-49h3"
I20260812 08:01:39.497738 11251 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:01:39.499863 11268 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:39.500922 11251 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:39.501031 11251 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "0daf92242f824102a4d9d0feaa7604dc"
format_stamp: "Formatted at 2026-08-12 08:01:39 on dist-test-slave-49h3"
I20260812 08:01:39.501137 11251 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:39.510701 11251 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:39.511307 11251 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:39.511471 11251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:39.512035 11251 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:39.513123 11251 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:01:39.513173 11251 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:39.513212 11251 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:01:39.513242 11251 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:39.525131 11251 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.71:38427
I20260812 08:01:39.525209 11404 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.71:38427 every 8 connection(s)
I20260812 08:01:39.526189 11251 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20260812 08:01:39.529596   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 11251
I20260812 08:01:39.529698   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20260812 08:01:39.534363 11405 heartbeater.cc:344] Connected to a master server at 127.0.186.126:39483
I20260812 08:01:39.534580 11405 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:39.535036 11405 heartbeater.cc:507] Master 127.0.186.126:39483 requested a full tablet report, sending...
I20260812 08:01:39.536024 10251 ts_manager.cc:194] Registered new tserver with Master: 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71:38427)
I20260812 08:01:39.536610 10251 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.71:58229
I20260812 08:01:39.543831   745 external_mini_cluster.cc:955] 7 TS(s) registered with all masters
I20260812 08:01:39.552611 10251 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33482:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
I20260812 08:01:39.577301 10632 heartbeater.cc:499] Master 127.0.186.126:39483 was elected leader, sending a full tablet report...
I20260812 08:01:39.577090 10395 tablet_service.cc:1511] Processing CreateTablet for tablet 9e91e10f1ecf4319b2b55a3eca65dfde (DEFAULT_TABLE table=TestTable [id=d1cff5d81143457a8a1c6eb392aae269]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:01:39.578807 10395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e91e10f1ecf4319b2b55a3eca65dfde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:01:39.579535 11326 tablet_service.cc:1511] Processing CreateTablet for tablet 9e91e10f1ecf4319b2b55a3eca65dfde (DEFAULT_TABLE table=TestTable [id=d1cff5d81143457a8a1c6eb392aae269]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:01:39.579888 10863 tablet_service.cc:1511] Processing CreateTablet for tablet 9e91e10f1ecf4319b2b55a3eca65dfde (DEFAULT_TABLE table=TestTable [id=d1cff5d81143457a8a1c6eb392aae269]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:01:39.580451 11326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e91e10f1ecf4319b2b55a3eca65dfde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:01:39.580711 10863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e91e10f1ecf4319b2b55a3eca65dfde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:01:39.583379 11427 tablet_bootstrap.cc:492] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec: Bootstrap starting.
I20260812 08:01:39.585279 11427 tablet_bootstrap.cc:654] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec: Neither blocks nor log segments found. Creating new log.
I20260812 08:01:39.586181 11427 log.cc:826] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:39.586795 11429 tablet_bootstrap.cc:492] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f: Bootstrap starting.
I20260812 08:01:39.587895 11427 tablet_bootstrap.cc:492] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec: No bootstrap required, opened a new log
I20260812 08:01:39.588105 11427 ts_tablet_manager.cc:1403] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.000s
I20260812 08:01:39.589365 11429 tablet_bootstrap.cc:654] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f: Neither blocks nor log segments found. Creating new log.
I20260812 08:01:39.590201 11429 log.cc:826] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:39.591137 11427 raft_consensus.cc:359] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } }
I20260812 08:01:39.591346 11427 raft_consensus.cc:385] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:01:39.591403 11427 raft_consensus.cc:740] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ef089e13bd1432e9fa75814933f3bec, State: Initialized, Role: FOLLOWER
I20260812 08:01:39.591836 11427 consensus_queue.cc:260] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } }
I20260812 08:01:39.591934 11429 tablet_bootstrap.cc:492] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f: No bootstrap required, opened a new log
I20260812 08:01:39.592115 11429 ts_tablet_manager.cc:1403] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:01:39.592957 11427 ts_tablet_manager.cc:1434] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260812 08:01:39.593518 11428 tablet_bootstrap.cc:492] T 9e91e10f1ecf4319b2b55a3eca65dfde P 0daf92242f824102a4d9d0feaa7604dc: Bootstrap starting.
I20260812 08:01:39.595283 11429 raft_consensus.cc:359] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } }
I20260812 08:01:39.595480 11429 raft_consensus.cc:385] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:01:39.595539 11429 raft_consensus.cc:740] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f0292e14d114be2be61f1b49041540f, State: Initialized, Role: FOLLOWER
I20260812 08:01:39.595639 11428 tablet_bootstrap.cc:654] T 9e91e10f1ecf4319b2b55a3eca65dfde P 0daf92242f824102a4d9d0feaa7604dc: Neither blocks nor log segments found. Creating new log.
I20260812 08:01:39.596401 11428 log.cc:826] T 9e91e10f1ecf4319b2b55a3eca65dfde P 0daf92242f824102a4d9d0feaa7604dc: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:39.597121 11429 consensus_queue.cc:260] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } }
I20260812 08:01:39.597935 11429 ts_tablet_manager.cc:1434] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260812 08:01:39.598253 10941 heartbeater.cc:499] Master 127.0.186.126:39483 was elected leader, sending a full tablet report...
I20260812 08:01:39.598419 11428 tablet_bootstrap.cc:492] T 9e91e10f1ecf4319b2b55a3eca65dfde P 0daf92242f824102a4d9d0feaa7604dc: No bootstrap required, opened a new log
I20260812 08:01:39.598603 11428 ts_tablet_manager.cc:1403] T 9e91e10f1ecf4319b2b55a3eca65dfde P 0daf92242f824102a4d9d0feaa7604dc: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:01:39.601732 11428 raft_consensus.cc:359] T 9e91e10f1ecf4319b2b55a3eca65dfde P 0daf92242f824102a4d9d0feaa7604dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } }
I20260812 08:01:39.601943 11428 raft_consensus.cc:385] T 9e91e10f1ecf4319b2b55a3eca65dfde P 0daf92242f824102a4d9d0feaa7604dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:01:39.601996 11428 raft_consensus.cc:740] T 9e91e10f1ecf4319b2b55a3eca65dfde P 0daf92242f824102a4d9d0feaa7604dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0daf92242f824102a4d9d0feaa7604dc, State: Initialized, Role: FOLLOWER
I20260812 08:01:39.602411 11428 consensus_queue.cc:260] T 9e91e10f1ecf4319b2b55a3eca65dfde P 0daf92242f824102a4d9d0feaa7604dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } }
I20260812 08:01:39.603024 11405 heartbeater.cc:499] Master 127.0.186.126:39483 was elected leader, sending a full tablet report...
I20260812 08:01:39.603271 11428 ts_tablet_manager.cc:1434] T 9e91e10f1ecf4319b2b55a3eca65dfde P 0daf92242f824102a4d9d0feaa7604dc: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
W20260812 08:01:39.636229 10477 tablet.cc:2406] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:01:39.656411 11438 raft_consensus.cc:493] T 9e91e10f1ecf4319b2b55a3eca65dfde P 0daf92242f824102a4d9d0feaa7604dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:01:39.656615 11438 raft_consensus.cc:515] T 9e91e10f1ecf4319b2b55a3eca65dfde P 0daf92242f824102a4d9d0feaa7604dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } }
I20260812 08:01:39.657752 11438 leader_election.cc:290] T 9e91e10f1ecf4319b2b55a3eca65dfde P 0daf92242f824102a4d9d0feaa7604dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ef089e13bd1432e9fa75814933f3bec (127.0.186.65:35839), 5f0292e14d114be2be61f1b49041540f (127.0.186.68:43825)
I20260812 08:01:39.662945 10419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e91e10f1ecf4319b2b55a3eca65dfde" candidate_uuid: "0daf92242f824102a4d9d0feaa7604dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ef089e13bd1432e9fa75814933f3bec" is_pre_election: true
I20260812 08:01:39.662990 10886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e91e10f1ecf4319b2b55a3eca65dfde" candidate_uuid: "0daf92242f824102a4d9d0feaa7604dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f0292e14d114be2be61f1b49041540f" is_pre_election: true
I20260812 08:01:39.663241 10419 raft_consensus.cc:2468] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0daf92242f824102a4d9d0feaa7604dc in term 0.
I20260812 08:01:39.663272 10886 raft_consensus.cc:2468] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0daf92242f824102a4d9d0feaa7604dc in term 0.
I20260812 08:01:39.663693 11269 leader_election.cc:304] T 9e91e10f1ecf4319b2b55a3eca65dfde P 0daf92242f824102a4d9d0feaa7604dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0daf92242f824102a4d9d0feaa7604dc, 5f0292e14d114be2be61f1b49041540f; no voters: 
I20260812 08:01:39.663938 11438 raft_consensus.cc:2804] T 9e91e10f1ecf4319b2b55a3eca65dfde P 0daf92242f824102a4d9d0feaa7604dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:01:39.664026 11438 raft_consensus.cc:493] T 9e91e10f1ecf4319b2b55a3eca65dfde P 0daf92242f824102a4d9d0feaa7604dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:01:39.664069 11438 raft_consensus.cc:3060] T 9e91e10f1ecf4319b2b55a3eca65dfde P 0daf92242f824102a4d9d0feaa7604dc [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:01:39.665231 11438 raft_consensus.cc:515] T 9e91e10f1ecf4319b2b55a3eca65dfde P 0daf92242f824102a4d9d0feaa7604dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } }
I20260812 08:01:39.665699 11438 leader_election.cc:290] T 9e91e10f1ecf4319b2b55a3eca65dfde P 0daf92242f824102a4d9d0feaa7604dc [CANDIDATE]: Term 1 election: Requested vote from peers 2ef089e13bd1432e9fa75814933f3bec (127.0.186.65:35839), 5f0292e14d114be2be61f1b49041540f (127.0.186.68:43825)
I20260812 08:01:39.666149 10419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e91e10f1ecf4319b2b55a3eca65dfde" candidate_uuid: "0daf92242f824102a4d9d0feaa7604dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ef089e13bd1432e9fa75814933f3bec"
I20260812 08:01:39.666143 10886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e91e10f1ecf4319b2b55a3eca65dfde" candidate_uuid: "0daf92242f824102a4d9d0feaa7604dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f0292e14d114be2be61f1b49041540f"
I20260812 08:01:39.666266 10419 raft_consensus.cc:3060] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:01:39.666267 10886 raft_consensus.cc:3060] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:01:39.667126 10419 raft_consensus.cc:2468] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0daf92242f824102a4d9d0feaa7604dc in term 1.
I20260812 08:01:39.667354 10886 raft_consensus.cc:2468] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0daf92242f824102a4d9d0feaa7604dc in term 1.
I20260812 08:01:39.667410 11269 leader_election.cc:304] T 9e91e10f1ecf4319b2b55a3eca65dfde P 0daf92242f824102a4d9d0feaa7604dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0daf92242f824102a4d9d0feaa7604dc, 2ef089e13bd1432e9fa75814933f3bec; no voters: 
I20260812 08:01:39.667577 11438 raft_consensus.cc:2804] T 9e91e10f1ecf4319b2b55a3eca65dfde P 0daf92242f824102a4d9d0feaa7604dc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:01:39.667802 11438 raft_consensus.cc:697] T 9e91e10f1ecf4319b2b55a3eca65dfde P 0daf92242f824102a4d9d0feaa7604dc [term 1 LEADER]: Becoming Leader. State: Replica: 0daf92242f824102a4d9d0feaa7604dc, State: Running, Role: LEADER
I20260812 08:01:39.668123 11438 consensus_queue.cc:237] T 9e91e10f1ecf4319b2b55a3eca65dfde P 0daf92242f824102a4d9d0feaa7604dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } }
I20260812 08:01:39.671630 10251 catalog_manager.cc:5719] T 9e91e10f1ecf4319b2b55a3eca65dfde P 0daf92242f824102a4d9d0feaa7604dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71). New cstate: current_term: 1 leader_uuid: "0daf92242f824102a4d9d0feaa7604dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } health_report { overall_health: UNKNOWN } } }
I20260812 08:01:39.678910   745 external_mini_cluster.cc:955] 7 TS(s) registered with all masters
I20260812 08:01:39.680264   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 2ef089e13bd1432e9fa75814933f3bec to finish bootstrapping
I20260812 08:01:39.686376   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 5f0292e14d114be2be61f1b49041540f to finish bootstrapping
I20260812 08:01:39.691473   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 0daf92242f824102a4d9d0feaa7604dc to finish bootstrapping
I20260812 08:01:39.704860   745 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
I20260812 08:01:39.732614 10886 raft_consensus.cc:1275] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 1 FOLLOWER]: Refusing update from remote peer 0daf92242f824102a4d9d0feaa7604dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:01:39.732802 10419 raft_consensus.cc:1275] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 1 FOLLOWER]: Refusing update from remote peer 0daf92242f824102a4d9d0feaa7604dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:01:39.733662 11438 consensus_queue.cc:1048] T 9e91e10f1ecf4319b2b55a3eca65dfde P 0daf92242f824102a4d9d0feaa7604dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:01:39.733829 11440 consensus_queue.cc:1048] T 9e91e10f1ecf4319b2b55a3eca65dfde P 0daf92242f824102a4d9d0feaa7604dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 08:01:39.735493 10942 tablet.cc:2406] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:01:39.742527 11459 mvcc.cc:204] Tried to move back new op lower bound from 7317592882103492608 to 7317592881843531776. Current Snapshot: MvccSnapshot[applied={T|T < 7317592882097483776}]
I20260812 08:01:39.744658 11469 mvcc.cc:204] Tried to move back new op lower bound from 7317592882106716160 to 7317592881843531776. Current Snapshot: MvccSnapshot[applied={T|T < 7317592882097483776}]
I20260812 08:01:39.761276 11470 mvcc.cc:204] Tried to move back new op lower bound from 7317592882109927424 to 7317592881843531776. Current Snapshot: MvccSnapshot[applied={T|T < 7317592882097483776}]
I20260812 08:01:39.770556   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 11251
I20260812 08:01:39.813555 10785 heartbeater.cc:499] Master 127.0.186.126:39483 was elected leader, sending a full tablet report...
I20260812 08:01:40.202122 11097 heartbeater.cc:499] Master 127.0.186.126:39483 was elected leader, sending a full tablet report...
I20260812 08:01:40.375097 11248 heartbeater.cc:499] Master 127.0.186.126:39483 was elected leader, sending a full tablet report...
I20260812 08:01:41.267432 11492 raft_consensus.cc:493] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0daf92242f824102a4d9d0feaa7604dc)
I20260812 08:01:41.267577 11492 raft_consensus.cc:515] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } }
I20260812 08:01:41.267788 11493 raft_consensus.cc:493] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0daf92242f824102a4d9d0feaa7604dc)
I20260812 08:01:41.267906 11493 raft_consensus.cc:515] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } }
I20260812 08:01:41.268687 11492 leader_election.cc:290] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71:38427), 5f0292e14d114be2be61f1b49041540f (127.0.186.68:43825)
I20260812 08:01:41.268915 11493 leader_election.cc:290] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ef089e13bd1432e9fa75814933f3bec (127.0.186.65:35839), 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71:38427)
W20260812 08:01:41.269521 10342 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.71:38427: connect: Connection refused (error 111)
W20260812 08:01:41.269928 10805 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.71:38427: connect: Connection refused (error 111)
W20260812 08:01:41.270896 10342 leader_election.cc:336] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71:38427): Network error: Client connection negotiation failed: client connection to 127.0.186.71:38427: connect: Connection refused (error 111)
W20260812 08:01:41.270990 10805 leader_election.cc:336] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71:38427): Network error: Client connection negotiation failed: client connection to 127.0.186.71:38427: connect: Connection refused (error 111)
I20260812 08:01:41.273133 10419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e91e10f1ecf4319b2b55a3eca65dfde" candidate_uuid: "5f0292e14d114be2be61f1b49041540f" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "2ef089e13bd1432e9fa75814933f3bec" is_pre_election: true
I20260812 08:01:41.273283 10419 raft_consensus.cc:2468] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f0292e14d114be2be61f1b49041540f in term 1.
I20260812 08:01:41.273393 10886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e91e10f1ecf4319b2b55a3eca65dfde" candidate_uuid: "2ef089e13bd1432e9fa75814933f3bec" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "5f0292e14d114be2be61f1b49041540f" is_pre_election: true
I20260812 08:01:41.273548 10886 raft_consensus.cc:2468] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ef089e13bd1432e9fa75814933f3bec in term 1.
I20260812 08:01:41.273671 10805 leader_election.cc:304] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ef089e13bd1432e9fa75814933f3bec, 5f0292e14d114be2be61f1b49041540f; no voters: 0daf92242f824102a4d9d0feaa7604dc
I20260812 08:01:41.273846 10342 leader_election.cc:304] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ef089e13bd1432e9fa75814933f3bec, 5f0292e14d114be2be61f1b49041540f; no voters: 0daf92242f824102a4d9d0feaa7604dc
I20260812 08:01:41.273891 11493 raft_consensus.cc:2804] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 08:01:41.273945 11493 raft_consensus.cc:493] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0daf92242f824102a4d9d0feaa7604dc)
I20260812 08:01:41.273991 11493 raft_consensus.cc:3060] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:01:41.274057 11492 raft_consensus.cc:2804] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 08:01:41.274174 11492 raft_consensus.cc:493] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0daf92242f824102a4d9d0feaa7604dc)
I20260812 08:01:41.274246 11492 raft_consensus.cc:3060] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:01:41.275120 11493 raft_consensus.cc:515] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } }
I20260812 08:01:41.275120 11492 raft_consensus.cc:515] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } }
I20260812 08:01:41.275480 11493 leader_election.cc:290] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [CANDIDATE]: Term 2 election: Requested vote from peers 2ef089e13bd1432e9fa75814933f3bec (127.0.186.65:35839), 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71:38427)
I20260812 08:01:41.275480 11492 leader_election.cc:290] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [CANDIDATE]: Term 2 election: Requested vote from peers 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71:38427), 5f0292e14d114be2be61f1b49041540f (127.0.186.68:43825)
I20260812 08:01:41.276042 10886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e91e10f1ecf4319b2b55a3eca65dfde" candidate_uuid: "2ef089e13bd1432e9fa75814933f3bec" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "5f0292e14d114be2be61f1b49041540f"
I20260812 08:01:41.276038 10419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e91e10f1ecf4319b2b55a3eca65dfde" candidate_uuid: "5f0292e14d114be2be61f1b49041540f" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "2ef089e13bd1432e9fa75814933f3bec"
I20260812 08:01:41.276199 10886 raft_consensus.cc:2393] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 2ef089e13bd1432e9fa75814933f3bec in current term 2: Already voted for candidate 5f0292e14d114be2be61f1b49041540f in this term.
I20260812 08:01:41.276198 10419 raft_consensus.cc:2393] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5f0292e14d114be2be61f1b49041540f in current term 2: Already voted for candidate 2ef089e13bd1432e9fa75814933f3bec in this term.
W20260812 08:01:41.276543 10342 leader_election.cc:336] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71:38427): Network error: Client connection negotiation failed: client connection to 127.0.186.71:38427: connect: Connection refused (error 111)
W20260812 08:01:41.276638 10805 leader_election.cc:336] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71:38427): Network error: Client connection negotiation failed: client connection to 127.0.186.71:38427: connect: Connection refused (error 111)
I20260812 08:01:41.276703 10805 leader_election.cc:304] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5f0292e14d114be2be61f1b49041540f; no voters: 0daf92242f824102a4d9d0feaa7604dc, 2ef089e13bd1432e9fa75814933f3bec
I20260812 08:01:41.276746 10342 leader_election.cc:304] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ef089e13bd1432e9fa75814933f3bec; no voters: 0daf92242f824102a4d9d0feaa7604dc, 5f0292e14d114be2be61f1b49041540f
I20260812 08:01:41.276839 11493 raft_consensus.cc:2749] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 08:01:41.276989 11492 raft_consensus.cc:2749] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 08:01:42.894816 11503 raft_consensus.cc:493] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:01:42.894951 11503 raft_consensus.cc:515] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } }
I20260812 08:01:42.895407 11503 leader_election.cc:290] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2ef089e13bd1432e9fa75814933f3bec (127.0.186.65:35839), 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71:38427)
I20260812 08:01:42.895903 10419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e91e10f1ecf4319b2b55a3eca65dfde" candidate_uuid: "5f0292e14d114be2be61f1b49041540f" candidate_term: 3 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "2ef089e13bd1432e9fa75814933f3bec" is_pre_election: true
I20260812 08:01:42.896078 10419 raft_consensus.cc:2468] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f0292e14d114be2be61f1b49041540f in term 2.
I20260812 08:01:42.896404 10805 leader_election.cc:304] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ef089e13bd1432e9fa75814933f3bec, 5f0292e14d114be2be61f1b49041540f; no voters: 
I20260812 08:01:42.896574 11503 raft_consensus.cc:2804] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 08:01:42.896631 11503 raft_consensus.cc:493] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:01:42.896657 11503 raft_consensus.cc:3060] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 2 FOLLOWER]: Advancing to term 3
W20260812 08:01:42.896911 10805 leader_election.cc:336] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71:38427): Network error: Client connection negotiation failed: client connection to 127.0.186.71:38427: connect: Connection refused (error 111)
I20260812 08:01:42.897615 11503 raft_consensus.cc:515] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } }
I20260812 08:01:42.897929 11503 leader_election.cc:290] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [CANDIDATE]: Term 3 election: Requested vote from peers 2ef089e13bd1432e9fa75814933f3bec (127.0.186.65:35839), 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71:38427)
I20260812 08:01:42.898486 10419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e91e10f1ecf4319b2b55a3eca65dfde" candidate_uuid: "5f0292e14d114be2be61f1b49041540f" candidate_term: 3 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "2ef089e13bd1432e9fa75814933f3bec"
I20260812 08:01:42.898612 10419 raft_consensus.cc:3060] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 2 FOLLOWER]: Advancing to term 3
W20260812 08:01:42.898958 10805 leader_election.cc:336] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71:38427): Network error: Client connection negotiation failed: client connection to 127.0.186.71:38427: connect: Connection refused (error 111)
I20260812 08:01:42.899498 10419 raft_consensus.cc:2468] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f0292e14d114be2be61f1b49041540f in term 3.
I20260812 08:01:42.899757 10805 leader_election.cc:304] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ef089e13bd1432e9fa75814933f3bec, 5f0292e14d114be2be61f1b49041540f; no voters: 0daf92242f824102a4d9d0feaa7604dc
I20260812 08:01:42.899928 11503 raft_consensus.cc:2804] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 3 FOLLOWER]: Leader election won for term 3
I20260812 08:01:42.900084 11503 raft_consensus.cc:697] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 3 LEADER]: Becoming Leader. State: Replica: 5f0292e14d114be2be61f1b49041540f, State: Running, Role: LEADER
I20260812 08:01:42.900386 11503 consensus_queue.cc:237] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } }
I20260812 08:01:42.902549 10245 catalog_manager.cc:5719] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f reported cstate change: term changed from 1 to 3, leader changed from 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71) to 5f0292e14d114be2be61f1b49041540f (127.0.186.68). New cstate: current_term: 3 leader_uuid: "5f0292e14d114be2be61f1b49041540f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } health_report { overall_health: HEALTHY } } }
W20260812 08:01:43.432998 10805 consensus_peers.cc:597] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f -> Peer 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71:38427): Couldn't send request to peer 0daf92242f824102a4d9d0feaa7604dc. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.71:38427: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 08:01:43.464006 10419 raft_consensus.cc:1275] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 3 FOLLOWER]: Refusing update from remote peer 5f0292e14d114be2be61f1b49041540f: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 3 index: 10. (index mismatch)
I20260812 08:01:43.464569 11512 consensus_queue.cc:1048] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260812 08:01:45.079268 11512 consensus_queue.cc:579] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [LEADER]: Leader has been unable to successfully communicate with peer 0daf92242f824102a4d9d0feaa7604dc for more than 2 seconds (2.178s)
I20260812 08:01:45.082834 10886 consensus_queue.cc:237] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 3.10, Last appended by leader: 9, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } } peers { permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: NON_VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: true } }
I20260812 08:01:45.084589 10419 raft_consensus.cc:1275] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 3 FOLLOWER]: Refusing update from remote peer 5f0292e14d114be2be61f1b49041540f: Log matching property violated. Preceding OpId in replica: term: 3 index: 10. Preceding OpId from leader: term: 3 index: 11. (index mismatch)
I20260812 08:01:45.085003 11512 consensus_queue.cc:1048] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260812 08:01:45.085633 10805 consensus_peers.cc:597] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f -> Peer 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71:38427): Couldn't send request to peer 0daf92242f824102a4d9d0feaa7604dc. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.71:38427: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 08:01:45.088200 11522 raft_consensus.cc:2955] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 3 LEADER]: Committing config change with OpId 3.11: config changed from index -1 to 11, NON_VOTER df29c7b34b2e4b8d9eddef7c5f166e06 (127.0.186.70) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } } peers { permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: NON_VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: true } } }
I20260812 08:01:45.088663 10419 raft_consensus.cc:2955] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 3 FOLLOWER]: Committing config change with OpId 3.11: config changed from index -1 to 11, NON_VOTER df29c7b34b2e4b8d9eddef7c5f166e06 (127.0.186.70) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } } peers { permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: NON_VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: true } } }
W20260812 08:01:45.089759 10805 consensus_peers.cc:597] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f -> Peer df29c7b34b2e4b8d9eddef7c5f166e06 (127.0.186.70:45261): Couldn't send request to peer df29c7b34b2e4b8d9eddef7c5f166e06. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9e91e10f1ecf4319b2b55a3eca65dfde. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:01:45.089977 10233 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 9e91e10f1ecf4319b2b55a3eca65dfde with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:01:45.091210 10245 catalog_manager.cc:5719] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f reported cstate change: config changed from index -1 to 11, NON_VOTER df29c7b34b2e4b8d9eddef7c5f166e06 (127.0.186.70) added. New cstate: current_term: 3 leader_uuid: "5f0292e14d114be2be61f1b49041540f" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: NON_VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260812 08:01:45.108135   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 10323
I20260812 08:01:45.465472 11539 ts_tablet_manager.cc:933] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06: Initiating tablet copy from peer 5f0292e14d114be2be61f1b49041540f (127.0.186.68:43825)
I20260812 08:01:45.466074 11539 tablet_copy_client.cc:323] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06: tablet copy: Beginning tablet copy session from remote peer at address 127.0.186.68:43825
I20260812 08:01:45.469795 10914 tablet_copy_service.cc:140] P 5f0292e14d114be2be61f1b49041540f: Received BeginTabletCopySession request for tablet 9e91e10f1ecf4319b2b55a3eca65dfde from peer df29c7b34b2e4b8d9eddef7c5f166e06 ({username='slave'} at 127.0.186.70:58725)
I20260812 08:01:45.469933 10914 tablet_copy_service.cc:161] P 5f0292e14d114be2be61f1b49041540f: Beginning new tablet copy session on tablet 9e91e10f1ecf4319b2b55a3eca65dfde from peer df29c7b34b2e4b8d9eddef7c5f166e06 at {username='slave'} at 127.0.186.70:58725: session id = df29c7b34b2e4b8d9eddef7c5f166e06-9e91e10f1ecf4319b2b55a3eca65dfde
I20260812 08:01:45.471143 10914 tablet_copy_source_session.cc:215] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 08:01:45.474532 11539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e91e10f1ecf4319b2b55a3eca65dfde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:01:45.479032 11539 tablet_copy_client.cc:806] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06: tablet copy: Starting download of 0 data blocks...
I20260812 08:01:45.479210 11539 tablet_copy_client.cc:670] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06: tablet copy: Starting download of 1 WAL segments...
I20260812 08:01:45.480845 11540 tablet_copy_client.cc:1025] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06: tablet copy: Injecting latency into file download: 10000
W20260812 08:01:45.568259 10805 consensus_peers.cc:597] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f -> Peer 2ef089e13bd1432e9fa75814933f3bec (127.0.186.65:35839): Couldn't send request to peer 2ef089e13bd1432e9fa75814933f3bec. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:35839: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260812 08:01:46.450659 10805 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.65:35839: connect: Connection refused (error 111) [suppressed 13 similar messages]
I20260812 08:01:47.222837 11544 consensus_queue.cc:579] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [LEADER]: Leader has been unable to successfully communicate with peer 0daf92242f824102a4d9d0feaa7604dc for more than 2 seconds (2.140s)
I20260812 08:01:47.587556 11547 consensus_queue.cc:579] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [LEADER]: Leader has been unable to successfully communicate with peer 2ef089e13bd1432e9fa75814933f3bec for more than 2 seconds (2.497s)
W20260812 08:01:47.635228 10805 consensus_peers.cc:597] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f -> Peer 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71:38427): Couldn't send request to peer 0daf92242f824102a4d9d0feaa7604dc. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.71:38427: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260812 08:01:48.050800 10805 consensus_peers.cc:597] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f -> Peer 2ef089e13bd1432e9fa75814933f3bec (127.0.186.65:35839): Couldn't send request to peer 2ef089e13bd1432e9fa75814933f3bec. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:35839: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260812 08:01:48.064248 10805 consensus_peers.cc:597] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f -> Peer df29c7b34b2e4b8d9eddef7c5f166e06 (127.0.186.70:45261): Couldn't send request to peer df29c7b34b2e4b8d9eddef7c5f166e06. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260812 08:01:50.027607 10805 consensus_peers.cc:597] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f -> Peer 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71:38427): Couldn't send request to peer 0daf92242f824102a4d9d0feaa7604dc. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.71:38427: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260812 08:01:50.464990 10805 consensus_peers.cc:597] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f -> Peer 2ef089e13bd1432e9fa75814933f3bec (127.0.186.65:35839): Couldn't send request to peer 2ef089e13bd1432e9fa75814933f3bec. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:35839: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260812 08:01:50.677790 10805 consensus_peers.cc:597] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f -> Peer df29c7b34b2e4b8d9eddef7c5f166e06 (127.0.186.70:45261): Couldn't send request to peer df29c7b34b2e4b8d9eddef7c5f166e06. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260812 08:01:51.484354 10805 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.71:38427: connect: Connection refused (error 111) [suppressed 20 similar messages]
W20260812 08:01:52.499570 10805 consensus_peers.cc:597] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f -> Peer 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71:38427): Couldn't send request to peer 0daf92242f824102a4d9d0feaa7604dc. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.71:38427: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260812 08:01:52.734318 10805 consensus_peers.cc:597] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f -> Peer 2ef089e13bd1432e9fa75814933f3bec (127.0.186.65:35839): Couldn't send request to peer 2ef089e13bd1432e9fa75814933f3bec. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:35839: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260812 08:01:53.292253 10805 consensus_peers.cc:597] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f -> Peer df29c7b34b2e4b8d9eddef7c5f166e06 (127.0.186.70:45261): Couldn't send request to peer df29c7b34b2e4b8d9eddef7c5f166e06. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
W20260812 08:01:54.952170 10805 consensus_peers.cc:597] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f -> Peer 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71:38427): Couldn't send request to peer 0daf92242f824102a4d9d0feaa7604dc. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.71:38427: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260812 08:01:55.164323 10805 consensus_peers.cc:597] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f -> Peer 2ef089e13bd1432e9fa75814933f3bec (127.0.186.65:35839): Couldn't send request to peer 2ef089e13bd1432e9fa75814933f3bec. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:35839: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260812 08:01:55.448922 10805 consensus_peers.cc:597] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f -> Peer df29c7b34b2e4b8d9eddef7c5f166e06 (127.0.186.70:45261): Couldn't send request to peer df29c7b34b2e4b8d9eddef7c5f166e06. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260812 08:01:55.481406 11539 tablet_copy_client.cc:538] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 08:01:55.483563 11539 tablet_bootstrap.cc:492] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06: Bootstrap starting.
I20260812 08:01:55.489420 11539 log.cc:826] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:55.506220 11539 tablet_bootstrap.cc:492] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=400 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:01:55.506613 11539 tablet_bootstrap.cc:492] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06: Bootstrap complete.
I20260812 08:01:55.507011 11539 ts_tablet_manager.cc:1403] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06: Time spent bootstrapping tablet: real 0.024s	user 0.021s	sys 0.002s
I20260812 08:01:55.509083 11539 raft_consensus.cc:359] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } } peers { permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: NON_VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: true } }
I20260812 08:01:55.509312 11539 raft_consensus.cc:740] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: df29c7b34b2e4b8d9eddef7c5f166e06, State: Initialized, Role: LEARNER
I20260812 08:01:55.509660 11539 consensus_queue.cc:260] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 3.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } } peers { permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: NON_VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: true } }
I20260812 08:01:55.510439 11539 ts_tablet_manager.cc:1434] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06: Time spent starting tablet: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:01:55.511268 10914 tablet_copy_service.cc:342] P 5f0292e14d114be2be61f1b49041540f: Request end of tablet copy session df29c7b34b2e4b8d9eddef7c5f166e06-9e91e10f1ecf4319b2b55a3eca65dfde received from {username='slave'} at 127.0.186.70:58725
I20260812 08:01:55.511391 10914 tablet_copy_service.cc:434] P 5f0292e14d114be2be61f1b49041540f: ending tablet copy session df29c7b34b2e4b8d9eddef7c5f166e06-9e91e10f1ecf4319b2b55a3eca65dfde on tablet 9e91e10f1ecf4319b2b55a3eca65dfde with peer df29c7b34b2e4b8d9eddef7c5f166e06
I20260812 08:01:55.518136   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 10481
I20260812 08:01:55.530591   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 10635
I20260812 08:01:55.539155   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 10788
I20260812 08:01:55.549521   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 10945
I20260812 08:01:55.557355   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 11100
I20260812 08:01:55.566849   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 10217
I20260812 08:01:55.577724   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:35839
--local_ip_for_outbound_sockets=127.0.186.65
--tserver_master_addrs=127.0.186.126:39483
--webserver_port=44845
--webserver_interface=127.0.186.65
--builtin_ntp_servers=127.0.186.84:46389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=10000
--follower_unavailable_considered_failed_sec=2
--consensus_rpc_timeout_ms=5000 with env {}
W20260812 08:01:55.683140 11557 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:55.683378 11557 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:55.683439 11557 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:55.686753 11557 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:55.686857 11557 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:01:55.690544 11557 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:46389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=2
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:35839
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=44845
--tablet_copy_download_file_inject_latency_ms=10000
--tserver_master_addrs=127.0.186.126:39483
--never_fsync=true
--heap_profile_path=/tmp/kudu.11557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:55.691540 11557 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:55.692624 11557 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:55.699070 11566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:55.699070 11563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:55.699144 11557 server_base.cc:1061] running on GCE node
W20260812 08:01:55.699178 11564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:55.699692 11557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:55.700173 11557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:55.701359 11557 hybrid_clock.cc:648] HybridClock initialized: now 1786521715701332 us; error 38 us; skew 500 ppm
I20260812 08:01:55.703052 11557 webserver.cc:533] Webserver started at http://127.0.186.65:44845/ using document root <none> and password file <none>
I20260812 08:01:55.703574 11557 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:55.703639 11557 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:55.706869 11557 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:01:55.708626 11572 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:55.709573 11557 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:55.709681 11557 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "2ef089e13bd1432e9fa75814933f3bec"
format_stamp: "Formatted at 2026-08-12 08:01:38 on dist-test-slave-49h3"
I20260812 08:01:55.710033 11557 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:55.729171 11557 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:55.729913 11557 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:55.730074 11557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:55.730636 11557 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:55.731863 11580 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:01:55.735219 11557 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:01:55.735281 11557 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:01:55.735338 11557 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:01:55.736851 11557 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:01:55.736894 11557 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:01:55.737229 11580 tablet_bootstrap.cc:492] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec: Bootstrap starting.
I20260812 08:01:55.747710 11557 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:35839
I20260812 08:01:55.747738 11708 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:35839 every 8 connection(s)
I20260812 08:01:55.748777 11557 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
W20260812 08:01:55.751832 11709 heartbeater.cc:646] Failed to heartbeat to 127.0.186.126:39483 (0 consecutive failures): Network error: Failed to ping master at 127.0.186.126:39483: Client connection negotiation failed: client connection to 127.0.186.126:39483: connect: Connection refused (error 111)
I20260812 08:01:55.752729 11580 log.cc:826] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec: Log is configured to *not* fsync() on all Append() calls
W20260812 08:01:55.753145 11709 heartbeater.cc:412] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 08:01:55.753541   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 11557
I20260812 08:01:55.754047   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.66:43103
--local_ip_for_outbound_sockets=127.0.186.66
--tserver_master_addrs=127.0.186.126:39483
--webserver_port=36465
--webserver_interface=127.0.186.66
--builtin_ntp_servers=127.0.186.84:46389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=10000
--follower_unavailable_considered_failed_sec=2
--consensus_rpc_timeout_ms=5000 with env {}
I20260812 08:01:55.776062 11580 tablet_bootstrap.cc:492] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=400 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:01:55.776597 11580 tablet_bootstrap.cc:492] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec: Bootstrap complete.
I20260812 08:01:55.778053 11580 ts_tablet_manager.cc:1403] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec: Time spent bootstrapping tablet: real 0.041s	user 0.037s	sys 0.004s
I20260812 08:01:55.781030 11580 raft_consensus.cc:359] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } } peers { permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: NON_VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: true } }
I20260812 08:01:55.781471 11580 raft_consensus.cc:740] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ef089e13bd1432e9fa75814933f3bec, State: Initialized, Role: FOLLOWER
I20260812 08:01:55.781973 11580 consensus_queue.cc:260] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 3.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } } peers { permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: NON_VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: true } }
I20260812 08:01:55.782866 11580 ts_tablet_manager.cc:1434] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.004s
W20260812 08:01:55.869084 11714 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:55.869302 11714 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:55.869355 11714 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:55.872654 11714 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:55.872779 11714 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.66
I20260812 08:01:55.876838 11714 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:46389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=2
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.66:43103
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.186.66
--webserver_port=36465
--tablet_copy_download_file_inject_latency_ms=10000
--tserver_master_addrs=127.0.186.126:39483
--never_fsync=true
--heap_profile_path=/tmp/kudu.11714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:55.878032 11714 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:55.879186 11714 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:55.885739 11722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:55.885830 11723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:55.885802 11714 server_base.cc:1061] running on GCE node
W20260812 08:01:55.885756 11725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:55.886297 11714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:55.886842 11714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:55.887987 11714 hybrid_clock.cc:648] HybridClock initialized: now 1786521715887971 us; error 31 us; skew 500 ppm
I20260812 08:01:55.889762 11714 webserver.cc:533] Webserver started at http://127.0.186.66:36465/ using document root <none> and password file <none>
I20260812 08:01:55.890316 11714 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:55.890389 11714 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:55.893666 11714 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.000s	sys 0.003s
I20260812 08:01:55.895445 11732 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:55.896410 11714 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:55.896507 11714 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "c5c99765f674430db87f0bd401e7a193"
format_stamp: "Formatted at 2026-08-12 08:01:38 on dist-test-slave-49h3"
I20260812 08:01:55.896890 11714 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:55.915498 11714 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:55.916177 11714 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:55.916358 11714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:55.916951 11714 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:55.918038 11714 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:01:55.918088 11714 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:55.918131 11714 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:01:55.918159 11714 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:55.926854 11714 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.66:43103
I20260812 08:01:55.926882 11870 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.66:43103 every 8 connection(s)
I20260812 08:01:55.927850 11714 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
W20260812 08:01:55.930948 11871 heartbeater.cc:646] Failed to heartbeat to 127.0.186.126:39483 (0 consecutive failures): Network error: Failed to ping master at 127.0.186.126:39483: Client connection negotiation failed: client connection to 127.0.186.126:39483: connect: Connection refused (error 111)
W20260812 08:01:55.932094 11871 heartbeater.cc:412] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 08:01:55.932129   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 11714
I20260812 08:01:55.932552   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.67:36027
--local_ip_for_outbound_sockets=127.0.186.67
--tserver_master_addrs=127.0.186.126:39483
--webserver_port=36731
--webserver_interface=127.0.186.67
--builtin_ntp_servers=127.0.186.84:46389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=10000
--follower_unavailable_considered_failed_sec=2
--consensus_rpc_timeout_ms=5000 with env {}
W20260812 08:01:56.044004 11875 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:56.044214 11875 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:56.044277 11875 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:56.047600 11875 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:56.047731 11875 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.67
I20260812 08:01:56.051501 11875 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:46389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=2
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.67:36027
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.186.67
--webserver_port=36731
--tablet_copy_download_file_inject_latency_ms=10000
--tserver_master_addrs=127.0.186.126:39483
--never_fsync=true
--heap_profile_path=/tmp/kudu.11875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:56.052560 11875 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:56.053730 11875 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:56.060163 11883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:56.060176 11884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:56.060240 11887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:56.060388 11875 server_base.cc:1061] running on GCE node
I20260812 08:01:56.060787 11875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:56.061352 11875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:56.062501 11875 hybrid_clock.cc:648] HybridClock initialized: now 1786521716062486 us; error 32 us; skew 500 ppm
I20260812 08:01:56.064311 11875 webserver.cc:533] Webserver started at http://127.0.186.67:36731/ using document root <none> and password file <none>
I20260812 08:01:56.064891 11875 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:56.064972 11875 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:56.068398 11875 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:56.070283 11894 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:56.071213 11875 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:56.071318 11875 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "0fc4212fb0854d5d8b405156a6f8c4bc"
format_stamp: "Formatted at 2026-08-12 08:01:38 on dist-test-slave-49h3"
I20260812 08:01:56.071674 11875 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:56.100494 11875 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:56.101184 11875 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:56.101367 11875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:56.101979 11875 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:56.102995 11875 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:01:56.103045 11875 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:56.103098 11875 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:01:56.103125 11875 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:56.111590 11875 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.67:36027
I20260812 08:01:56.111655 12024 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.67:36027 every 8 connection(s)
I20260812 08:01:56.112582 11875 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
W20260812 08:01:56.115617 12025 heartbeater.cc:646] Failed to heartbeat to 127.0.186.126:39483 (0 consecutive failures): Network error: Failed to ping master at 127.0.186.126:39483: Client connection negotiation failed: client connection to 127.0.186.126:39483: connect: Connection refused (error 111)
W20260812 08:01:56.116720 12025 heartbeater.cc:412] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 08:01:56.120071   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 11875
I20260812 08:01:56.120508   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.68:43825
--local_ip_for_outbound_sockets=127.0.186.68
--tserver_master_addrs=127.0.186.126:39483
--webserver_port=38503
--webserver_interface=127.0.186.68
--builtin_ntp_servers=127.0.186.84:46389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=10000
--follower_unavailable_considered_failed_sec=2
--consensus_rpc_timeout_ms=5000 with env {}
W20260812 08:01:56.226979 12028 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:56.227186 12028 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:56.227250 12028 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:56.230518 12028 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:56.230659 12028 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.68
I20260812 08:01:56.234445 12028 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:46389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=2
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.68:43825
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.0.186.68
--webserver_port=38503
--tablet_copy_download_file_inject_latency_ms=10000
--tserver_master_addrs=127.0.186.126:39483
--never_fsync=true
--heap_profile_path=/tmp/kudu.12028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.68

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:56.235541 12028 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:56.236760 12028 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:56.242908 12038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:56.242921 12035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:56.242921 12036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:56.243249 12028 server_base.cc:1061] running on GCE node
I20260812 08:01:56.243604 12028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:56.244117 12028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:56.245273 12028 hybrid_clock.cc:648] HybridClock initialized: now 1786521716245257 us; error 29 us; skew 500 ppm
I20260812 08:01:56.247090 12028 webserver.cc:533] Webserver started at http://127.0.186.68:38503/ using document root <none> and password file <none>
I20260812 08:01:56.247641 12028 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:56.247710 12028 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:56.251008 12028 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:01:56.252787 12044 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:56.253737 12028 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:56.253834 12028 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "5f0292e14d114be2be61f1b49041540f"
format_stamp: "Formatted at 2026-08-12 08:01:38 on dist-test-slave-49h3"
I20260812 08:01:56.254220 12028 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:56.274400 12028 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:56.275082 12028 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:56.275257 12028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:56.275833 12028 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:56.277158 12053 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:01:56.280611 12028 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:01:56.280683 12028 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:01:56.280751 12028 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:01:56.282318 12028 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:01:56.282364 12028 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:01:56.282748 12053 tablet_bootstrap.cc:492] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f: Bootstrap starting.
I20260812 08:01:56.292845 12028 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.68:43825
I20260812 08:01:56.292896 12177 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.68:43825 every 8 connection(s)
I20260812 08:01:56.293857 12028 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
W20260812 08:01:56.297037 12180 heartbeater.cc:646] Failed to heartbeat to 127.0.186.126:39483 (0 consecutive failures): Network error: Failed to ping master at 127.0.186.126:39483: Client connection negotiation failed: client connection to 127.0.186.126:39483: connect: Connection refused (error 111)
I20260812 08:01:56.297425   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 12028
I20260812 08:01:56.297955   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.69:46205
--local_ip_for_outbound_sockets=127.0.186.69
--tserver_master_addrs=127.0.186.126:39483
--webserver_port=40787
--webserver_interface=127.0.186.69
--builtin_ntp_servers=127.0.186.84:46389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=10000
--follower_unavailable_considered_failed_sec=2
--consensus_rpc_timeout_ms=5000 with env {}
W20260812 08:01:56.298411 12180 heartbeater.cc:412] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 08:01:56.298727 12053 log.cc:826] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:56.323040 12053 tablet_bootstrap.cc:492] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=400 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:01:56.323531 12053 tablet_bootstrap.cc:492] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f: Bootstrap complete.
I20260812 08:01:56.324790 12053 ts_tablet_manager.cc:1403] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f: Time spent bootstrapping tablet: real 0.042s	user 0.027s	sys 0.012s
I20260812 08:01:56.327898 12053 raft_consensus.cc:359] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } } peers { permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: NON_VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: true } }
I20260812 08:01:56.328337 12053 raft_consensus.cc:740] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f0292e14d114be2be61f1b49041540f, State: Initialized, Role: FOLLOWER
I20260812 08:01:56.328852 12053 consensus_queue.cc:260] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 3.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } } peers { permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: NON_VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: true } }
I20260812 08:01:56.329843 12053 ts_tablet_manager.cc:1434] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
W20260812 08:01:56.412853 12184 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:56.413076 12184 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:56.413125 12184 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:56.416476 12184 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:56.416607 12184 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.69
I20260812 08:01:56.420708 12184 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:46389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=2
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.69:46205
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.0.186.69
--webserver_port=40787
--tablet_copy_download_file_inject_latency_ms=10000
--tserver_master_addrs=127.0.186.126:39483
--never_fsync=true
--heap_profile_path=/tmp/kudu.12184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.69

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:56.421885 12184 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:56.423058 12184 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:56.429733 12193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:56.429873 12184 server_base.cc:1061] running on GCE node
W20260812 08:01:56.430014 12192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:56.430524 12195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:56.430917 12184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:56.431463 12184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:56.432637 12184 hybrid_clock.cc:648] HybridClock initialized: now 1786521716432617 us; error 28 us; skew 500 ppm
I20260812 08:01:56.434451 12184 webserver.cc:533] Webserver started at http://127.0.186.69:40787/ using document root <none> and password file <none>
I20260812 08:01:56.435000 12184 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:56.435081 12184 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:56.438489 12184 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.000s	sys 0.003s
I20260812 08:01:56.440331 12203 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:56.441321 12184 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.001s
I20260812 08:01:56.441426 12184 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "b8e3b126cd934c6e85b1c5f9442ac7f5"
format_stamp: "Formatted at 2026-08-12 08:01:39 on dist-test-slave-49h3"
I20260812 08:01:56.441852 12184 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:56.452210 12184 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:56.452781 12184 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:56.452943 12184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:56.453507 12184 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:56.454491 12184 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:01:56.454542 12184 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:56.454592 12184 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:01:56.454620 12184 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:56.463944 12184 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.69:46205
I20260812 08:01:56.463971 12334 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.69:46205 every 8 connection(s)
I20260812 08:01:56.465032 12184 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-4/data/info.pb
W20260812 08:01:56.468273 12336 heartbeater.cc:646] Failed to heartbeat to 127.0.186.126:39483 (0 consecutive failures): Network error: Failed to ping master at 127.0.186.126:39483: Client connection negotiation failed: client connection to 127.0.186.126:39483: connect: Connection refused (error 111)
W20260812 08:01:56.469339 12336 heartbeater.cc:412] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 08:01:56.475231   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 12184
I20260812 08:01:56.475630   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.70:45261
--local_ip_for_outbound_sockets=127.0.186.70
--tserver_master_addrs=127.0.186.126:39483
--webserver_port=33971
--webserver_interface=127.0.186.70
--builtin_ntp_servers=127.0.186.84:46389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=10000
--follower_unavailable_considered_failed_sec=2
--consensus_rpc_timeout_ms=5000 with env {}
W20260812 08:01:56.583491 12340 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:56.583734 12340 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:56.583781 12340 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:56.587260 12340 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:56.587401 12340 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.70
I20260812 08:01:56.591568 12340 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:46389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=5000
--follower_unavailable_considered_failed_sec=2
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.70:45261
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.0.186.70
--webserver_port=33971
--tablet_copy_download_file_inject_latency_ms=10000
--tserver_master_addrs=127.0.186.126:39483
--never_fsync=true
--heap_profile_path=/tmp/kudu.12340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.70

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:56.592763 12340 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:56.594007 12340 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:56.600526 12345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:56.600526 12346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:56.600740 12350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:56.600751 12340 server_base.cc:1061] running on GCE node
I20260812 08:01:56.601161 12340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:56.601727 12340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:56.602866 12340 hybrid_clock.cc:648] HybridClock initialized: now 1786521716602853 us; error 28 us; skew 500 ppm
I20260812 08:01:56.604533 12340 webserver.cc:533] Webserver started at http://127.0.186.70:33971/ using document root <none> and password file <none>
I20260812 08:01:56.605100 12340 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:56.605180 12340 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:56.608541 12340 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:56.610440 12357 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:56.611370 12340 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:56.611472 12340 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "df29c7b34b2e4b8d9eddef7c5f166e06"
format_stamp: "Formatted at 2026-08-12 08:01:39 on dist-test-slave-49h3"
I20260812 08:01:56.611831 12340 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:56.629993 12340 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:56.630647 12340 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:56.630807 12340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:56.631374 12340 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:56.632581 12364 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:01:56.635890 12340 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:01:56.635957 12340 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:01:56.636019 12340 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:01:56.637566 12340 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:01:56.637610 12340 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:56.637925 12364 tablet_bootstrap.cc:492] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06: Bootstrap starting.
I20260812 08:01:56.647676 12340 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.70:45261
I20260812 08:01:56.647707 12493 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.70:45261 every 8 connection(s)
I20260812 08:01:56.648725 12340 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/ts-5/data/info.pb
W20260812 08:01:56.651777 12494 heartbeater.cc:646] Failed to heartbeat to 127.0.186.126:39483 (0 consecutive failures): Network error: Failed to ping master at 127.0.186.126:39483: Client connection negotiation failed: client connection to 127.0.186.126:39483: connect: Connection refused (error 111)
I20260812 08:01:56.652403   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 12340
I20260812 08:01:56.652838   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:39483
--webserver_interface=127.0.186.126
--webserver_port=37805
--builtin_ntp_servers=127.0.186.84:46389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:39483
--raft_prepare_replacement_before_eviction=true with env {}
I20260812 08:01:56.654088 12364 log.cc:826] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06: Log is configured to *not* fsync() on all Append() calls
W20260812 08:01:56.654117 12494 heartbeater.cc:412] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 08:01:56.671379 12364 tablet_bootstrap.cc:492] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=400 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:01:56.671747 12364 tablet_bootstrap.cc:492] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06: Bootstrap complete.
I20260812 08:01:56.672608 12364 ts_tablet_manager.cc:1403] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06: Time spent bootstrapping tablet: real 0.035s	user 0.023s	sys 0.008s
I20260812 08:01:56.675024 12364 raft_consensus.cc:359] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } } peers { permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: NON_VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: true } }
I20260812 08:01:56.675357 12364 raft_consensus.cc:740] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: df29c7b34b2e4b8d9eddef7c5f166e06, State: Initialized, Role: LEARNER
I20260812 08:01:56.675717 12364 consensus_queue.cc:260] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 3.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } } peers { permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: NON_VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: true } }
I20260812 08:01:56.676565 12364 ts_tablet_manager.cc:1434] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06: Time spent starting tablet: real 0.004s	user 0.006s	sys 0.000s
W20260812 08:01:56.761018 12502 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:56.761261 12502 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:56.761307 12502 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:56.764633 12502 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:01:56.764698 12502 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:56.764719 12502 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:01:56.764735 12502 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:01:56.768755 12502 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:46389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:39483
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:39483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=37805
--never_fsync=true
--heap_profile_path=/tmp/kudu.12502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:56.769994 12502 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:56.771162 12502 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:56.776517 12510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:56.776528 12509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:56.776556 12512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:56.776767 12502 server_base.cc:1061] running on GCE node
I20260812 08:01:56.777194 12502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:56.778100 12502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:56.779246 12502 hybrid_clock.cc:648] HybridClock initialized: now 1786521716779235 us; error 26 us; skew 500 ppm
I20260812 08:01:56.780835 12502 webserver.cc:533] Webserver started at http://127.0.186.126:37805/ using document root <none> and password file <none>
I20260812 08:01:56.781369 12502 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:56.781461 12502 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:56.784612 12502 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:56.786425 12519 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:56.787336 12502 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:01:56.787436 12502 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "717537b7ea584eafbbe1e6a991ab9e2f"
format_stamp: "Formatted at 2026-08-12 08:01:38 on dist-test-slave-49h3"
I20260812 08:01:56.787809 12502 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:56.806077 12502 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:56.806738 12502 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:56.806905 12502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:56.813843 12502 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:39483
I20260812 08:01:56.813849 12577 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:39483 every 8 connection(s)
I20260812 08:01:56.814970 12502 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.RestartClusterWithNonVoter.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:01:56.819033 12578 sys_catalog.cc:263] Verifying existing consensus state
I20260812 08:01:56.819315   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 12502
I20260812 08:01:56.820775 12578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f: Bootstrap starting.
I20260812 08:01:56.834657 12578 log.cc:826] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:56.842995 12578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:01:56.843355 12578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f: Bootstrap complete.
I20260812 08:01:56.846967 12578 raft_consensus.cc:359] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "717537b7ea584eafbbe1e6a991ab9e2f" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 39483 } }
I20260812 08:01:56.847293 12578 raft_consensus.cc:740] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 717537b7ea584eafbbe1e6a991ab9e2f, State: Initialized, Role: FOLLOWER
I20260812 08:01:56.847715 12578 consensus_queue.cc:260] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "717537b7ea584eafbbe1e6a991ab9e2f" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 39483 } }
I20260812 08:01:56.847862 12578 raft_consensus.cc:399] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:01:56.847910 12578 raft_consensus.cc:493] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:01:56.847997 12578 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:01:56.849574 12578 raft_consensus.cc:515] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "717537b7ea584eafbbe1e6a991ab9e2f" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 39483 } }
I20260812 08:01:56.849871 12578 leader_election.cc:304] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 717537b7ea584eafbbe1e6a991ab9e2f; no voters: 
I20260812 08:01:56.850121 12578 leader_election.cc:290] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [CANDIDATE]: Term 2 election: Requested vote from peers 
I20260812 08:01:56.850251 12584 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:01:56.850476 12584 raft_consensus.cc:697] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [term 2 LEADER]: Becoming Leader. State: Replica: 717537b7ea584eafbbe1e6a991ab9e2f, State: Running, Role: LEADER
I20260812 08:01:56.850771 12584 consensus_queue.cc:237] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "717537b7ea584eafbbe1e6a991ab9e2f" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 39483 } }
I20260812 08:01:56.851214 12578 sys_catalog.cc:565] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:01:56.852182 12585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "717537b7ea584eafbbe1e6a991ab9e2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "717537b7ea584eafbbe1e6a991ab9e2f" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 39483 } } }
I20260812 08:01:56.852196 12586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 717537b7ea584eafbbe1e6a991ab9e2f. Latest consensus state: current_term: 2 leader_uuid: "717537b7ea584eafbbe1e6a991ab9e2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "717537b7ea584eafbbe1e6a991ab9e2f" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 39483 } } }
I20260812 08:01:56.852330 12586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [sys.catalog]: This master's current role is: LEADER
I20260812 08:01:56.852583 12585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f [sys.catalog]: This master's current role is: LEADER
I20260812 08:01:56.852787 12595 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:01:56.855984 12595 catalog_manager.cc:679] Loaded metadata for table TestTable [id=d1cff5d81143457a8a1c6eb392aae269]
I20260812 08:01:56.857295 12595 tablet_loader.cc:96] loaded metadata for tablet 9e91e10f1ecf4319b2b55a3eca65dfde (table TestTable [id=d1cff5d81143457a8a1c6eb392aae269])
I20260812 08:01:56.857471 12595 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:01:56.858120 12595 catalog_manager.cc:1295] Loaded cluster ID: 87fd517f85154b10ac39f25b50bd7d8f
I20260812 08:01:56.858263 12595 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:01:56.859995 12595 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:01:56.860646 12595 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 717537b7ea584eafbbe1e6a991ab9e2f: Loaded TSK: 0
I20260812 08:01:56.861506 12595 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:01:56.939786 11871 heartbeater.cc:344] Connected to a master server at 127.0.186.126:39483
I20260812 08:01:56.940027 11871 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:56.940747 11871 heartbeater.cc:507] Master 127.0.186.126:39483 requested a full tablet report, sending...
I20260812 08:01:56.942512 12540 ts_manager.cc:194] Registered new tserver with Master: c5c99765f674430db87f0bd401e7a193 (127.0.186.66:43103)
I20260812 08:01:56.943769 12540 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.66:55783
I20260812 08:01:56.997897 12605 raft_consensus.cc:493] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:01:56.998116 12605 raft_consensus.cc:515] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } } peers { permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: NON_VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: true } }
I20260812 08:01:56.999250 12605 leader_election.cc:290] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71:38427), 5f0292e14d114be2be61f1b49041540f (127.0.186.68:43825)
W20260812 08:01:57.000000 11574 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.71:38427: connect: Connection refused (error 111)
W20260812 08:01:57.001868 11574 leader_election.cc:336] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71:38427): Network error: Client connection negotiation failed: client connection to 127.0.186.71:38427: connect: Connection refused (error 111)
I20260812 08:01:57.007859 12124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e91e10f1ecf4319b2b55a3eca65dfde" candidate_uuid: "2ef089e13bd1432e9fa75814933f3bec" candidate_term: 4 candidate_status { last_received { term: 3 index: 11 } } ignore_live_leader: false dest_uuid: "5f0292e14d114be2be61f1b49041540f" is_pre_election: true
I20260812 08:01:57.008198 12124 raft_consensus.cc:2468] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ef089e13bd1432e9fa75814933f3bec in term 3.
I20260812 08:01:57.009109 11574 leader_election.cc:304] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ef089e13bd1432e9fa75814933f3bec, 5f0292e14d114be2be61f1b49041540f; no voters: 0daf92242f824102a4d9d0feaa7604dc
I20260812 08:01:57.009372 12605 raft_consensus.cc:2804] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 08:01:57.009488 12605 raft_consensus.cc:493] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:01:57.009560 12605 raft_consensus.cc:3060] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 3 FOLLOWER]: Advancing to term 4
I20260812 08:01:57.011301 12605 raft_consensus.cc:515] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 4 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } } peers { permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: NON_VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: true } }
I20260812 08:01:57.011727 12605 leader_election.cc:290] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [CANDIDATE]: Term 4 election: Requested vote from peers 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71:38427), 5f0292e14d114be2be61f1b49041540f (127.0.186.68:43825)
I20260812 08:01:57.012539 12124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e91e10f1ecf4319b2b55a3eca65dfde" candidate_uuid: "2ef089e13bd1432e9fa75814933f3bec" candidate_term: 4 candidate_status { last_received { term: 3 index: 11 } } ignore_live_leader: false dest_uuid: "5f0292e14d114be2be61f1b49041540f"
I20260812 08:01:57.012692 12124 raft_consensus.cc:3060] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 3 FOLLOWER]: Advancing to term 4
I20260812 08:01:57.014818 12124 raft_consensus.cc:2468] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ef089e13bd1432e9fa75814933f3bec in term 4.
I20260812 08:01:57.015152 11574 leader_election.cc:304] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ef089e13bd1432e9fa75814933f3bec, 5f0292e14d114be2be61f1b49041540f; no voters: 
I20260812 08:01:57.015362 12605 raft_consensus.cc:2804] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 4 FOLLOWER]: Leader election won for term 4
I20260812 08:01:57.015609 12605 raft_consensus.cc:697] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 4 LEADER]: Becoming Leader. State: Replica: 2ef089e13bd1432e9fa75814933f3bec, State: Running, Role: LEADER
W20260812 08:01:57.015774 11574 leader_election.cc:336] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71:38427): Network error: Client connection negotiation failed: client connection to 127.0.186.71:38427: connect: Connection refused (error 111)
I20260812 08:01:57.016019 12605 consensus_queue.cc:237] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 3.11, Last appended by leader: 11, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } } peers { permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: NON_VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: true } }
I20260812 08:01:57.019920 11709 heartbeater.cc:344] Connected to a master server at 127.0.186.126:39483
I20260812 08:01:57.020094 11709 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:57.020653 11709 heartbeater.cc:507] Master 127.0.186.126:39483 requested a full tablet report, sending...
I20260812 08:01:57.022787 12540 ts_manager.cc:194] Registered new tserver with Master: 2ef089e13bd1432e9fa75814933f3bec (127.0.186.65:35839)
I20260812 08:01:57.023561 12540 catalog_manager.cc:5719] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec reported cstate change: term changed from 3 to 4, leader changed from 5f0292e14d114be2be61f1b49041540f (127.0.186.68) to 2ef089e13bd1432e9fa75814933f3bec (127.0.186.65). New cstate: current_term: 4 leader_uuid: "2ef089e13bd1432e9fa75814933f3bec" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: NON_VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260812 08:01:57.029471 12540 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.65:38525
I20260812 08:01:57.124151 12025 heartbeater.cc:344] Connected to a master server at 127.0.186.126:39483
I20260812 08:01:57.124364 12025 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:57.124948 12025 heartbeater.cc:507] Master 127.0.186.126:39483 requested a full tablet report, sending...
I20260812 08:01:57.126024 12540 ts_manager.cc:194] Registered new tserver with Master: 0fc4212fb0854d5d8b405156a6f8c4bc (127.0.186.67:36027)
I20260812 08:01:57.126593 12540 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.67:47359
I20260812 08:01:57.335565 12180 heartbeater.cc:344] Connected to a master server at 127.0.186.126:39483
I20260812 08:01:57.335757 12180 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:57.336360 12180 heartbeater.cc:507] Master 127.0.186.126:39483 requested a full tablet report, sending...
I20260812 08:01:57.337975 12539 ts_manager.cc:194] Registered new tserver with Master: 5f0292e14d114be2be61f1b49041540f (127.0.186.68:43825)
I20260812 08:01:57.338773 12539 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.68:44781
I20260812 08:01:57.417186 12439 raft_consensus.cc:3060] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06 [term 3 LEARNER]: Advancing to term 4
I20260812 08:01:57.419281 12439 raft_consensus.cc:1275] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06 [term 4 LEARNER]: Refusing update from remote peer 2ef089e13bd1432e9fa75814933f3bec: Log matching property violated. Preceding OpId in replica: term: 3 index: 11. Preceding OpId from leader: term: 4 index: 12. (index mismatch)
I20260812 08:01:57.420307 12605 consensus_queue.cc:1048] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [LEADER]: Connected to new peer: Peer: permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: NON_VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260812 08:01:57.423264 12605 raft_consensus.cc:1064] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec: attempting to promote NON_VOTER df29c7b34b2e4b8d9eddef7c5f166e06 to VOTER
W20260812 08:01:57.423424 12605 raft_consensus.cc:1067] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec: Unable to promote non-voter df29c7b34b2e4b8d9eddef7c5f166e06: Illegal state: Leader has not yet committed an operation in its own term
I20260812 08:01:57.425508 12494 heartbeater.cc:344] Connected to a master server at 127.0.186.126:39483
I20260812 08:01:57.425657 12494 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:57.426167 12494 heartbeater.cc:507] Master 127.0.186.126:39483 requested a full tablet report, sending...
I20260812 08:01:57.427685 12539 ts_manager.cc:194] Registered new tserver with Master: df29c7b34b2e4b8d9eddef7c5f166e06 (127.0.186.70:45261)
I20260812 08:01:57.428431 12539 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.70:37111
I20260812 08:01:57.476035 12336 heartbeater.cc:344] Connected to a master server at 127.0.186.126:39483
I20260812 08:01:57.476233 12336 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:57.476778 12336 heartbeater.cc:507] Master 127.0.186.126:39483 requested a full tablet report, sending...
I20260812 08:01:57.477754 12539 ts_manager.cc:194] Registered new tserver with Master: b8e3b126cd934c6e85b1c5f9442ac7f5 (127.0.186.69:46205)
I20260812 08:01:57.478279 12539 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.69:36171
I20260812 08:01:57.479815   745 external_mini_cluster.cc:955] 6 TS(s) registered with all masters
W20260812 08:01:57.529327 11574 consensus_peers.cc:597] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec -> Peer 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71:38427): Couldn't send request to peer 0daf92242f824102a4d9d0feaa7604dc. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.71:38427: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 08:01:57.604209 12124 raft_consensus.cc:1275] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 4 FOLLOWER]: Refusing update from remote peer 2ef089e13bd1432e9fa75814933f3bec: Log matching property violated. Preceding OpId in replica: term: 3 index: 11. Preceding OpId from leader: term: 4 index: 12. (index mismatch)
I20260812 08:01:57.604885 12605 consensus_queue.cc:1048] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260812 08:01:57.606324 12180 heartbeater.cc:499] Master 127.0.186.126:39483 was elected leader, sending a full tablet report...
I20260812 08:01:57.608656 11709 heartbeater.cc:499] Master 127.0.186.126:39483 was elected leader, sending a full tablet report...
I20260812 08:01:57.609929 12494 heartbeater.cc:499] Master 127.0.186.126:39483 was elected leader, sending a full tablet report...
I20260812 08:01:57.610365 12605 raft_consensus.cc:1064] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec: attempting to promote NON_VOTER df29c7b34b2e4b8d9eddef7c5f166e06 to VOTER
I20260812 08:01:57.611120 12605 consensus_queue.cc:237] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 4.12, Last appended by leader: 11, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } } peers { permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: false } }
I20260812 08:01:57.612310 12124 raft_consensus.cc:1275] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 4 FOLLOWER]: Refusing update from remote peer 2ef089e13bd1432e9fa75814933f3bec: Log matching property violated. Preceding OpId in replica: term: 4 index: 12. Preceding OpId from leader: term: 4 index: 13. (index mismatch)
I20260812 08:01:57.612399 12439 raft_consensus.cc:1275] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06 [term 4 LEARNER]: Refusing update from remote peer 2ef089e13bd1432e9fa75814933f3bec: Log matching property violated. Preceding OpId in replica: term: 4 index: 12. Preceding OpId from leader: term: 4 index: 13. (index mismatch)
I20260812 08:01:57.612782 12605 consensus_queue.cc:1048] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260812 08:01:57.612902 12632 consensus_queue.cc:1048] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [LEADER]: Connected to new peer: Peer: permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260812 08:01:57.613286 11574 consensus_peers.cc:597] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec -> Peer 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71:38427): Couldn't send request to peer 0daf92242f824102a4d9d0feaa7604dc. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.71:38427: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 08:01:57.614701 12632 raft_consensus.cc:2955] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 4 LEADER]: Committing config change with OpId 4.13: config changed from index 11 to 13, df29c7b34b2e4b8d9eddef7c5f166e06 (127.0.186.70) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } } peers { permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: false } } }
I20260812 08:01:57.615231 12439 raft_consensus.cc:2955] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06 [term 4 FOLLOWER]: Committing config change with OpId 4.13: config changed from index 11 to 13, df29c7b34b2e4b8d9eddef7c5f166e06 (127.0.186.70) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } } peers { permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: false } } }
I20260812 08:01:57.616839 12124 raft_consensus.cc:2955] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 4 FOLLOWER]: Committing config change with OpId 4.13: config changed from index 11 to 13, df29c7b34b2e4b8d9eddef7c5f166e06 (127.0.186.70) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } } peers { permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: false } } }
I20260812 08:01:57.617564 12539 catalog_manager.cc:5719] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec reported cstate change: config changed from index 11 to 13, df29c7b34b2e4b8d9eddef7c5f166e06 (127.0.186.70) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "2ef089e13bd1432e9fa75814933f3bec" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0daf92242f824102a4d9d0feaa7604dc" member_type: VOTER last_known_addr { host: "127.0.186.71" port: 38427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 08:01:57.626256 11654 consensus_queue.cc:237] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 4.13, Last appended by leader: 11, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } } peers { permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: false } }
I20260812 08:01:57.627409 12124 raft_consensus.cc:1275] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 4 FOLLOWER]: Refusing update from remote peer 2ef089e13bd1432e9fa75814933f3bec: Log matching property violated. Preceding OpId in replica: term: 4 index: 13. Preceding OpId from leader: term: 4 index: 14. (index mismatch)
I20260812 08:01:57.627417 12439 raft_consensus.cc:1275] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06 [term 4 FOLLOWER]: Refusing update from remote peer 2ef089e13bd1432e9fa75814933f3bec: Log matching property violated. Preceding OpId in replica: term: 4 index: 13. Preceding OpId from leader: term: 4 index: 14. (index mismatch)
I20260812 08:01:57.627782 12632 consensus_queue.cc:1048] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [LEADER]: Connected to new peer: Peer: permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260812 08:01:57.627915 12605 consensus_queue.cc:1048] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260812 08:01:57.629670 12611 raft_consensus.cc:2955] T 9e91e10f1ecf4319b2b55a3eca65dfde P 2ef089e13bd1432e9fa75814933f3bec [term 4 LEADER]: Committing config change with OpId 4.14: config changed from index 13 to 14, VOTER 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } } peers { permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: false } } }
I20260812 08:01:57.630157 12439 raft_consensus.cc:2955] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06 [term 4 FOLLOWER]: Committing config change with OpId 4.14: config changed from index 13 to 14, VOTER 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } } peers { permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: false } } }
I20260812 08:01:57.630330 12124 raft_consensus.cc:2955] T 9e91e10f1ecf4319b2b55a3eca65dfde P 5f0292e14d114be2be61f1b49041540f [term 4 FOLLOWER]: Committing config change with OpId 4.14: config changed from index 13 to 14, VOTER 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } } peers { permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: false } } }
I20260812 08:01:57.631929 12520 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 9e91e10f1ecf4319b2b55a3eca65dfde with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:01:57.632682 12540 catalog_manager.cc:5719] T 9e91e10f1ecf4319b2b55a3eca65dfde P df29c7b34b2e4b8d9eddef7c5f166e06 reported cstate change: config changed from index 13 to 14, VOTER 0daf92242f824102a4d9d0feaa7604dc (127.0.186.71) evicted. New cstate: current_term: 4 leader_uuid: "2ef089e13bd1432e9fa75814933f3bec" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "2ef089e13bd1432e9fa75814933f3bec" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 35839 } } peers { permanent_uuid: "5f0292e14d114be2be61f1b49041540f" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 43825 } } peers { permanent_uuid: "df29c7b34b2e4b8d9eddef7c5f166e06" member_type: VOTER last_known_addr { host: "127.0.186.70" port: 45261 } attrs { promote: false } } }
W20260812 08:01:57.634646 12540 catalog_manager.cc:5911] Failed to send DeleteTablet RPC for tablet 9e91e10f1ecf4319b2b55a3eca65dfde on TS 0daf92242f824102a4d9d0feaa7604dc: Not found: failed to reset TS proxy: Could not find TS for UUID 0daf92242f824102a4d9d0feaa7604dc
I20260812 08:01:57.655591   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 2ef089e13bd1432e9fa75814933f3bec to finish bootstrapping
I20260812 08:01:57.660945   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 5f0292e14d114be2be61f1b49041540f to finish bootstrapping
I20260812 08:01:57.665952   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver df29c7b34b2e4b8d9eddef7c5f166e06 to finish bootstrapping
I20260812 08:01:57.679533   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 11557
W20260812 08:01:57.687881 12521 catalog_manager.cc:4832] Async tablet task DeleteTablet RPC for tablet 9e91e10f1ecf4319b2b55a3eca65dfde on TS 0daf92242f824102a4d9d0feaa7604dc failed: Not found: failed to reset TS proxy: Could not find TS for UUID 0daf92242f824102a4d9d0feaa7604dc
I20260812 08:01:57.693185   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 11714
I20260812 08:01:57.703552   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 11875
I20260812 08:01:57.723068   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 12028
I20260812 08:01:57.735857   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 12184
I20260812 08:01:57.746236   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 12340
I20260812 08:01:57.757406   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 12502
2026-08-12T08:01:57Z chronyd exiting
[       OK ] RaftConsensusNonVoterITest.RestartClusterWithNonVoter (19815 ms)
[ RUN      ] RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue
I20260812 08:01:57.788893   745 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:01:57.788944   745 ts_itest-base.cc:116] --------------
I20260812 08:01:57.788959   745 ts_itest-base.cc:117] 4 tablet servers
I20260812 08:01:57.788971   745 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:01:57.788983   745 ts_itest-base.cc:119] --------------
2026-08-12T08:01:57Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:01:57Z Disabled control of system clock
I20260812 08:01:57.797854   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:41315
--webserver_interface=127.0.186.126
--webserver_port=0
--builtin_ntp_servers=127.0.186.84:33421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:41315
--catalog_manager_evict_excess_replicas=false with env {}
W20260812 08:01:57.903873 12653 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:57.904098 12653 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:57.904152 12653 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:57.907459 12653 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:01:57.907524 12653 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:57.907552 12653 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:01:57.907575 12653 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:01:57.911523 12653 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:33421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--catalog_manager_evict_excess_replicas=false
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:41315
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:41315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.12653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:57.912662 12653 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:57.913868 12653 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:57.919142 12660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:57.919140 12659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:57.919288 12653 server_base.cc:1061] running on GCE node
W20260812 08:01:57.919147 12665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:57.919844 12653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:57.920683 12653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:57.921828 12653 hybrid_clock.cc:648] HybridClock initialized: now 1786521717921819 us; error 27 us; skew 500 ppm
I20260812 08:01:57.923513 12653 webserver.cc:533] Webserver started at http://127.0.186.126:45391/ using document root <none> and password file <none>
I20260812 08:01:57.924055 12653 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:57.924114 12653 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:57.924291 12653 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:01:57.925838 12653 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "3794f10c99d240d39b1721580f2ff634"
format_stamp: "Formatted at 2026-08-12 08:01:57 on dist-test-slave-49h3"
I20260812 08:01:57.926322 12653 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "3794f10c99d240d39b1721580f2ff634"
format_stamp: "Formatted at 2026-08-12 08:01:57 on dist-test-slave-49h3"
I20260812 08:01:57.929483 12653 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:01:57.931571 12673 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:57.932525 12653 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:57.932649 12653 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "3794f10c99d240d39b1721580f2ff634"
format_stamp: "Formatted at 2026-08-12 08:01:57 on dist-test-slave-49h3"
I20260812 08:01:57.932732 12653 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:57.946856 12653 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:57.947765 12653 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:57.947937 12653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:57.955720 12731 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:41315 every 8 connection(s)
I20260812 08:01:57.955736 12653 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:41315
I20260812 08:01:57.956840 12653 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:01:57.959759 12732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:01:57.964841 12732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3794f10c99d240d39b1721580f2ff634: Bootstrap starting.
I20260812 08:01:57.966003   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 12653
I20260812 08:01:57.966153   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:01:57.967487 12732 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3794f10c99d240d39b1721580f2ff634: Neither blocks nor log segments found. Creating new log.
I20260812 08:01:57.968323 12732 log.cc:826] T 00000000000000000000000000000000 P 3794f10c99d240d39b1721580f2ff634: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:57.970248 12732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3794f10c99d240d39b1721580f2ff634: No bootstrap required, opened a new log
I20260812 08:01:57.973052 12732 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3794f10c99d240d39b1721580f2ff634 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3794f10c99d240d39b1721580f2ff634" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41315 } }
I20260812 08:01:57.973352 12732 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3794f10c99d240d39b1721580f2ff634 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:01:57.973425 12732 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3794f10c99d240d39b1721580f2ff634 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3794f10c99d240d39b1721580f2ff634, State: Initialized, Role: FOLLOWER
I20260812 08:01:57.973943 12732 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3794f10c99d240d39b1721580f2ff634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3794f10c99d240d39b1721580f2ff634" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41315 } }
I20260812 08:01:57.974116 12732 raft_consensus.cc:399] T 00000000000000000000000000000000 P 3794f10c99d240d39b1721580f2ff634 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:01:57.974180 12732 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3794f10c99d240d39b1721580f2ff634 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:01:57.974321 12732 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3794f10c99d240d39b1721580f2ff634 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:01:57.975174 12732 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3794f10c99d240d39b1721580f2ff634 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3794f10c99d240d39b1721580f2ff634" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41315 } }
I20260812 08:01:57.975584 12732 leader_election.cc:304] T 00000000000000000000000000000000 P 3794f10c99d240d39b1721580f2ff634 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3794f10c99d240d39b1721580f2ff634; no voters: 
I20260812 08:01:57.975965 12732 leader_election.cc:290] T 00000000000000000000000000000000 P 3794f10c99d240d39b1721580f2ff634 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:01:57.976080 12738 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3794f10c99d240d39b1721580f2ff634 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:01:57.976279 12738 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3794f10c99d240d39b1721580f2ff634 [term 1 LEADER]: Becoming Leader. State: Replica: 3794f10c99d240d39b1721580f2ff634, State: Running, Role: LEADER
I20260812 08:01:57.976600 12738 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3794f10c99d240d39b1721580f2ff634 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3794f10c99d240d39b1721580f2ff634" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41315 } }
I20260812 08:01:57.977140 12732 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3794f10c99d240d39b1721580f2ff634 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:01:57.978232 12740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3794f10c99d240d39b1721580f2ff634 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3794f10c99d240d39b1721580f2ff634. Latest consensus state: current_term: 1 leader_uuid: "3794f10c99d240d39b1721580f2ff634" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3794f10c99d240d39b1721580f2ff634" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41315 } } }
I20260812 08:01:57.978366 12740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3794f10c99d240d39b1721580f2ff634 [sys.catalog]: This master's current role is: LEADER
I20260812 08:01:57.978632 12739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3794f10c99d240d39b1721580f2ff634 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3794f10c99d240d39b1721580f2ff634" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3794f10c99d240d39b1721580f2ff634" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41315 } } }
I20260812 08:01:57.978722 12739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3794f10c99d240d39b1721580f2ff634 [sys.catalog]: This master's current role is: LEADER
I20260812 08:01:57.978991 12747 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:01:57.982048 12747 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:01:57.987393 12747 catalog_manager.cc:1383] Generated new cluster ID: 7ebb1b961ee2422fae72354e50edf66f
I20260812 08:01:57.987475 12747 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:01:58.000757 12747 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:01:58.001821 12747 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:01:58.011336 12747 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 3794f10c99d240d39b1721580f2ff634: Generated new TSK 0
I20260812 08:01:58.011994 12747 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:01:58.019550   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:0
--local_ip_for_outbound_sockets=127.0.186.65
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:41315
--builtin_ntp_servers=127.0.186.84:33421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=50 with env {}
W20260812 08:01:58.134137 12759 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:58.134372 12759 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:58.134415 12759 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:58.138002 12759 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:58.138125 12759 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:01:58.142226 12759 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:33421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=50
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:41315
--never_fsync=true
--heap_profile_path=/tmp/kudu.12759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:58.143537 12759 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:58.144837 12759 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:01:58.148191 12759 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 25
W20260812 08:01:58.152767 12768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:58.152860 12765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:58.152769 12764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:58.152854 12759 server_base.cc:1061] running on GCE node
I20260812 08:01:58.153570 12759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:58.154258 12759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:58.155432 12759 hybrid_clock.cc:648] HybridClock initialized: now 1786521718155409 us; error 37 us; skew 500 ppm
I20260812 08:01:58.157409 12759 webserver.cc:533] Webserver started at http://127.0.186.65:43183/ using document root <none> and password file <none>
I20260812 08:01:58.158097 12759 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:58.158159 12759 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:58.158355 12759 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:01:58.160023 12759 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "b3f8fc0e0371462da810db3e892efcf2"
format_stamp: "Formatted at 2026-08-12 08:01:58 on dist-test-slave-49h3"
I20260812 08:01:58.160527 12759 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "b3f8fc0e0371462da810db3e892efcf2"
format_stamp: "Formatted at 2026-08-12 08:01:58 on dist-test-slave-49h3"
I20260812 08:01:58.164213 12759 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.000s	sys 0.006s
I20260812 08:01:58.166734 12778 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:58.167876 12759 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:01:58.168009 12759 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "b3f8fc0e0371462da810db3e892efcf2"
format_stamp: "Formatted at 2026-08-12 08:01:58 on dist-test-slave-49h3"
I20260812 08:01:58.168144 12759 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:58.188469 12759 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:58.189297 12759 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:58.189508 12759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:58.190150 12759 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:58.191208 12759 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:01:58.191258 12759 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:58.191313 12759 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:01:58.191341 12759 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:58.200788 12759 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:44207
I20260812 08:01:58.200804 12906 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:44207 every 8 connection(s)
I20260812 08:01:58.201825 12759 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:01:58.207145   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 12759
I20260812 08:01:58.207270   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:01:58.209954   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.66:0
--local_ip_for_outbound_sockets=127.0.186.66
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:41315
--builtin_ntp_servers=127.0.186.84:33421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=50 with env {}
I20260812 08:01:58.211323 12907 heartbeater.cc:344] Connected to a master server at 127.0.186.126:41315
I20260812 08:01:58.211594 12907 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:58.212155 12907 heartbeater.cc:507] Master 127.0.186.126:41315 requested a full tablet report, sending...
I20260812 08:01:58.213915 12692 ts_manager.cc:194] Registered new tserver with Master: b3f8fc0e0371462da810db3e892efcf2 (127.0.186.65:44207)
I20260812 08:01:58.215628 12692 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.65:38783
W20260812 08:01:58.320497 12910 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:58.320737 12910 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:58.320781 12910 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:58.324124 12910 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:58.324229 12910 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.66
I20260812 08:01:58.328037 12910 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:33421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=50
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:41315
--never_fsync=true
--heap_profile_path=/tmp/kudu.12910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:58.329115 12910 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:58.330245 12910 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:01:58.333108 12910 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 25
W20260812 08:01:58.336802 12917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:58.336827 12920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:58.336829 12916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:58.337015 12910 server_base.cc:1061] running on GCE node
I20260812 08:01:58.337427 12910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:58.338050 12910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:58.339213 12910 hybrid_clock.cc:648] HybridClock initialized: now 1786521718339200 us; error 26 us; skew 500 ppm
I20260812 08:01:58.341014 12910 webserver.cc:533] Webserver started at http://127.0.186.66:37923/ using document root <none> and password file <none>
I20260812 08:01:58.341650 12910 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:58.341708 12910 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:58.341905 12910 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:01:58.343446 12910 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "2b912f3ad71d482e95cad27e3dd97cb6"
format_stamp: "Formatted at 2026-08-12 08:01:58 on dist-test-slave-49h3"
I20260812 08:01:58.343917 12910 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "2b912f3ad71d482e95cad27e3dd97cb6"
format_stamp: "Formatted at 2026-08-12 08:01:58 on dist-test-slave-49h3"
I20260812 08:01:58.347463 12910 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:01:58.349700 12926 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:58.350675 12910 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 08:01:58.350778 12910 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "2b912f3ad71d482e95cad27e3dd97cb6"
format_stamp: "Formatted at 2026-08-12 08:01:58 on dist-test-slave-49h3"
I20260812 08:01:58.350876 12910 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:58.361989 12910 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:58.362622 12910 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:58.362792 12910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:58.363360 12910 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:58.364374 12910 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:01:58.364423 12910 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:58.364475 12910 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:01:58.364506 12910 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:58.373277 12910 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.66:38399
I20260812 08:01:58.373306 13057 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.66:38399 every 8 connection(s)
I20260812 08:01:58.374361 12910 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:01:58.377501   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 12910
I20260812 08:01:58.377624   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:01:58.380096   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.67:0
--local_ip_for_outbound_sockets=127.0.186.67
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:41315
--builtin_ntp_servers=127.0.186.84:33421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=50 with env {}
I20260812 08:01:58.383550 13058 heartbeater.cc:344] Connected to a master server at 127.0.186.126:41315
I20260812 08:01:58.383810 13058 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:58.384352 13058 heartbeater.cc:507] Master 127.0.186.126:41315 requested a full tablet report, sending...
I20260812 08:01:58.385393 12692 ts_manager.cc:194] Registered new tserver with Master: 2b912f3ad71d482e95cad27e3dd97cb6 (127.0.186.66:38399)
I20260812 08:01:58.385963 12692 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.66:55905
W20260812 08:01:58.488966 13062 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:58.489192 13062 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:58.489254 13062 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:58.492543 13062 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:58.492676 13062 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.67
I20260812 08:01:58.496482 13062 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:33421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=50
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:41315
--never_fsync=true
--heap_profile_path=/tmp/kudu.13062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:58.497705 13062 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:58.498878 13062 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:01:58.501930 13062 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 25
W20260812 08:01:58.505918 13067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:58.505949 13070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:58.505914 13068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:58.506366 13062 server_base.cc:1061] running on GCE node
I20260812 08:01:58.506721 13062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:58.507267 13062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:58.508410 13062 hybrid_clock.cc:648] HybridClock initialized: now 1786521718508394 us; error 31 us; skew 500 ppm
I20260812 08:01:58.510298 13062 webserver.cc:533] Webserver started at http://127.0.186.67:33035/ using document root <none> and password file <none>
I20260812 08:01:58.510880 13062 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:58.510937 13062 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:58.511137 13062 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:01:58.512748 13062 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "24a9ea5c15da49ae9108750c73143b20"
format_stamp: "Formatted at 2026-08-12 08:01:58 on dist-test-slave-49h3"
I20260812 08:01:58.513234 13062 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "24a9ea5c15da49ae9108750c73143b20"
format_stamp: "Formatted at 2026-08-12 08:01:58 on dist-test-slave-49h3"
I20260812 08:01:58.516580 13062 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.000s
I20260812 08:01:58.518761 13077 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:58.519732 13062 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:01:58.519833 13062 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "24a9ea5c15da49ae9108750c73143b20"
format_stamp: "Formatted at 2026-08-12 08:01:58 on dist-test-slave-49h3"
I20260812 08:01:58.519924 13062 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:58.539189 13062 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:58.539892 13062 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:58.540076 13062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:58.540691 13062 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:58.541841 13062 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:01:58.541894 13062 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:58.541939 13062 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:01:58.541967 13062 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:58.551397 13062 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.67:44469
I20260812 08:01:58.551434 13207 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.67:44469 every 8 connection(s)
I20260812 08:01:58.552425 13062 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:01:58.557152   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 13062
I20260812 08:01:58.557243   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:01:58.559551   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.68:0
--local_ip_for_outbound_sockets=127.0.186.68
--webserver_interface=127.0.186.68
--webserver_port=0
--tserver_master_addrs=127.0.186.126:41315
--builtin_ntp_servers=127.0.186.84:33421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=50 with env {}
I20260812 08:01:58.561472 13208 heartbeater.cc:344] Connected to a master server at 127.0.186.126:41315
I20260812 08:01:58.561719 13208 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:58.562220 13208 heartbeater.cc:507] Master 127.0.186.126:41315 requested a full tablet report, sending...
I20260812 08:01:58.563269 12692 ts_manager.cc:194] Registered new tserver with Master: 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469)
I20260812 08:01:58.563812 12692 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.67:60319
W20260812 08:01:58.666970 13213 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:58.667214 13213 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:58.667274 13213 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:58.670627 13213 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:58.670768 13213 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.68
I20260812 08:01:58.674933 13213 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:33421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=50
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.0.186.68
--webserver_port=0
--tserver_master_addrs=127.0.186.126:41315
--never_fsync=true
--heap_profile_path=/tmp/kudu.13213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.68

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:58.676124 13213 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:58.677314 13213 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:01:58.680375 13213 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 25
W20260812 08:01:58.684020 13218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:58.684056 13219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:58.684301 13213 server_base.cc:1061] running on GCE node
W20260812 08:01:58.684307 13221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:58.684660 13213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:58.685194 13213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:58.686369 13213 hybrid_clock.cc:648] HybridClock initialized: now 1786521718686337 us; error 48 us; skew 500 ppm
I20260812 08:01:58.688112 13213 webserver.cc:533] Webserver started at http://127.0.186.68:43157/ using document root <none> and password file <none>
I20260812 08:01:58.688683 13213 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:58.688741 13213 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:58.688930 13213 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:01:58.690626 13213 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "7dbf915c265f4d24b93e39bf4bafeb68"
format_stamp: "Formatted at 2026-08-12 08:01:58 on dist-test-slave-49h3"
I20260812 08:01:58.691102 13213 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "7dbf915c265f4d24b93e39bf4bafeb68"
format_stamp: "Formatted at 2026-08-12 08:01:58 on dist-test-slave-49h3"
I20260812 08:01:58.694743 13213 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:01:58.697012 13229 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:58.698081 13213 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:58.698204 13213 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "7dbf915c265f4d24b93e39bf4bafeb68"
format_stamp: "Formatted at 2026-08-12 08:01:58 on dist-test-slave-49h3"
I20260812 08:01:58.698297 13213 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:58.714788 13213 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:58.715514 13213 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:58.715699 13213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:58.716336 13213 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:58.717469 13213 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:01:58.717530 13213 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:58.717590 13213 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:01:58.717617 13213 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:58.727250 13213 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.68:46427
I20260812 08:01:58.727329 13359 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.68:46427 every 8 connection(s)
I20260812 08:01:58.728344 13213 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20260812 08:01:58.736370   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 13213
I20260812 08:01:58.736496   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20260812 08:01:58.737156 13360 heartbeater.cc:344] Connected to a master server at 127.0.186.126:41315
I20260812 08:01:58.737421 13360 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:58.738057 13360 heartbeater.cc:507] Master 127.0.186.126:41315 requested a full tablet report, sending...
I20260812 08:01:58.739078 12692 ts_manager.cc:194] Registered new tserver with Master: 7dbf915c265f4d24b93e39bf4bafeb68 (127.0.186.68:46427)
I20260812 08:01:58.739610 12692 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.68:34463
I20260812 08:01:58.750809   745 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20260812 08:01:58.762094 12692 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35226:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
I20260812 08:01:58.781332 12833 tablet_service.cc:1511] Processing CreateTablet for tablet 1ca0bf4c951f4e71963efbc1ebd7e4e8 (DEFAULT_TABLE table=TestTable [id=14c27606b874433ab346d723dbe5fa63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:01:58.781332 13286 tablet_service.cc:1511] Processing CreateTablet for tablet 1ca0bf4c951f4e71963efbc1ebd7e4e8 (DEFAULT_TABLE table=TestTable [id=14c27606b874433ab346d723dbe5fa63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:01:58.782550 12833 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ca0bf4c951f4e71963efbc1ebd7e4e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:01:58.782550 13286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ca0bf4c951f4e71963efbc1ebd7e4e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:01:58.783334 13129 tablet_service.cc:1511] Processing CreateTablet for tablet 1ca0bf4c951f4e71963efbc1ebd7e4e8 (DEFAULT_TABLE table=TestTable [id=14c27606b874433ab346d723dbe5fa63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:01:58.784209 13129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ca0bf4c951f4e71963efbc1ebd7e4e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:01:58.788655 13381 tablet_bootstrap.cc:492] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68: Bootstrap starting.
I20260812 08:01:58.789115 13379 tablet_bootstrap.cc:492] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2: Bootstrap starting.
I20260812 08:01:58.789901 13383 tablet_bootstrap.cc:492] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 24a9ea5c15da49ae9108750c73143b20: Bootstrap starting.
I20260812 08:01:58.791460 13379 tablet_bootstrap.cc:654] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2: Neither blocks nor log segments found. Creating new log.
I20260812 08:01:58.791726 13381 tablet_bootstrap.cc:654] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68: Neither blocks nor log segments found. Creating new log.
I20260812 08:01:58.791726 13383 tablet_bootstrap.cc:654] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 24a9ea5c15da49ae9108750c73143b20: Neither blocks nor log segments found. Creating new log.
I20260812 08:01:58.792200 13379 log.cc:826] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:58.792526 13383 log.cc:826] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 24a9ea5c15da49ae9108750c73143b20: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:58.792526 13381 log.cc:826] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:58.794028 13381 tablet_bootstrap.cc:492] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68: No bootstrap required, opened a new log
I20260812 08:01:58.794016 13379 tablet_bootstrap.cc:492] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2: No bootstrap required, opened a new log
I20260812 08:01:58.794085 13383 tablet_bootstrap.cc:492] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 24a9ea5c15da49ae9108750c73143b20: No bootstrap required, opened a new log
I20260812 08:01:58.794225 13379 ts_tablet_manager.cc:1403] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:01:58.794225 13383 ts_tablet_manager.cc:1403] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 24a9ea5c15da49ae9108750c73143b20: Time spent bootstrapping tablet: real 0.004s	user 0.003s	sys 0.000s
I20260812 08:01:58.794225 13381 ts_tablet_manager.cc:1403] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68: Time spent bootstrapping tablet: real 0.006s	user 0.004s	sys 0.000s
I20260812 08:01:58.796675 13381 raft_consensus.cc:359] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f8fc0e0371462da810db3e892efcf2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44207 } } peers { permanent_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 46427 } } peers { permanent_uuid: "24a9ea5c15da49ae9108750c73143b20" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44469 } }
I20260812 08:01:58.796895 13381 raft_consensus.cc:385] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:01:58.796962 13381 raft_consensus.cc:740] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dbf915c265f4d24b93e39bf4bafeb68, State: Initialized, Role: FOLLOWER
I20260812 08:01:58.797164 13383 raft_consensus.cc:359] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 24a9ea5c15da49ae9108750c73143b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f8fc0e0371462da810db3e892efcf2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44207 } } peers { permanent_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 46427 } } peers { permanent_uuid: "24a9ea5c15da49ae9108750c73143b20" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44469 } }
I20260812 08:01:58.797163 13379 raft_consensus.cc:359] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f8fc0e0371462da810db3e892efcf2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44207 } } peers { permanent_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 46427 } } peers { permanent_uuid: "24a9ea5c15da49ae9108750c73143b20" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44469 } }
I20260812 08:01:58.797353 13379 raft_consensus.cc:385] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:01:58.797353 13383 raft_consensus.cc:385] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 24a9ea5c15da49ae9108750c73143b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:01:58.797415 13379 raft_consensus.cc:740] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3f8fc0e0371462da810db3e892efcf2, State: Initialized, Role: FOLLOWER
I20260812 08:01:58.797425 13383 raft_consensus.cc:740] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 24a9ea5c15da49ae9108750c73143b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24a9ea5c15da49ae9108750c73143b20, State: Initialized, Role: FOLLOWER
I20260812 08:01:58.797494 13381 consensus_queue.cc:260] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f8fc0e0371462da810db3e892efcf2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44207 } } peers { permanent_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 46427 } } peers { permanent_uuid: "24a9ea5c15da49ae9108750c73143b20" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44469 } }
I20260812 08:01:58.797695 13389 raft_consensus.cc:493] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:01:58.797889 13389 raft_consensus.cc:515] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f8fc0e0371462da810db3e892efcf2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44207 } } peers { permanent_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 46427 } } peers { permanent_uuid: "24a9ea5c15da49ae9108750c73143b20" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44469 } }
I20260812 08:01:58.797940 13383 consensus_queue.cc:260] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 24a9ea5c15da49ae9108750c73143b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f8fc0e0371462da810db3e892efcf2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44207 } } peers { permanent_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 46427 } } peers { permanent_uuid: "24a9ea5c15da49ae9108750c73143b20" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44469 } }
I20260812 08:01:58.797940 13379 consensus_queue.cc:260] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f8fc0e0371462da810db3e892efcf2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44207 } } peers { permanent_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 46427 } } peers { permanent_uuid: "24a9ea5c15da49ae9108750c73143b20" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44469 } }
I20260812 08:01:58.798418 13381 ts_tablet_manager.cc:1434] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260812 08:01:58.798764 13379 ts_tablet_manager.cc:1434] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:01:58.798764 13383 ts_tablet_manager.cc:1434] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 24a9ea5c15da49ae9108750c73143b20: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:01:58.798774 13360 heartbeater.cc:499] Master 127.0.186.126:41315 was elected leader, sending a full tablet report...
I20260812 08:01:58.799104 12907 heartbeater.cc:499] Master 127.0.186.126:41315 was elected leader, sending a full tablet report...
I20260812 08:01:58.799104 13208 heartbeater.cc:499] Master 127.0.186.126:41315 was elected leader, sending a full tablet report...
I20260812 08:01:58.799706 13389 leader_election.cc:290] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3f8fc0e0371462da810db3e892efcf2 (127.0.186.65:44207), 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469)
W20260812 08:01:58.803362 13210 tablet.cc:2406] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 24a9ea5c15da49ae9108750c73143b20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:01:58.805008 13392 raft_consensus.cc:493] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:01:58.805189 13392 raft_consensus.cc:515] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f8fc0e0371462da810db3e892efcf2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44207 } } peers { permanent_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 46427 } } peers { permanent_uuid: "24a9ea5c15da49ae9108750c73143b20" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44469 } }
I20260812 08:01:58.805761 13151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca0bf4c951f4e71963efbc1ebd7e4e8" candidate_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24a9ea5c15da49ae9108750c73143b20" is_pre_election: true
I20260812 08:01:58.806041 13151 raft_consensus.cc:2468] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 24a9ea5c15da49ae9108750c73143b20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dbf915c265f4d24b93e39bf4bafeb68 in term 0.
I20260812 08:01:58.806511 13233 leader_election.cc:304] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24a9ea5c15da49ae9108750c73143b20, 7dbf915c265f4d24b93e39bf4bafeb68; no voters: 
I20260812 08:01:58.806798 13390 raft_consensus.cc:2804] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:01:58.806885 13390 raft_consensus.cc:493] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:01:58.806945 13390 raft_consensus.cc:3060] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:01:58.807139 12857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca0bf4c951f4e71963efbc1ebd7e4e8" candidate_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3f8fc0e0371462da810db3e892efcf2" is_pre_election: true
I20260812 08:01:58.807322 12857 raft_consensus.cc:2468] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dbf915c265f4d24b93e39bf4bafeb68 in term 0.
I20260812 08:01:58.807935 13392 leader_election.cc:290] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dbf915c265f4d24b93e39bf4bafeb68 (127.0.186.68:46427), 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469)
I20260812 08:01:58.808132 13390 raft_consensus.cc:515] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f8fc0e0371462da810db3e892efcf2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44207 } } peers { permanent_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 46427 } } peers { permanent_uuid: "24a9ea5c15da49ae9108750c73143b20" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44469 } }
I20260812 08:01:58.808851 12857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca0bf4c951f4e71963efbc1ebd7e4e8" candidate_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3f8fc0e0371462da810db3e892efcf2"
I20260812 08:01:58.808858 13151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca0bf4c951f4e71963efbc1ebd7e4e8" candidate_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24a9ea5c15da49ae9108750c73143b20"
I20260812 08:01:58.808959 12857 raft_consensus.cc:3060] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:01:58.808966 13151 raft_consensus.cc:3060] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 24a9ea5c15da49ae9108750c73143b20 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:01:58.808621 13390 leader_election.cc:290] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 [CANDIDATE]: Term 1 election: Requested vote from peers b3f8fc0e0371462da810db3e892efcf2 (127.0.186.65:44207), 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469)
I20260812 08:01:58.810071 13151 raft_consensus.cc:2468] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 24a9ea5c15da49ae9108750c73143b20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dbf915c265f4d24b93e39bf4bafeb68 in term 1.
I20260812 08:01:58.810092 12857 raft_consensus.cc:2468] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dbf915c265f4d24b93e39bf4bafeb68 in term 1.
I20260812 08:01:58.810441 13230 leader_election.cc:304] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dbf915c265f4d24b93e39bf4bafeb68, b3f8fc0e0371462da810db3e892efcf2; no voters: 
I20260812 08:01:58.810727 13390 raft_consensus.cc:2804] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:01:58.811309 13390 raft_consensus.cc:697] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 [term 1 LEADER]: Becoming Leader. State: Replica: 7dbf915c265f4d24b93e39bf4bafeb68, State: Running, Role: LEADER
I20260812 08:01:58.811609 13390 consensus_queue.cc:237] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f8fc0e0371462da810db3e892efcf2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44207 } } peers { permanent_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 46427 } } peers { permanent_uuid: "24a9ea5c15da49ae9108750c73143b20" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44469 } }
I20260812 08:01:58.811672 13309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca0bf4c951f4e71963efbc1ebd7e4e8" candidate_uuid: "b3f8fc0e0371462da810db3e892efcf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" is_pre_election: true
I20260812 08:01:58.811805 13151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca0bf4c951f4e71963efbc1ebd7e4e8" candidate_uuid: "b3f8fc0e0371462da810db3e892efcf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24a9ea5c15da49ae9108750c73143b20" is_pre_election: true
I20260812 08:01:58.811977 13151 raft_consensus.cc:2393] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 24a9ea5c15da49ae9108750c73143b20 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3f8fc0e0371462da810db3e892efcf2 in current term 1: Already voted for candidate 7dbf915c265f4d24b93e39bf4bafeb68 in this term.
I20260812 08:01:58.812502 12782 leader_election.cc:304] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3f8fc0e0371462da810db3e892efcf2; no voters: 24a9ea5c15da49ae9108750c73143b20, 7dbf915c265f4d24b93e39bf4bafeb68
I20260812 08:01:58.812727 13392 raft_consensus.cc:2749] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 08:01:58.815135 12692 catalog_manager.cc:5719] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7dbf915c265f4d24b93e39bf4bafeb68 (127.0.186.68). New cstate: current_term: 1 leader_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f8fc0e0371462da810db3e892efcf2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 46427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24a9ea5c15da49ae9108750c73143b20" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44469 } health_report { overall_health: UNKNOWN } } }
I20260812 08:01:58.827567   745 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20260812 08:01:58.828905   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver b3f8fc0e0371462da810db3e892efcf2 to finish bootstrapping
I20260812 08:01:58.834333   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 24a9ea5c15da49ae9108750c73143b20 to finish bootstrapping
I20260812 08:01:58.839672   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 7dbf915c265f4d24b93e39bf4bafeb68 to finish bootstrapping
I20260812 08:01:58.857932   745 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 0.0 1.1
I20260812 08:01:58.858553 13390 consensus_queue.cc:1048] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3f8fc0e0371462da810db3e892efcf2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:01:58.863719 13389 consensus_queue.cc:1048] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24a9ea5c15da49ae9108750c73143b20" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 08:01:58.961869 12901 generic_service.cc:139] {username='slave'} at 127.0.0.1:46868 forcing change of non-runtime-safe flag enable_leader_failure_detection
I20260812 08:01:58.962003 12901 generic_service.cc:175] {username='slave'} at 127.0.0.1:46868 changed flags via RPC: enable_leader_failure_detection from 'true' to 'false'
W20260812 08:01:58.968830 13052 generic_service.cc:139] {username='slave'} at 127.0.0.1:50332 forcing change of non-runtime-safe flag enable_leader_failure_detection
I20260812 08:01:58.968958 13052 generic_service.cc:175] {username='slave'} at 127.0.0.1:50332 changed flags via RPC: enable_leader_failure_detection from 'true' to 'false'
W20260812 08:01:58.970561 13201 generic_service.cc:139] {username='slave'} at 127.0.0.1:34406 forcing change of non-runtime-safe flag enable_leader_failure_detection
I20260812 08:01:58.970710 13201 generic_service.cc:175] {username='slave'} at 127.0.0.1:34406 changed flags via RPC: enable_leader_failure_detection from 'true' to 'false'
W20260812 08:01:58.971724 13354 generic_service.cc:139] {username='slave'} at 127.0.0.1:53112 forcing change of non-runtime-safe flag enable_leader_failure_detection
I20260812 08:01:58.971827 13354 generic_service.cc:175] {username='slave'} at 127.0.0.1:53112 changed flags via RPC: enable_leader_failure_detection from 'true' to 'false'
I20260812 08:01:58.972198   745 raft_consensus_nonvoter-itest.cc:1798] Getting leader replica...
I20260812 08:01:58.974754   745 raft_consensus_nonvoter-itest.cc:1802] Shutting down non-leader replicas...
I20260812 08:01:58.974848   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 12759
I20260812 08:01:58.985131   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 13062
I20260812 08:01:58.996394   745 raft_consensus_nonvoter-itest.cc:1809] Adding NON_VOTER replica...
I20260812 08:01:58.996546   745 raft_consensus_nonvoter-itest.cc:1818] Waiting for pending config...
I20260812 08:01:58.998198 13309 consensus_queue.cc:237] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b3f8fc0e0371462da810db3e892efcf2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44207 } } peers { permanent_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 46427 } } peers { permanent_uuid: "24a9ea5c15da49ae9108750c73143b20" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44469 } } peers { permanent_uuid: "2b912f3ad71d482e95cad27e3dd97cb6" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 38399 } attrs { } }
W20260812 08:01:58.999827 13230 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.65:44207: connect: Connection refused (error 111)
W20260812 08:01:59.000809 13230 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer b3f8fc0e0371462da810db3e892efcf2 (127.0.186.65:44207): Couldn't send request to peer b3f8fc0e0371462da810db3e892efcf2. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:44207: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260812 08:01:59.001278 13233 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469): Couldn't send request to peer 24a9ea5c15da49ae9108750c73143b20. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:44469: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260812 08:01:59.004356 13234 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer 2b912f3ad71d482e95cad27e3dd97cb6 (127.0.186.66:38399): Couldn't send request to peer 2b912f3ad71d482e95cad27e3dd97cb6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1ca0bf4c951f4e71963efbc1ebd7e4e8. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:01:59.061017 13412 ts_tablet_manager.cc:933] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 2b912f3ad71d482e95cad27e3dd97cb6: Initiating tablet copy from peer 7dbf915c265f4d24b93e39bf4bafeb68 (127.0.186.68:46427)
I20260812 08:01:59.061689 13412 tablet_copy_client.cc:323] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 2b912f3ad71d482e95cad27e3dd97cb6: tablet copy: Beginning tablet copy session from remote peer at address 127.0.186.68:46427
I20260812 08:01:59.065994 13334 tablet_copy_service.cc:140] P 7dbf915c265f4d24b93e39bf4bafeb68: Received BeginTabletCopySession request for tablet 1ca0bf4c951f4e71963efbc1ebd7e4e8 from peer 2b912f3ad71d482e95cad27e3dd97cb6 ({username='slave'} at 127.0.186.66:39179)
I20260812 08:01:59.066192 13334 tablet_copy_service.cc:161] P 7dbf915c265f4d24b93e39bf4bafeb68: Beginning new tablet copy session on tablet 1ca0bf4c951f4e71963efbc1ebd7e4e8 from peer 2b912f3ad71d482e95cad27e3dd97cb6 at {username='slave'} at 127.0.186.66:39179: session id = 2b912f3ad71d482e95cad27e3dd97cb6-1ca0bf4c951f4e71963efbc1ebd7e4e8
I20260812 08:01:59.067462 13334 tablet_copy_source_session.cc:215] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 08:01:59.071321 13412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ca0bf4c951f4e71963efbc1ebd7e4e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:01:59.076694 13412 tablet_copy_client.cc:806] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 2b912f3ad71d482e95cad27e3dd97cb6: tablet copy: Starting download of 0 data blocks...
I20260812 08:01:59.076944 13412 tablet_copy_client.cc:670] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 2b912f3ad71d482e95cad27e3dd97cb6: tablet copy: Starting download of 1 WAL segments...
I20260812 08:01:59.079231 13412 tablet_copy_client.cc:538] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 2b912f3ad71d482e95cad27e3dd97cb6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 08:01:59.081339 13412 tablet_bootstrap.cc:492] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 2b912f3ad71d482e95cad27e3dd97cb6: Bootstrap starting.
I20260812 08:01:59.087270 13412 log.cc:826] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 2b912f3ad71d482e95cad27e3dd97cb6: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:59.090258 13412 tablet_bootstrap.cc:492] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 2b912f3ad71d482e95cad27e3dd97cb6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260812 08:01:59.090590 13412 tablet_bootstrap.cc:492] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 2b912f3ad71d482e95cad27e3dd97cb6: Bootstrap complete.
I20260812 08:01:59.090901 13412 ts_tablet_manager.cc:1403] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 2b912f3ad71d482e95cad27e3dd97cb6: Time spent bootstrapping tablet: real 0.010s	user 0.010s	sys 0.000s
I20260812 08:01:59.093348 13412 raft_consensus.cc:359] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 2b912f3ad71d482e95cad27e3dd97cb6 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f8fc0e0371462da810db3e892efcf2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44207 } } peers { permanent_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 46427 } } peers { permanent_uuid: "24a9ea5c15da49ae9108750c73143b20" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44469 } }
I20260812 08:01:59.093936 13412 raft_consensus.cc:740] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 2b912f3ad71d482e95cad27e3dd97cb6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2b912f3ad71d482e95cad27e3dd97cb6, State: Initialized, Role: LEARNER
I20260812 08:01:59.094228 13412 consensus_queue.cc:260] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 2b912f3ad71d482e95cad27e3dd97cb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b3f8fc0e0371462da810db3e892efcf2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44207 } } peers { permanent_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 46427 } } peers { permanent_uuid: "24a9ea5c15da49ae9108750c73143b20" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44469 } } peers { permanent_uuid: "2b912f3ad71d482e95cad27e3dd97cb6" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 38399 } attrs { } }
I20260812 08:01:59.094957 13058 heartbeater.cc:499] Master 127.0.186.126:41315 was elected leader, sending a full tablet report...
I20260812 08:01:59.095067 13412 ts_tablet_manager.cc:1434] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 2b912f3ad71d482e95cad27e3dd97cb6: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:01:59.096048 13334 tablet_copy_service.cc:342] P 7dbf915c265f4d24b93e39bf4bafeb68: Request end of tablet copy session 2b912f3ad71d482e95cad27e3dd97cb6-1ca0bf4c951f4e71963efbc1ebd7e4e8 received from {username='slave'} at 127.0.186.66:39179
I20260812 08:01:59.096191 13334 tablet_copy_service.cc:434] P 7dbf915c265f4d24b93e39bf4bafeb68: ending tablet copy session 2b912f3ad71d482e95cad27e3dd97cb6-1ca0bf4c951f4e71963efbc1ebd7e4e8 on tablet 1ca0bf4c951f4e71963efbc1ebd7e4e8 with peer 2b912f3ad71d482e95cad27e3dd97cb6
I20260812 08:01:59.108423 13000 raft_consensus.cc:1217] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 2b912f3ad71d482e95cad27e3dd97cb6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260812 08:01:59.228495 13233 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469): Couldn't send request to peer 24a9ea5c15da49ae9108750c73143b20. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:44469: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260812 08:01:59.246672 13230 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer b3f8fc0e0371462da810db3e892efcf2 (127.0.186.65:44207): Couldn't send request to peer b3f8fc0e0371462da810db3e892efcf2. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:44207: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260812 08:01:59.476423 13233 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469): Couldn't send request to peer 24a9ea5c15da49ae9108750c73143b20. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:44469: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260812 08:01:59.508419 13230 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer b3f8fc0e0371462da810db3e892efcf2 (127.0.186.65:44207): Couldn't send request to peer b3f8fc0e0371462da810db3e892efcf2. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:44207: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260812 08:01:59.726998 13233 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469): Couldn't send request to peer 24a9ea5c15da49ae9108750c73143b20. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:44469: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260812 08:01:59.749820 13230 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer b3f8fc0e0371462da810db3e892efcf2 (127.0.186.65:44207): Couldn't send request to peer b3f8fc0e0371462da810db3e892efcf2. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:44207: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260812 08:01:59.956908 13233 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469): Couldn't send request to peer 24a9ea5c15da49ae9108750c73143b20. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:44469: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260812 08:01:59.982013 13230 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer b3f8fc0e0371462da810db3e892efcf2 (127.0.186.65:44207): Couldn't send request to peer b3f8fc0e0371462da810db3e892efcf2. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:44207: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260812 08:02:00.218142 13233 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469): Couldn't send request to peer 24a9ea5c15da49ae9108750c73143b20. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:44469: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260812 08:02:00.247807 13230 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer b3f8fc0e0371462da810db3e892efcf2 (127.0.186.65:44207): Couldn't send request to peer b3f8fc0e0371462da810db3e892efcf2. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:44207: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260812 08:02:00.455593 13233 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469): Couldn't send request to peer 24a9ea5c15da49ae9108750c73143b20. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:44469: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260812 08:02:00.495883 13230 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer b3f8fc0e0371462da810db3e892efcf2 (127.0.186.65:44207): Couldn't send request to peer b3f8fc0e0371462da810db3e892efcf2. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:44207: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260812 08:02:00.680862 13233 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469): Couldn't send request to peer 24a9ea5c15da49ae9108750c73143b20. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:44469: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260812 08:02:00.752861 13230 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer b3f8fc0e0371462da810db3e892efcf2 (127.0.186.65:44207): Couldn't send request to peer b3f8fc0e0371462da810db3e892efcf2. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:44207: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260812 08:02:00.928802 13233 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469): Couldn't send request to peer 24a9ea5c15da49ae9108750c73143b20. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:44469: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260812 08:02:00.978118 13230 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer b3f8fc0e0371462da810db3e892efcf2 (127.0.186.65:44207): Couldn't send request to peer b3f8fc0e0371462da810db3e892efcf2. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:44207: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260812 08:02:01.179433 13233 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469): Couldn't send request to peer 24a9ea5c15da49ae9108750c73143b20. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:44469: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260812 08:02:01.241463 13230 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer b3f8fc0e0371462da810db3e892efcf2 (127.0.186.65:44207): Couldn't send request to peer b3f8fc0e0371462da810db3e892efcf2. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:44207: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260812 08:02:01.439200 13233 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469): Couldn't send request to peer 24a9ea5c15da49ae9108750c73143b20. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:44469: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260812 08:02:01.512022 13230 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer b3f8fc0e0371462da810db3e892efcf2 (127.0.186.65:44207): Couldn't send request to peer b3f8fc0e0371462da810db3e892efcf2. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:44207: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260812 08:02:01.650810 13233 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469): Couldn't send request to peer 24a9ea5c15da49ae9108750c73143b20. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:44469: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260812 08:02:01.791182 13230 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer b3f8fc0e0371462da810db3e892efcf2 (127.0.186.65:44207): Couldn't send request to peer b3f8fc0e0371462da810db3e892efcf2. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:44207: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260812 08:02:01.898849 13233 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469): Couldn't send request to peer 24a9ea5c15da49ae9108750c73143b20. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:44469: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260812 08:02:02.054239 13230 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer b3f8fc0e0371462da810db3e892efcf2 (127.0.186.65:44207): Couldn't send request to peer b3f8fc0e0371462da810db3e892efcf2. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:44207: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260812 08:02:02.158021 13233 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469): Couldn't send request to peer 24a9ea5c15da49ae9108750c73143b20. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:44469: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260812 08:02:02.300506 13230 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer b3f8fc0e0371462da810db3e892efcf2 (127.0.186.65:44207): Couldn't send request to peer b3f8fc0e0371462da810db3e892efcf2. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:44207: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260812 08:02:02.412895 13233 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469): Couldn't send request to peer 24a9ea5c15da49ae9108750c73143b20. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:44469: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260812 08:02:02.543066 13230 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer b3f8fc0e0371462da810db3e892efcf2 (127.0.186.65:44207): Couldn't send request to peer b3f8fc0e0371462da810db3e892efcf2. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:44207: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260812 08:02:02.640477 13233 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469): Couldn't send request to peer 24a9ea5c15da49ae9108750c73143b20. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:44469: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260812 08:02:02.799285 13230 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer b3f8fc0e0371462da810db3e892efcf2 (127.0.186.65:44207): Couldn't send request to peer b3f8fc0e0371462da810db3e892efcf2. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:44207: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260812 08:02:02.884003 13233 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469): Couldn't send request to peer 24a9ea5c15da49ae9108750c73143b20. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:44469: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260812 08:02:03.027814 13230 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer b3f8fc0e0371462da810db3e892efcf2 (127.0.186.65:44207): Couldn't send request to peer b3f8fc0e0371462da810db3e892efcf2. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:44207: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260812 08:02:03.125735 13233 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469): Couldn't send request to peer 24a9ea5c15da49ae9108750c73143b20. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:44469: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260812 08:02:03.261261 13230 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer b3f8fc0e0371462da810db3e892efcf2 (127.0.186.65:44207): Couldn't send request to peer b3f8fc0e0371462da810db3e892efcf2. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:44207: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260812 08:02:03.405557 13233 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469): Couldn't send request to peer 24a9ea5c15da49ae9108750c73143b20. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:44469: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
W20260812 08:02:03.515177 13230 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer b3f8fc0e0371462da810db3e892efcf2 (127.0.186.65:44207): Couldn't send request to peer b3f8fc0e0371462da810db3e892efcf2. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:44207: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
W20260812 08:02:03.660115 13233 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469): Couldn't send request to peer 24a9ea5c15da49ae9108750c73143b20. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:44469: connect: Connection refused (error 111). This is attempt 96: this message will repeat every 5th retry.
W20260812 08:02:03.793871 13230 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer b3f8fc0e0371462da810db3e892efcf2 (127.0.186.65:44207): Couldn't send request to peer b3f8fc0e0371462da810db3e892efcf2. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:44207: connect: Connection refused (error 111). This is attempt 96: this message will repeat every 5th retry.
W20260812 08:02:03.909202 13233 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469): Couldn't send request to peer 24a9ea5c15da49ae9108750c73143b20. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:44469: connect: Connection refused (error 111). This is attempt 101: this message will repeat every 5th retry.
I20260812 08:02:04.002755   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:44207
--local_ip_for_outbound_sockets=127.0.186.65
--tserver_master_addrs=127.0.186.126:41315
--webserver_port=43183
--webserver_interface=127.0.186.65
--builtin_ntp_servers=127.0.186.84:33421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=50 with env {}
W20260812 08:02:04.009615 13233 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.67:44469: connect: Connection refused (error 111) [suppressed 201 similar messages]
W20260812 08:02:04.020423 13230 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer b3f8fc0e0371462da810db3e892efcf2 (127.0.186.65:44207): Couldn't send request to peer b3f8fc0e0371462da810db3e892efcf2. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:44207: connect: Connection refused (error 111). This is attempt 101: this message will repeat every 5th retry.
W20260812 08:02:04.126824 13423 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:04.127105 13423 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:04.127156 13423 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:04.130935 13423 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:04.131072 13423 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:02:04.135438 13423 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:33421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=50
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:44207
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=43183
--tserver_master_addrs=127.0.186.126:41315
--never_fsync=true
--heap_profile_path=/tmp/kudu.13423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
W20260812 08:02:04.136276 13233 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469): Couldn't send request to peer 24a9ea5c15da49ae9108750c73143b20. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:44469: connect: Connection refused (error 111). This is attempt 106: this message will repeat every 5th retry.
I20260812 08:02:04.136804 13423 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:04.137981 13423 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:02:04.140883 13423 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 25
W20260812 08:02:04.144764 13428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:04.144809 13431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:04.144906 13423 server_base.cc:1061] running on GCE node
W20260812 08:02:04.144825 13433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:04.145423 13423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:04.145994 13423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:04.147178 13423 hybrid_clock.cc:648] HybridClock initialized: now 1786521724147168 us; error 26 us; skew 500 ppm
I20260812 08:02:04.149109 13423 webserver.cc:533] Webserver started at http://127.0.186.65:43183/ using document root <none> and password file <none>
I20260812 08:02:04.149775 13423 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:04.149861 13423 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:04.153689 13423 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.001s	sys 0.003s
I20260812 08:02:04.155927 13439 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:04.157029 13423 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:04.157147 13423 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "b3f8fc0e0371462da810db3e892efcf2"
format_stamp: "Formatted at 2026-08-12 08:01:58 on dist-test-slave-49h3"
I20260812 08:02:04.157598 13423 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:04.175278 13423 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:04.175904 13423 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:04.176072 13423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:04.176648 13423 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:04.177923 13448 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:02:04.181324 13423 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:02:04.181383 13423 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:02:04.181468 13423 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:02:04.182888 13423 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:02:04.182931 13423 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
I20260812 08:02:04.183305 13448 tablet_bootstrap.cc:492] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2: Bootstrap starting.
I20260812 08:02:04.193964 13423 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:44207
I20260812 08:02:04.194013 13573 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:44207 every 8 connection(s)
I20260812 08:02:04.194939 13423 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:02:04.197170 13448 log.cc:826] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:04.199669 13448 tablet_bootstrap.cc:492] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:02:04.200032 13448 tablet_bootstrap.cc:492] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2: Bootstrap complete.
I20260812 08:02:04.200831   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 13423
I20260812 08:02:04.201176 13448 ts_tablet_manager.cc:1403] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2: Time spent bootstrapping tablet: real 0.018s	user 0.003s	sys 0.012s
I20260812 08:02:04.201340   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.67:44469
--local_ip_for_outbound_sockets=127.0.186.67
--tserver_master_addrs=127.0.186.126:41315
--webserver_port=33035
--webserver_interface=127.0.186.67
--builtin_ntp_servers=127.0.186.84:33421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=50 with env {}
I20260812 08:02:04.204043 13448 raft_consensus.cc:359] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f8fc0e0371462da810db3e892efcf2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44207 } } peers { permanent_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 46427 } } peers { permanent_uuid: "24a9ea5c15da49ae9108750c73143b20" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44469 } }
I20260812 08:02:04.204421 13448 raft_consensus.cc:740] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3f8fc0e0371462da810db3e892efcf2, State: Initialized, Role: FOLLOWER
I20260812 08:02:04.204886 13448 consensus_queue.cc:260] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f8fc0e0371462da810db3e892efcf2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44207 } } peers { permanent_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 46427 } } peers { permanent_uuid: "24a9ea5c15da49ae9108750c73143b20" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44469 } }
I20260812 08:02:04.205832 13448 ts_tablet_manager.cc:1434] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.003s
I20260812 08:02:04.208084 13574 heartbeater.cc:344] Connected to a master server at 127.0.186.126:41315
I20260812 08:02:04.208360 13574 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:04.208947 13574 heartbeater.cc:507] Master 127.0.186.126:41315 requested a full tablet report, sending...
I20260812 08:02:04.210960 12692 ts_manager.cc:194] Re-registered known tserver with Master: b3f8fc0e0371462da810db3e892efcf2 (127.0.186.65:44207)
I20260812 08:02:04.211728 12692 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.65:33265
I20260812 08:02:04.213521 13574 heartbeater.cc:499] Master 127.0.186.126:41315 was elected leader, sending a full tablet report...
I20260812 08:02:04.229065 13584 raft_consensus.cc:2955] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b912f3ad71d482e95cad27e3dd97cb6 (127.0.186.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b3f8fc0e0371462da810db3e892efcf2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44207 } } peers { permanent_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 46427 } } peers { permanent_uuid: "24a9ea5c15da49ae9108750c73143b20" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44469 } } peers { permanent_uuid: "2b912f3ad71d482e95cad27e3dd97cb6" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 38399 } attrs { } } }
I20260812 08:02:04.229732 13520 raft_consensus.cc:2955] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b912f3ad71d482e95cad27e3dd97cb6 (127.0.186.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b3f8fc0e0371462da810db3e892efcf2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44207 } } peers { permanent_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 46427 } } peers { permanent_uuid: "24a9ea5c15da49ae9108750c73143b20" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44469 } } peers { permanent_uuid: "2b912f3ad71d482e95cad27e3dd97cb6" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 38399 } attrs { } } }
I20260812 08:02:04.231284 13584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:53120 (request call id 3) took 5234 ms. Trace:
I20260812 08:02:04.231370 13584 rpcz_store.cc:276] 0812 08:01:58.997117 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:58.997209 (+    92us) service_pool.cc:224] Handling call
0812 08:01:58.998979 (+  1770us) log.cc:844] Serialized 493 byte log entry
0812 08:02:04.231266 (+5232287us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":61,"raft.queue_time_us":333,"raft.run_cpu_time_us":423,"raft.run_wall_time_us":605,"spinlock_wait_cycles":35456}
I20260812 08:02:04.232972 12692 catalog_manager.cc:5719] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P b3f8fc0e0371462da810db3e892efcf2 reported cstate change: config changed from index -1 to 2, NON_VOTER 2b912f3ad71d482e95cad27e3dd97cb6 (127.0.186.66) added. New cstate: current_term: 1 leader_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b3f8fc0e0371462da810db3e892efcf2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44207 } } peers { permanent_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 46427 } } peers { permanent_uuid: "24a9ea5c15da49ae9108750c73143b20" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44469 } } peers { permanent_uuid: "2b912f3ad71d482e95cad27e3dd97cb6" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 38399 } attrs { } } }
I20260812 08:02:04.233891 13000 raft_consensus.cc:2955] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 2b912f3ad71d482e95cad27e3dd97cb6 [term 1 LEARNER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b912f3ad71d482e95cad27e3dd97cb6 (127.0.186.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b3f8fc0e0371462da810db3e892efcf2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44207 } } peers { permanent_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 46427 } } peers { permanent_uuid: "24a9ea5c15da49ae9108750c73143b20" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44469 } } peers { permanent_uuid: "2b912f3ad71d482e95cad27e3dd97cb6" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 38399 } attrs { } } }
W20260812 08:02:04.321262 13579 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:04.321568 13579 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:04.321650 13579 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:04.325165 13579 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:04.325327 13579 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.67
I20260812 08:02:04.329888 13579 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:33421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=50
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.67:44469
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.186.67
--webserver_port=33035
--tserver_master_addrs=127.0.186.126:41315
--never_fsync=true
--heap_profile_path=/tmp/kudu.13579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:04.331202 13579 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:04.332540 13579 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:02:04.335983 13579 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 25
W20260812 08:02:04.340467 13597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:04.340467 13598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:04.340644 13579 server_base.cc:1061] running on GCE node
W20260812 08:02:04.340646 13604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:04.341192 13579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:04.341888 13579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:04.343103 13579 hybrid_clock.cc:648] HybridClock initialized: now 1786521724343073 us; error 42 us; skew 500 ppm
I20260812 08:02:04.345397 13579 webserver.cc:533] Webserver started at http://127.0.186.67:33035/ using document root <none> and password file <none>
I20260812 08:02:04.346174 13579 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:04.346269 13579 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:04.351318 13579 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.002s	sys 0.002s
I20260812 08:02:04.353849 13613 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:04.355225 13579 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260812 08:02:04.355396 13579 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "24a9ea5c15da49ae9108750c73143b20"
format_stamp: "Formatted at 2026-08-12 08:01:58 on dist-test-slave-49h3"
I20260812 08:02:04.355976 13579 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:04.373322 13579 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:04.374164 13579 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:04.374367 13579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:04.375008 13579 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:04.376318 13620 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:02:04.380880 13579 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:02:04.380966 13579 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.005s	user 0.001s	sys 0.000s
I20260812 08:02:04.381042 13579 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:02:04.382855 13579 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:02:04.382927 13579 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:02:04.383337 13620 tablet_bootstrap.cc:492] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 24a9ea5c15da49ae9108750c73143b20: Bootstrap starting.
I20260812 08:02:04.393620 13579 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.67:44469
I20260812 08:02:04.393666 13750 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.67:44469 every 8 connection(s)
I20260812 08:02:04.394683 13579 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:02:04.399667 13620 log.cc:826] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 24a9ea5c15da49ae9108750c73143b20: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:04.399786   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 13579
I20260812 08:02:04.401932   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 13423
I20260812 08:02:04.404614 13620 tablet_bootstrap.cc:492] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 24a9ea5c15da49ae9108750c73143b20: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:02:04.405102 13620 tablet_bootstrap.cc:492] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 24a9ea5c15da49ae9108750c73143b20: Bootstrap complete.
I20260812 08:02:04.406236 13620 ts_tablet_manager.cc:1403] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 24a9ea5c15da49ae9108750c73143b20: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.010s
I20260812 08:02:04.409282 13620 raft_consensus.cc:359] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 24a9ea5c15da49ae9108750c73143b20 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f8fc0e0371462da810db3e892efcf2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44207 } } peers { permanent_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 46427 } } peers { permanent_uuid: "24a9ea5c15da49ae9108750c73143b20" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44469 } }
I20260812 08:02:04.409843 13620 raft_consensus.cc:740] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 24a9ea5c15da49ae9108750c73143b20 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24a9ea5c15da49ae9108750c73143b20, State: Initialized, Role: FOLLOWER
I20260812 08:02:04.410496 13620 consensus_queue.cc:260] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 24a9ea5c15da49ae9108750c73143b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f8fc0e0371462da810db3e892efcf2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44207 } } peers { permanent_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 46427 } } peers { permanent_uuid: "24a9ea5c15da49ae9108750c73143b20" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44469 } }
I20260812 08:02:04.411732 13620 ts_tablet_manager.cc:1434] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 24a9ea5c15da49ae9108750c73143b20: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:02:04.414693 13751 heartbeater.cc:344] Connected to a master server at 127.0.186.126:41315
I20260812 08:02:04.414973 13751 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:04.415710 13751 heartbeater.cc:507] Master 127.0.186.126:41315 requested a full tablet report, sending...
I20260812 08:02:04.416100 13699 raft_consensus.cc:2955] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 24a9ea5c15da49ae9108750c73143b20 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b912f3ad71d482e95cad27e3dd97cb6 (127.0.186.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b3f8fc0e0371462da810db3e892efcf2" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44207 } } peers { permanent_uuid: "7dbf915c265f4d24b93e39bf4bafeb68" member_type: VOTER last_known_addr { host: "127.0.186.68" port: 46427 } } peers { permanent_uuid: "24a9ea5c15da49ae9108750c73143b20" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44469 } } peers { permanent_uuid: "2b912f3ad71d482e95cad27e3dd97cb6" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 38399 } attrs { } } }
I20260812 08:02:04.421190 12692 ts_manager.cc:194] Re-registered known tserver with Master: 24a9ea5c15da49ae9108750c73143b20 (127.0.186.67:44469)
I20260812 08:02:04.422099 12692 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.67:34131
I20260812 08:02:04.422430   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 12910
I20260812 08:02:04.427068 13751 heartbeater.cc:499] Master 127.0.186.126:41315 was elected leader, sending a full tablet report...
W20260812 08:02:04.431510 13230 consensus_peers.cc:597] T 1ca0bf4c951f4e71963efbc1ebd7e4e8 P 7dbf915c265f4d24b93e39bf4bafeb68 -> Peer b3f8fc0e0371462da810db3e892efcf2 (127.0.186.65:44207): Couldn't send request to peer b3f8fc0e0371462da810db3e892efcf2. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.65:44207: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 08:02:04.437194   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 13579
I20260812 08:02:04.446224   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 13213
I20260812 08:02:04.456768   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 12653
2026-08-12T08:02:04Z chronyd exiting
[       OK ] RaftConsensusNonVoterITest.NonVoterReplicasInConsensusQueue (6695 ms)
[----------] 13 tests from RaftConsensusNonVoterITest (115245 ms total)

[----------] 4 tests from IncompatibleReplicaReplacementSchemesITest
[ RUN      ] IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig/0
I20260812 08:02:04.484674   745 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:02:04.484723   745 ts_itest-base.cc:116] --------------
I20260812 08:02:04.484738   745 ts_itest-base.cc:117] 1 tablet servers
I20260812 08:02:04.484751   745 ts_itest-base.cc:118] 1 replicas per TS
I20260812 08:02:04.484764   745 ts_itest-base.cc:119] --------------
2026-08-12T08:02:04Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:02:04Z Disabled control of system clock
I20260812 08:02:04.493989   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:36459
--webserver_interface=127.0.186.126
--webserver_port=0
--builtin_ntp_servers=127.0.186.84:35245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:36459
--raft_prepare_replacement_before_eviction=false with env {}
W20260812 08:02:04.604586 13765 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:04.604822 13765 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:04.604862 13765 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:04.608256 13765 flags.cc:432] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20260812 08:02:04.608345 13765 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:02:04.608364 13765 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:04.608382 13765 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:02:04.608398 13765 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:02:04.612442 13765 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:35245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:36459
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:36459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.13765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:04.613660 13765 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:04.614841 13765 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:04.620198 13771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:04.620278 13765 server_base.cc:1061] running on GCE node
W20260812 08:02:04.620195 13773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:04.620203 13770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:04.620852 13765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:04.621757 13765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:04.622908 13765 hybrid_clock.cc:648] HybridClock initialized: now 1786521724622895 us; error 33 us; skew 500 ppm
I20260812 08:02:04.624600 13765 webserver.cc:533] Webserver started at http://127.0.186.126:39187/ using document root <none> and password file <none>
I20260812 08:02:04.625160 13765 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:04.625219 13765 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:04.625408 13765 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:04.626987 13765 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "cab8f5683392432d8bcc5c430012526b"
format_stamp: "Formatted at 2026-08-12 08:02:04 on dist-test-slave-49h3"
I20260812 08:02:04.627475 13765 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "cab8f5683392432d8bcc5c430012526b"
format_stamp: "Formatted at 2026-08-12 08:02:04 on dist-test-slave-49h3"
I20260812 08:02:04.631206 13765 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:02:04.633651 13780 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:04.634832 13765 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:02:04.634971 13765 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "cab8f5683392432d8bcc5c430012526b"
format_stamp: "Formatted at 2026-08-12 08:02:04 on dist-test-slave-49h3"
I20260812 08:02:04.635100 13765 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:04.658872 13765 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:04.659679 13765 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:04.659864 13765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:04.667666 13839 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:36459 every 8 connection(s)
I20260812 08:02:04.667681 13765 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:36459
I20260812 08:02:04.668768 13765 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:02:04.670113   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 13765
I20260812 08:02:04.670235   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:02:04.672850 13840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:04.677711 13840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b: Bootstrap starting.
I20260812 08:02:04.680202 13840 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:04.680925 13840 log.cc:826] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:04.682828 13840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b: No bootstrap required, opened a new log
I20260812 08:02:04.685810 13840 raft_consensus.cc:359] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab8f5683392432d8bcc5c430012526b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 36459 } }
I20260812 08:02:04.686017 13840 raft_consensus.cc:385] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:04.686103 13840 raft_consensus.cc:740] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cab8f5683392432d8bcc5c430012526b, State: Initialized, Role: FOLLOWER
I20260812 08:02:04.686511 13840 consensus_queue.cc:260] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab8f5683392432d8bcc5c430012526b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 36459 } }
I20260812 08:02:04.686640 13840 raft_consensus.cc:399] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:04.686703 13840 raft_consensus.cc:493] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:04.686813 13840 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:04.687641 13840 raft_consensus.cc:515] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab8f5683392432d8bcc5c430012526b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 36459 } }
I20260812 08:02:04.688023 13840 leader_election.cc:304] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cab8f5683392432d8bcc5c430012526b; no voters: 
I20260812 08:02:04.688355 13840 leader_election.cc:290] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:02:04.688458 13848 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:04.688681 13848 raft_consensus.cc:697] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [term 1 LEADER]: Becoming Leader. State: Replica: cab8f5683392432d8bcc5c430012526b, State: Running, Role: LEADER
I20260812 08:02:04.688962 13848 consensus_queue.cc:237] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab8f5683392432d8bcc5c430012526b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 36459 } }
I20260812 08:02:04.689513 13840 sys_catalog.cc:565] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:02:04.690521 13849 sys_catalog.cc:455] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cab8f5683392432d8bcc5c430012526b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab8f5683392432d8bcc5c430012526b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 36459 } } }
I20260812 08:02:04.690655 13849 sys_catalog.cc:458] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:04.690897 13851 sys_catalog.cc:455] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cab8f5683392432d8bcc5c430012526b. Latest consensus state: current_term: 1 leader_uuid: "cab8f5683392432d8bcc5c430012526b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab8f5683392432d8bcc5c430012526b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 36459 } } }
I20260812 08:02:04.690982 13851 sys_catalog.cc:458] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:04.691305 13858 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:02:04.694334 13858 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:02:04.699383 13858 catalog_manager.cc:1383] Generated new cluster ID: 24f0e495e4e0411fb2dc0494ebe766c4
I20260812 08:02:04.699440 13858 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:02:04.723954 13858 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:02:04.724819 13858 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:02:04.736348 13858 catalog_manager.cc:6092] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b: Generated new TSK 0
I20260812 08:02:04.737020 13858 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:02:04.748394   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:0
--local_ip_for_outbound_sockets=127.0.186.65
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:36459
--builtin_ntp_servers=127.0.186.84:35245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_incompatible_replica_management_is_fatal=false
--heartbeat_interval_ms=500
--raft_prepare_replacement_before_eviction=false with env {}
W20260812 08:02:04.862042 13870 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:04.862278 13870 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:04.862334 13870 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:04.865700 13870 flags.cc:432] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20260812 08:02:04.865772 13870 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:04.865861 13870 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:02:04.869827 13870 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:35245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=0
--heartbeat_incompatible_replica_management_is_fatal=false
--heartbeat_interval_ms=500
--tserver_master_addrs=127.0.186.126:36459
--never_fsync=true
--heap_profile_path=/tmp/kudu.13870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:04.871017 13870 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:04.872336 13870 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:04.879426 13878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:04.879426 13875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:04.879426 13876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:04.879653 13870 server_base.cc:1061] running on GCE node
I20260812 08:02:04.880107 13870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:04.880692 13870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:04.881839 13870 hybrid_clock.cc:648] HybridClock initialized: now 1786521724881824 us; error 25 us; skew 500 ppm
I20260812 08:02:04.883607 13870 webserver.cc:533] Webserver started at http://127.0.186.65:33109/ using document root <none> and password file <none>
I20260812 08:02:04.884188 13870 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:04.884259 13870 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:04.884490 13870 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:04.886286 13870 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "3e1cdd412cd24425a13a6a5b068bc2a5"
format_stamp: "Formatted at 2026-08-12 08:02:04 on dist-test-slave-49h3"
I20260812 08:02:04.886817 13870 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "3e1cdd412cd24425a13a6a5b068bc2a5"
format_stamp: "Formatted at 2026-08-12 08:02:04 on dist-test-slave-49h3"
I20260812 08:02:04.890502 13870 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.001s
I20260812 08:02:04.892995 13885 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:04.894186 13870 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 08:02:04.894304 13870 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "3e1cdd412cd24425a13a6a5b068bc2a5"
format_stamp: "Formatted at 2026-08-12 08:02:04 on dist-test-slave-49h3"
I20260812 08:02:04.894414 13870 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:04.922277 13870 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:04.922981 13870 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:04.923138 13870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:04.923691 13870 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:04.924702 13870 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:04.924751 13870 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:04.924793 13870 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:04.924811 13870 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:04.933779 13870 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:46427
I20260812 08:02:04.933828 14026 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:46427 every 8 connection(s)
I20260812 08:02:04.934746 13870 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:02:04.935434   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 13870
I20260812 08:02:04.935554   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:02:04.943421 14027 heartbeater.cc:344] Connected to a master server at 127.0.186.126:36459
I20260812 08:02:04.943758 14027 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:04.944355 14027 heartbeater.cc:507] Master 127.0.186.126:36459 requested a full tablet report, sending...
I20260812 08:02:04.946344 13797 ts_manager.cc:194] Registered new tserver with Master: 3e1cdd412cd24425a13a6a5b068bc2a5 (127.0.186.65:46427)
I20260812 08:02:04.947680 13797 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.65:43197
I20260812 08:02:04.950500   745 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260812 08:02:04.959789 13797 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34810:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
I20260812 08:02:04.976261 13943 tablet_service.cc:1511] Processing CreateTablet for tablet bc3c580a158748b7898ff8fe50047137 (DEFAULT_TABLE table=TestTable [id=d7d09ac0180f4f1eac667da4edc19e9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:04.977207 13943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc3c580a158748b7898ff8fe50047137. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:04.982120 14039 tablet_bootstrap.cc:492] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5: Bootstrap starting.
I20260812 08:02:04.984115 14039 tablet_bootstrap.cc:654] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:04.984864 14039 log.cc:826] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:04.986651 14039 tablet_bootstrap.cc:492] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5: No bootstrap required, opened a new log
I20260812 08:02:04.986828 14039 ts_tablet_manager.cc:1403] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.000s
I20260812 08:02:04.989099 14039 raft_consensus.cc:359] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e1cdd412cd24425a13a6a5b068bc2a5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 46427 } }
I20260812 08:02:04.989267 14039 raft_consensus.cc:385] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:04.989300 14039 raft_consensus.cc:740] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e1cdd412cd24425a13a6a5b068bc2a5, State: Initialized, Role: FOLLOWER
I20260812 08:02:04.989676 14039 consensus_queue.cc:260] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e1cdd412cd24425a13a6a5b068bc2a5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 46427 } }
I20260812 08:02:04.989795 14039 raft_consensus.cc:399] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:04.989838 14039 raft_consensus.cc:493] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:04.989919 14039 raft_consensus.cc:3060] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:04.990654 14039 raft_consensus.cc:515] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e1cdd412cd24425a13a6a5b068bc2a5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 46427 } }
I20260812 08:02:04.990967 14039 leader_election.cc:304] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e1cdd412cd24425a13a6a5b068bc2a5; no voters: 
I20260812 08:02:04.991282 14039 leader_election.cc:290] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:02:04.991389 14041 raft_consensus.cc:2804] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:04.991623 14041 raft_consensus.cc:697] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [term 1 LEADER]: Becoming Leader. State: Replica: 3e1cdd412cd24425a13a6a5b068bc2a5, State: Running, Role: LEADER
I20260812 08:02:04.991912 14027 heartbeater.cc:499] Master 127.0.186.126:36459 was elected leader, sending a full tablet report...
I20260812 08:02:04.991915 14041 consensus_queue.cc:237] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e1cdd412cd24425a13a6a5b068bc2a5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 46427 } }
I20260812 08:02:04.992141 14039 ts_tablet_manager.cc:1434] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260812 08:02:04.995023 13797 catalog_manager.cc:5719] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e1cdd412cd24425a13a6a5b068bc2a5 (127.0.186.65). New cstate: current_term: 1 leader_uuid: "3e1cdd412cd24425a13a6a5b068bc2a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e1cdd412cd24425a13a6a5b068bc2a5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 46427 } } }
I20260812 08:02:05.002653   745 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260812 08:02:05.004014   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 3e1cdd412cd24425a13a6a5b068bc2a5 to finish bootstrapping
I20260812 08:02:05.010681   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 13870
I20260812 08:02:05.023207   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 13765
I20260812 08:02:05.033815   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:36459
--webserver_interface=127.0.186.126
--webserver_port=39187
--builtin_ntp_servers=127.0.186.84:35245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:36459
--raft_prepare_replacement_before_eviction=false with env {}
W20260812 08:02:05.143563 14047 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:05.143811 14047 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:05.143857 14047 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:05.147228 14047 flags.cc:432] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20260812 08:02:05.147295 14047 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:02:05.147315 14047 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:05.147332 14047 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:02:05.147348 14047 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:02:05.151175 14047 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:35245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:36459
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:36459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=39187
--never_fsync=true
--heap_profile_path=/tmp/kudu.14047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:05.152273 14047 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:05.153369 14047 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 08:02:05.159041 14047 server_base.cc:1061] running on GCE node
W20260812 08:02:05.158934 14055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:05.158943 14053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:05.158938 14052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:05.159673 14047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:05.160559 14047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:05.161707 14047 hybrid_clock.cc:648] HybridClock initialized: now 1786521725161691 us; error 33 us; skew 500 ppm
I20260812 08:02:05.163324 14047 webserver.cc:533] Webserver started at http://127.0.186.126:39187/ using document root <none> and password file <none>
I20260812 08:02:05.163887 14047 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:05.163967 14047 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:05.167260 14047 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.000s	sys 0.003s
I20260812 08:02:05.169085 14062 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:05.170066 14047 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.001s
I20260812 08:02:05.170177 14047 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "cab8f5683392432d8bcc5c430012526b"
format_stamp: "Formatted at 2026-08-12 08:02:04 on dist-test-slave-49h3"
I20260812 08:02:05.170549 14047 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:05.190150 14047 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:05.190961 14047 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:05.191150 14047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:05.198729 14123 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:36459 every 8 connection(s)
I20260812 08:02:05.198742 14047 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:36459
I20260812 08:02:05.199829 14047 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:02:05.200181   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 14047
I20260812 08:02:05.200691   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:46427
--local_ip_for_outbound_sockets=127.0.186.65
--tserver_master_addrs=127.0.186.126:36459
--webserver_port=33109
--webserver_interface=127.0.186.65
--builtin_ntp_servers=127.0.186.84:35245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_incompatible_replica_management_is_fatal=false
--heartbeat_interval_ms=500
--raft_prepare_replacement_before_eviction=false
--raft_prepare_replacement_before_eviction=true with env {}
I20260812 08:02:05.204435 14124 sys_catalog.cc:263] Verifying existing consensus state
I20260812 08:02:05.205981 14124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b: Bootstrap starting.
I20260812 08:02:05.220165 14124 log.cc:826] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:05.227847 14124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=5 ignored=0} mutations{seen=2 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:02:05.228358 14124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b: Bootstrap complete.
I20260812 08:02:05.232018 14124 raft_consensus.cc:359] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab8f5683392432d8bcc5c430012526b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 36459 } }
I20260812 08:02:05.232393 14124 raft_consensus.cc:740] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cab8f5683392432d8bcc5c430012526b, State: Initialized, Role: FOLLOWER
I20260812 08:02:05.232939 14124 consensus_queue.cc:260] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab8f5683392432d8bcc5c430012526b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 36459 } }
I20260812 08:02:05.233065 14124 raft_consensus.cc:399] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:05.233139 14124 raft_consensus.cc:493] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:05.233255 14124 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:02:05.234886 14124 raft_consensus.cc:515] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab8f5683392432d8bcc5c430012526b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 36459 } }
I20260812 08:02:05.235296 14124 leader_election.cc:304] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cab8f5683392432d8bcc5c430012526b; no voters: 
I20260812 08:02:05.235738 14124 leader_election.cc:290] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [CANDIDATE]: Term 2 election: Requested vote from peers 
I20260812 08:02:05.235906 14129 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:02:05.236133 14129 raft_consensus.cc:697] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [term 2 LEADER]: Becoming Leader. State: Replica: cab8f5683392432d8bcc5c430012526b, State: Running, Role: LEADER
I20260812 08:02:05.236507 14129 consensus_queue.cc:237] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab8f5683392432d8bcc5c430012526b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 36459 } }
I20260812 08:02:05.236992 14124 sys_catalog.cc:565] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:02:05.238008 14131 sys_catalog.cc:455] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cab8f5683392432d8bcc5c430012526b. Latest consensus state: current_term: 2 leader_uuid: "cab8f5683392432d8bcc5c430012526b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab8f5683392432d8bcc5c430012526b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 36459 } } }
I20260812 08:02:05.238131 14131 sys_catalog.cc:458] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:05.238847 14130 sys_catalog.cc:455] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "cab8f5683392432d8bcc5c430012526b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab8f5683392432d8bcc5c430012526b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 36459 } } }
I20260812 08:02:05.238940 14130 sys_catalog.cc:458] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:05.239217 14139 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:02:05.242965 14139 catalog_manager.cc:679] Loaded metadata for table TestTable [id=d7d09ac0180f4f1eac667da4edc19e9c]
I20260812 08:02:05.244236 14139 tablet_loader.cc:96] loaded metadata for tablet bc3c580a158748b7898ff8fe50047137 (table TestTable [id=d7d09ac0180f4f1eac667da4edc19e9c])
I20260812 08:02:05.244390 14139 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:02:05.245002 14139 catalog_manager.cc:1295] Loaded cluster ID: 24f0e495e4e0411fb2dc0494ebe766c4
I20260812 08:02:05.245059 14139 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:02:05.247540 14139 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:02:05.248245 14139 catalog_manager.cc:6103] T 00000000000000000000000000000000 P cab8f5683392432d8bcc5c430012526b: Loaded TSK: 0
I20260812 08:02:05.249205 14139 catalog_manager.cc:1550] Initializing in-progress tserver states...
W20260812 08:02:05.319847 14126 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:05.320068 14126 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:05.320123 14126 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:05.323508 14126 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:05.323632 14126 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:02:05.327456 14126 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:35245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:46427
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=33109
--heartbeat_incompatible_replica_management_is_fatal=false
--heartbeat_interval_ms=500
--tserver_master_addrs=127.0.186.126:36459
--never_fsync=true
--heap_profile_path=/tmp/kudu.14126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:05.328565 14126 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:05.329806 14126 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:05.336287 14155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:05.336300 14157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:05.336328 14159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:05.336830 14126 server_base.cc:1061] running on GCE node
I20260812 08:02:05.337185 14126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:05.337750 14126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:05.338927 14126 hybrid_clock.cc:648] HybridClock initialized: now 1786521725338910 us; error 36 us; skew 500 ppm
I20260812 08:02:05.340646 14126 webserver.cc:533] Webserver started at http://127.0.186.65:33109/ using document root <none> and password file <none>
I20260812 08:02:05.341214 14126 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:05.341287 14126 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:05.344753 14126 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:05.346817 14166 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:05.347872 14126 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:02:05.347980 14126 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "3e1cdd412cd24425a13a6a5b068bc2a5"
format_stamp: "Formatted at 2026-08-12 08:02:04 on dist-test-slave-49h3"
I20260812 08:02:05.348361 14126 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:05.381816 14126 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:05.382537 14126 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:05.382727 14126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:05.383347 14126 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:05.384768 14176 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:02:05.389288 14126 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:02:05.389349 14126 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.005s	user 0.001s	sys 0.000s
I20260812 08:02:05.389416 14126 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:02:05.390879 14126 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:02:05.390923 14126 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:02:05.391346 14176 tablet_bootstrap.cc:492] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5: Bootstrap starting.
I20260812 08:02:05.401655 14126 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:46427
I20260812 08:02:05.401847 14299 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:46427 every 8 connection(s)
I20260812 08:02:05.402788 14126 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:02:05.408107   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 14126
I20260812 08:02:05.408481 14176 log.cc:826] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:05.411134 14176 tablet_bootstrap.cc:492] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:02:05.411482 14176 tablet_bootstrap.cc:492] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5: Bootstrap complete.
I20260812 08:02:05.412498 14176 ts_tablet_manager.cc:1403] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5: Time spent bootstrapping tablet: real 0.021s	user 0.005s	sys 0.013s
I20260812 08:02:05.414713 14300 heartbeater.cc:344] Connected to a master server at 127.0.186.126:36459
I20260812 08:02:05.414973 14300 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:05.415526 14300 heartbeater.cc:507] Master 127.0.186.126:36459 requested a full tablet report, sending...
I20260812 08:02:05.415437 14176 raft_consensus.cc:359] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e1cdd412cd24425a13a6a5b068bc2a5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 46427 } }
I20260812 08:02:05.415808 14176 raft_consensus.cc:740] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e1cdd412cd24425a13a6a5b068bc2a5, State: Initialized, Role: FOLLOWER
I20260812 08:02:05.416254 14176 consensus_queue.cc:260] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e1cdd412cd24425a13a6a5b068bc2a5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 46427 } }
I20260812 08:02:05.416390 14176 raft_consensus.cc:399] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:05.416501 14176 raft_consensus.cc:493] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:05.416651 14176 raft_consensus.cc:3060] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:02:05.418460 14176 raft_consensus.cc:515] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e1cdd412cd24425a13a6a5b068bc2a5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 46427 } }
W20260812 08:02:05.418781 14079 master_service.cc:413] Configuration error: replica replacement scheme (PREPARE_REPLACEMENT_BEFORE_EVICTION) of the tablet server 3e1cdd412cd24425a13a6a5b068bc2a5 at 127.0.186.65:38113 differs from the catalog manager's (EVICT_FIRST); they must be run with the same scheme (controlled by the --raft_prepare_replacement_before_eviction flag)
I20260812 08:02:05.418809 14176 leader_election.cc:304] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e1cdd412cd24425a13a6a5b068bc2a5; no voters: 
I20260812 08:02:05.419123 14176 leader_election.cc:290] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [CANDIDATE]: Term 2 election: Requested vote from peers 
I20260812 08:02:05.419265 14306 raft_consensus.cc:2804] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:02:05.419483 14306 raft_consensus.cc:697] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [term 2 LEADER]: Becoming Leader. State: Replica: 3e1cdd412cd24425a13a6a5b068bc2a5, State: Running, Role: LEADER
I20260812 08:02:05.419797 14306 consensus_queue.cc:237] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e1cdd412cd24425a13a6a5b068bc2a5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 46427 } }
I20260812 08:02:05.420050 14176 ts_tablet_manager.cc:1434] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5: Time spent starting tablet: real 0.007s	user 0.010s	sys 0.000s
W20260812 08:02:05.420064 14300 heartbeater.cc:646] Failed to heartbeat to 127.0.186.126:36459 (0 consecutive failures): Configuration error: replica replacement scheme (PREPARE_REPLACEMENT_BEFORE_EVICTION) of the tablet server 3e1cdd412cd24425a13a6a5b068bc2a5 at 127.0.186.65:38113 differs from the catalog manager's (EVICT_FIRST); they must be run with the same scheme (controlled by the --raft_prepare_replacement_before_eviction flag)
E20260812 08:02:05.420166 14300 heartbeater.cc:687] master detected incompatibility: Configuration error: replica replacement scheme (PREPARE_REPLACEMENT_BEFORE_EVICTION) of the tablet server 3e1cdd412cd24425a13a6a5b068bc2a5 at 127.0.186.65:38113 differs from the catalog manager's (EVICT_FIRST); they must be run with the same scheme (controlled by the --raft_prepare_replacement_before_eviction flag)
I20260812 08:02:05.420254 14300 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:05.420481 14300 heartbeater.cc:507] Master 127.0.186.126:36459 requested a full tablet report, sending...
W20260812 08:02:05.421967 14079 master_service.cc:413] Configuration error: replica replacement scheme (PREPARE_REPLACEMENT_BEFORE_EVICTION) of the tablet server 3e1cdd412cd24425a13a6a5b068bc2a5 at 127.0.186.65:38113 differs from the catalog manager's (EVICT_FIRST); they must be run with the same scheme (controlled by the --raft_prepare_replacement_before_eviction flag)
I20260812 08:02:05.422425 14300 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:05.422649 14300 heartbeater.cc:507] Master 127.0.186.126:36459 requested a full tablet report, sending...
W20260812 08:02:05.423413 14079 master_service.cc:413] Configuration error: replica replacement scheme (PREPARE_REPLACEMENT_BEFORE_EVICTION) of the tablet server 3e1cdd412cd24425a13a6a5b068bc2a5 at 127.0.186.65:38113 differs from the catalog manager's (EVICT_FIRST); they must be run with the same scheme (controlled by the --raft_prepare_replacement_before_eviction flag)
W20260812 08:02:05.423781 14300 heartbeater.cc:412] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 08:02:05.924021 14300 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:05.924425 14300 heartbeater.cc:507] Master 127.0.186.126:36459 requested a full tablet report, sending...
W20260812 08:02:05.925729 14079 master_service.cc:413] Configuration error: replica replacement scheme (PREPARE_REPLACEMENT_BEFORE_EVICTION) of the tablet server 3e1cdd412cd24425a13a6a5b068bc2a5 at 127.0.186.65:38113 differs from the catalog manager's (EVICT_FIRST); they must be run with the same scheme (controlled by the --raft_prepare_replacement_before_eviction flag)
I20260812 08:02:06.426479 14300 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:06.426893 14300 heartbeater.cc:507] Master 127.0.186.126:36459 requested a full tablet report, sending...
W20260812 08:02:06.428175 14079 master_service.cc:413] Configuration error: replica replacement scheme (PREPARE_REPLACEMENT_BEFORE_EVICTION) of the tablet server 3e1cdd412cd24425a13a6a5b068bc2a5 at 127.0.186.65:38113 differs from the catalog manager's (EVICT_FIRST); they must be run with the same scheme (controlled by the --raft_prepare_replacement_before_eviction flag)
I20260812 08:02:06.913865   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 14126
I20260812 08:02:06.924901   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:46427
--local_ip_for_outbound_sockets=127.0.186.65
--tserver_master_addrs=127.0.186.126:36459
--webserver_port=33109
--webserver_interface=127.0.186.65
--builtin_ntp_servers=127.0.186.84:35245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_incompatible_replica_management_is_fatal=false
--heartbeat_interval_ms=500
--raft_prepare_replacement_before_eviction=false
--heartbeat_inject_required_feature_flag=999 with env {}
W20260812 08:02:07.038143 14314 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:07.038404 14314 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:07.038451 14314 flags.cc:432] Enabled unsafe flag: --heartbeat_inject_required_feature_flag=999
W20260812 08:02:07.038475 14314 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:07.042066 14314 flags.cc:432] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20260812 08:02:07.042142 14314 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:07.042224 14314 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:02:07.046571 14314 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:35245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:46427
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=33109
--heartbeat_incompatible_replica_management_is_fatal=false
--heartbeat_inject_required_feature_flag=999
--heartbeat_interval_ms=500
--tserver_master_addrs=127.0.186.126:36459
--never_fsync=true
--heap_profile_path=/tmp/kudu.14314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:07.047883 14314 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:07.049096 14314 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:07.056373 14324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:07.056368 14326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:07.056474 14314 server_base.cc:1061] running on GCE node
W20260812 08:02:07.056368 14323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:07.056988 14314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:07.057664 14314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:07.058820 14314 hybrid_clock.cc:648] HybridClock initialized: now 1786521727058807 us; error 32 us; skew 500 ppm
I20260812 08:02:07.060631 14314 webserver.cc:533] Webserver started at http://127.0.186.65:33109/ using document root <none> and password file <none>
I20260812 08:02:07.061255 14314 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:07.061318 14314 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:07.064819 14314 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:07.066797 14336 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:07.067803 14314 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:02:07.067911 14314 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "3e1cdd412cd24425a13a6a5b068bc2a5"
format_stamp: "Formatted at 2026-08-12 08:02:04 on dist-test-slave-49h3"
I20260812 08:02:07.068316 14314 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:07.081161 14314 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:07.081892 14314 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:07.082072 14314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:07.082719 14314 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:07.084141 14344 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:02:07.088198 14314 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:02:07.088303 14314 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.001s	sys 0.000s
I20260812 08:02:07.088393 14314 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:02:07.090188 14314 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:02:07.090257 14314 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:02:07.090610 14344 tablet_bootstrap.cc:492] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5: Bootstrap starting.
I20260812 08:02:07.101089 14314 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:46427
I20260812 08:02:07.101212 14469 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:46427 every 8 connection(s)
I20260812 08:02:07.102288 14314 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:02:07.106670 14344 log.cc:826] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:07.109345 14344 tablet_bootstrap.cc:492] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:02:07.109737 14344 tablet_bootstrap.cc:492] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5: Bootstrap complete.
I20260812 08:02:07.110917 14344 ts_tablet_manager.cc:1403] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.008s
I20260812 08:02:07.111101 14470 heartbeater.cc:344] Connected to a master server at 127.0.186.126:36459
I20260812 08:02:07.111366 14470 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:07.111855   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 14314
I20260812 08:02:07.112020 14470 heartbeater.cc:507] Master 127.0.186.126:36459 requested a full tablet report, sending...
W20260812 08:02:07.113981 14063 messenger.cc:376] Unable to handle RPC call: Not implemented: call requires unsupported application feature flags: 999
I20260812 08:02:07.114038 14344 raft_consensus.cc:359] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e1cdd412cd24425a13a6a5b068bc2a5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 46427 } }
I20260812 08:02:07.114427 14344 raft_consensus.cc:740] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e1cdd412cd24425a13a6a5b068bc2a5, State: Initialized, Role: FOLLOWER
W20260812 08:02:07.114640 14470 heartbeater.cc:646] Failed to heartbeat to 127.0.186.126:36459 (0 consecutive failures): Remote error: Failed to send heartbeat to master: unsupported feature flags
I20260812 08:02:07.114936 14344 consensus_queue.cc:260] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e1cdd412cd24425a13a6a5b068bc2a5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 46427 } }
I20260812 08:02:07.115087 14344 raft_consensus.cc:399] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:07.115149 14344 raft_consensus.cc:493] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:07.115263 14344 raft_consensus.cc:3060] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [term 2 FOLLOWER]: Advancing to term 3
E20260812 08:02:07.116390 14470 heartbeater.cc:687] master does not support required feature flag(s) : Remote error: Failed to send heartbeat to master: unsupported feature flags
I20260812 08:02:07.116506 14470 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:07.116768 14470 heartbeater.cc:507] Master 127.0.186.126:36459 requested a full tablet report, sending...
I20260812 08:02:07.117588 14344 raft_consensus.cc:515] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e1cdd412cd24425a13a6a5b068bc2a5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 46427 } }
I20260812 08:02:07.117985 14344 leader_election.cc:304] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e1cdd412cd24425a13a6a5b068bc2a5; no voters: 
I20260812 08:02:07.118407 14344 leader_election.cc:290] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [CANDIDATE]: Term 3 election: Requested vote from peers 
I20260812 08:02:07.118525 14475 raft_consensus.cc:2804] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 08:02:07.118736 14475 raft_consensus.cc:697] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [term 3 LEADER]: Becoming Leader. State: Replica: 3e1cdd412cd24425a13a6a5b068bc2a5, State: Running, Role: LEADER
I20260812 08:02:07.119102 14475 consensus_queue.cc:237] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e1cdd412cd24425a13a6a5b068bc2a5" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 46427 } }
W20260812 08:02:07.119240 14063 messenger.cc:376] Unable to handle RPC call: Not implemented: call requires unsupported application feature flags: 999
I20260812 08:02:07.119488 14344 ts_tablet_manager.cc:1434] T bc3c580a158748b7898ff8fe50047137 P 3e1cdd412cd24425a13a6a5b068bc2a5: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.004s
I20260812 08:02:07.119674 14470 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:07.119933 14470 heartbeater.cc:507] Master 127.0.186.126:36459 requested a full tablet report, sending...
W20260812 08:02:07.120920 14063 messenger.cc:376] Unable to handle RPC call: Not implemented: call requires unsupported application feature flags: 999
W20260812 08:02:07.121287 14470 heartbeater.cc:412] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 08:02:07.121367 14470 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:07.121596 14470 heartbeater.cc:507] Master 127.0.186.126:36459 requested a full tablet report, sending...
W20260812 08:02:07.122156 14063 messenger.cc:376] Unable to handle RPC call: Not implemented: call requires unsupported application feature flags: 999
I20260812 08:02:07.622577 14470 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:07.622905 14470 heartbeater.cc:507] Master 127.0.186.126:36459 requested a full tablet report, sending...
W20260812 08:02:07.623819 14063 messenger.cc:376] Unable to handle RPC call: Not implemented: call requires unsupported application feature flags: 999
I20260812 08:02:08.124394 14470 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:08.124830 14470 heartbeater.cc:507] Master 127.0.186.126:36459 requested a full tablet report, sending...
W20260812 08:02:08.125871 14063 messenger.cc:376] Unable to handle RPC call: Not implemented: call requires unsupported application feature flags: 999
I20260812 08:02:08.613869   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 14314
I20260812 08:02:08.624292   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 14047
2026-08-12T08:02:08Z chronyd exiting
[       OK ] IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig/0 (4164 ms)
[ RUN      ] IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig/1
I20260812 08:02:08.649617   745 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:02:08.649669   745 ts_itest-base.cc:116] --------------
I20260812 08:02:08.649679   745 ts_itest-base.cc:117] 1 tablet servers
I20260812 08:02:08.649689   745 ts_itest-base.cc:118] 1 replicas per TS
I20260812 08:02:08.649698   745 ts_itest-base.cc:119] --------------
2026-08-12T08:02:08Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:02:08Z Disabled control of system clock
I20260812 08:02:08.659463   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:42271
--webserver_interface=127.0.186.126
--webserver_port=0
--builtin_ntp_servers=127.0.186.84:40929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:42271
--raft_prepare_replacement_before_eviction=true with env {}
W20260812 08:02:08.764001 14490 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:08.764262 14490 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:08.764320 14490 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:08.767890 14490 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:02:08.767988 14490 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:08.768018 14490 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:02:08.768049 14490 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:02:08.772272 14490 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:40929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:42271
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:42271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.14490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:08.773610 14490 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:08.774837 14490 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:08.780402 14499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:08.780396 14495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:08.780539 14490 server_base.cc:1061] running on GCE node
W20260812 08:02:08.780454 14496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:08.781121 14490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:08.782076 14490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:08.783234 14490 hybrid_clock.cc:648] HybridClock initialized: now 1786521728783216 us; error 33 us; skew 500 ppm
I20260812 08:02:08.784976 14490 webserver.cc:533] Webserver started at http://127.0.186.126:35793/ using document root <none> and password file <none>
I20260812 08:02:08.785593 14490 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:08.785652 14490 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:08.785827 14490 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:08.787377 14490 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "889ab14631a346f6b5b07e0fc7269496"
format_stamp: "Formatted at 2026-08-12 08:02:08 on dist-test-slave-49h3"
I20260812 08:02:08.787813 14490 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "889ab14631a346f6b5b07e0fc7269496"
format_stamp: "Formatted at 2026-08-12 08:02:08 on dist-test-slave-49h3"
I20260812 08:02:08.791096 14490 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:02:08.793212 14505 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:08.794211 14490 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:08.794323 14490 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "889ab14631a346f6b5b07e0fc7269496"
format_stamp: "Formatted at 2026-08-12 08:02:08 on dist-test-slave-49h3"
I20260812 08:02:08.794409 14490 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:08.818611 14490 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:08.819332 14490 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:08.819510 14490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:08.826932 14570 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:42271 every 8 connection(s)
I20260812 08:02:08.826933 14490 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:42271
I20260812 08:02:08.828009 14490 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:02:08.830900 14571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:08.835222   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 14490
I20260812 08:02:08.835345   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:02:08.836131 14571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496: Bootstrap starting.
I20260812 08:02:08.838471 14571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:08.839354 14571 log.cc:826] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:08.841253 14571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496: No bootstrap required, opened a new log
I20260812 08:02:08.844321 14571 raft_consensus.cc:359] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "889ab14631a346f6b5b07e0fc7269496" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 42271 } }
I20260812 08:02:08.844555 14571 raft_consensus.cc:385] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:08.844640 14571 raft_consensus.cc:740] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 889ab14631a346f6b5b07e0fc7269496, State: Initialized, Role: FOLLOWER
I20260812 08:02:08.845058 14571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "889ab14631a346f6b5b07e0fc7269496" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 42271 } }
I20260812 08:02:08.845196 14571 raft_consensus.cc:399] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:08.845259 14571 raft_consensus.cc:493] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:08.845367 14571 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:08.846190 14571 raft_consensus.cc:515] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "889ab14631a346f6b5b07e0fc7269496" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 42271 } }
I20260812 08:02:08.846515 14571 leader_election.cc:304] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 889ab14631a346f6b5b07e0fc7269496; no voters: 
I20260812 08:02:08.846802 14571 leader_election.cc:290] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:02:08.846936 14577 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:08.847175 14577 raft_consensus.cc:697] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [term 1 LEADER]: Becoming Leader. State: Replica: 889ab14631a346f6b5b07e0fc7269496, State: Running, Role: LEADER
I20260812 08:02:08.847541 14577 consensus_queue.cc:237] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "889ab14631a346f6b5b07e0fc7269496" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 42271 } }
I20260812 08:02:08.847967 14571 sys_catalog.cc:565] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:02:08.849349 14580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 889ab14631a346f6b5b07e0fc7269496. Latest consensus state: current_term: 1 leader_uuid: "889ab14631a346f6b5b07e0fc7269496" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "889ab14631a346f6b5b07e0fc7269496" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 42271 } } }
I20260812 08:02:08.849375 14578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "889ab14631a346f6b5b07e0fc7269496" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "889ab14631a346f6b5b07e0fc7269496" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 42271 } } }
I20260812 08:02:08.849538 14580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:08.849550 14578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:08.850370 14592 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:02:08.853005 14592 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:02:08.858474 14592 catalog_manager.cc:1383] Generated new cluster ID: 9736a7eb7359475cb2f735ff876e476e
I20260812 08:02:08.858544 14592 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:02:08.870417 14592 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:02:08.871587 14592 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:02:08.887861 14592 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496: Generated new TSK 0
I20260812 08:02:08.888510 14592 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:02:08.901480   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:0
--local_ip_for_outbound_sockets=127.0.186.65
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:42271
--builtin_ntp_servers=127.0.186.84:40929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_incompatible_replica_management_is_fatal=false
--heartbeat_interval_ms=500
--raft_prepare_replacement_before_eviction=true with env {}
W20260812 08:02:09.027092 14598 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:09.027320 14598 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:09.027380 14598 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:09.030952 14598 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:09.031121 14598 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:02:09.035286 14598 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:40929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=0
--heartbeat_incompatible_replica_management_is_fatal=false
--heartbeat_interval_ms=500
--tserver_master_addrs=127.0.186.126:42271
--never_fsync=true
--heap_profile_path=/tmp/kudu.14598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:09.036418 14598 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:09.037652 14598 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:09.043946 14608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:09.043946 14605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:09.043967 14606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:09.044162 14598 server_base.cc:1061] running on GCE node
I20260812 08:02:09.044490 14598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:09.045027 14598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:09.046173 14598 hybrid_clock.cc:648] HybridClock initialized: now 1786521729046164 us; error 26 us; skew 500 ppm
I20260812 08:02:09.047821 14598 webserver.cc:533] Webserver started at http://127.0.186.65:40735/ using document root <none> and password file <none>
I20260812 08:02:09.048357 14598 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:09.048416 14598 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:09.048607 14598 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:09.050256 14598 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "622fc79708ae4a65b254cce1c881f0c1"
format_stamp: "Formatted at 2026-08-12 08:02:09 on dist-test-slave-49h3"
I20260812 08:02:09.050719 14598 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "622fc79708ae4a65b254cce1c881f0c1"
format_stamp: "Formatted at 2026-08-12 08:02:09 on dist-test-slave-49h3"
I20260812 08:02:09.053911 14598 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:02:09.056048 14616 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:09.057117 14598 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:09.057227 14598 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "622fc79708ae4a65b254cce1c881f0c1"
format_stamp: "Formatted at 2026-08-12 08:02:09 on dist-test-slave-49h3"
I20260812 08:02:09.057317 14598 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:09.066833 14598 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:09.067466 14598 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:09.067643 14598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:09.068243 14598 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:09.069257 14598 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:09.069306 14598 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:09.069358 14598 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:09.069386 14598 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:09.078445 14598 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:44551
I20260812 08:02:09.078482 14749 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:44551 every 8 connection(s)
I20260812 08:02:09.079399 14598 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:02:09.087493 14750 heartbeater.cc:344] Connected to a master server at 127.0.186.126:42271
I20260812 08:02:09.087743 14750 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:09.088254 14750 heartbeater.cc:507] Master 127.0.186.126:42271 requested a full tablet report, sending...
I20260812 08:02:09.088552   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 14598
I20260812 08:02:09.088651   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:02:09.089787 14525 ts_manager.cc:194] Registered new tserver with Master: 622fc79708ae4a65b254cce1c881f0c1 (127.0.186.65:44551)
I20260812 08:02:09.091027   745 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260812 08:02:09.091230 14525 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.65:48759
I20260812 08:02:09.099375 14525 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42780:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
I20260812 08:02:09.116598 14672 tablet_service.cc:1511] Processing CreateTablet for tablet 5f11401241104137928a1f9de1b19521 (DEFAULT_TABLE table=TestTable [id=25e36a2dfacf409b82aea43e952dcb03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:09.117659 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f11401241104137928a1f9de1b19521. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:09.122792 14762 tablet_bootstrap.cc:492] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1: Bootstrap starting.
I20260812 08:02:09.124935 14762 tablet_bootstrap.cc:654] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:09.125716 14762 log.cc:826] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:09.127523 14762 tablet_bootstrap.cc:492] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1: No bootstrap required, opened a new log
I20260812 08:02:09.127725 14762 ts_tablet_manager.cc:1403] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:02:09.130331 14762 raft_consensus.cc:359] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622fc79708ae4a65b254cce1c881f0c1" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44551 } }
I20260812 08:02:09.130532 14762 raft_consensus.cc:385] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:09.130590 14762 raft_consensus.cc:740] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 622fc79708ae4a65b254cce1c881f0c1, State: Initialized, Role: FOLLOWER
I20260812 08:02:09.131004 14762 consensus_queue.cc:260] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622fc79708ae4a65b254cce1c881f0c1" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44551 } }
I20260812 08:02:09.131142 14762 raft_consensus.cc:399] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:09.131209 14762 raft_consensus.cc:493] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:09.131322 14762 raft_consensus.cc:3060] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:09.132247 14762 raft_consensus.cc:515] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622fc79708ae4a65b254cce1c881f0c1" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44551 } }
I20260812 08:02:09.132639 14762 leader_election.cc:304] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 622fc79708ae4a65b254cce1c881f0c1; no voters: 
I20260812 08:02:09.132970 14762 leader_election.cc:290] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:02:09.133106 14765 raft_consensus.cc:2804] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:09.133313 14765 raft_consensus.cc:697] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [term 1 LEADER]: Becoming Leader. State: Replica: 622fc79708ae4a65b254cce1c881f0c1, State: Running, Role: LEADER
I20260812 08:02:09.133597 14750 heartbeater.cc:499] Master 127.0.186.126:42271 was elected leader, sending a full tablet report...
I20260812 08:02:09.133669 14765 consensus_queue.cc:237] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622fc79708ae4a65b254cce1c881f0c1" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44551 } }
I20260812 08:02:09.133878 14762 ts_tablet_manager.cc:1434] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260812 08:02:09.137208 14525 catalog_manager.cc:5719] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 622fc79708ae4a65b254cce1c881f0c1 (127.0.186.65). New cstate: current_term: 1 leader_uuid: "622fc79708ae4a65b254cce1c881f0c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622fc79708ae4a65b254cce1c881f0c1" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44551 } health_report { overall_health: HEALTHY } } }
I20260812 08:02:09.143435   745 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260812 08:02:09.144716   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 622fc79708ae4a65b254cce1c881f0c1 to finish bootstrapping
I20260812 08:02:09.150735   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 14598
I20260812 08:02:09.163899   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 14490
I20260812 08:02:09.173271   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:42271
--webserver_interface=127.0.186.126
--webserver_port=35793
--builtin_ntp_servers=127.0.186.84:40929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:42271
--raft_prepare_replacement_before_eviction=true with env {}
W20260812 08:02:09.282480 14770 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:09.282701 14770 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:09.282740 14770 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:09.285996 14770 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:02:09.286072 14770 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:09.286092 14770 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:02:09.286111 14770 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:02:09.289872 14770 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:40929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:42271
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:42271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=35793
--never_fsync=true
--heap_profile_path=/tmp/kudu.14770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:09.290921 14770 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:09.292050 14770 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:09.297827 14776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:09.297950 14778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:09.298179 14775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:09.298242 14770 server_base.cc:1061] running on GCE node
I20260812 08:02:09.298859 14770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:09.299880 14770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:09.301034 14770 hybrid_clock.cc:648] HybridClock initialized: now 1786521729301025 us; error 41 us; skew 500 ppm
I20260812 08:02:09.303109 14770 webserver.cc:533] Webserver started at http://127.0.186.126:35793/ using document root <none> and password file <none>
I20260812 08:02:09.303704 14770 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:09.303776 14770 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:09.307228 14770 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:02:09.309118 14785 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:09.310110 14770 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:09.310212 14770 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "889ab14631a346f6b5b07e0fc7269496"
format_stamp: "Formatted at 2026-08-12 08:02:08 on dist-test-slave-49h3"
I20260812 08:02:09.310597 14770 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:09.320633 14770 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:09.321285 14770 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:09.321501 14770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:09.328732 14846 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:42271 every 8 connection(s)
I20260812 08:02:09.328747 14770 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:42271
I20260812 08:02:09.329943 14770 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:02:09.334331 14848 sys_catalog.cc:263] Verifying existing consensus state
I20260812 08:02:09.335961 14848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496: Bootstrap starting.
I20260812 08:02:09.339341   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 14770
I20260812 08:02:09.339870   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:44551
--local_ip_for_outbound_sockets=127.0.186.65
--tserver_master_addrs=127.0.186.126:42271
--webserver_port=40735
--webserver_interface=127.0.186.65
--builtin_ntp_servers=127.0.186.84:40929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_incompatible_replica_management_is_fatal=false
--heartbeat_interval_ms=500
--raft_prepare_replacement_before_eviction=true
--raft_prepare_replacement_before_eviction=false with env {}
I20260812 08:02:09.351011 14848 log.cc:826] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:09.358995 14848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=5 ignored=0} mutations{seen=2 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:02:09.359370 14848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496: Bootstrap complete.
I20260812 08:02:09.362808 14848 raft_consensus.cc:359] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "889ab14631a346f6b5b07e0fc7269496" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 42271 } }
I20260812 08:02:09.363158 14848 raft_consensus.cc:740] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 889ab14631a346f6b5b07e0fc7269496, State: Initialized, Role: FOLLOWER
I20260812 08:02:09.363679 14848 consensus_queue.cc:260] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "889ab14631a346f6b5b07e0fc7269496" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 42271 } }
I20260812 08:02:09.363807 14848 raft_consensus.cc:399] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:09.363873 14848 raft_consensus.cc:493] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:09.363979 14848 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:02:09.365585 14848 raft_consensus.cc:515] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "889ab14631a346f6b5b07e0fc7269496" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 42271 } }
I20260812 08:02:09.365927 14848 leader_election.cc:304] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 889ab14631a346f6b5b07e0fc7269496; no voters: 
I20260812 08:02:09.366253 14848 leader_election.cc:290] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [CANDIDATE]: Term 2 election: Requested vote from peers 
I20260812 08:02:09.366367 14853 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:02:09.366559 14853 raft_consensus.cc:697] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [term 2 LEADER]: Becoming Leader. State: Replica: 889ab14631a346f6b5b07e0fc7269496, State: Running, Role: LEADER
I20260812 08:02:09.366922 14853 consensus_queue.cc:237] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "889ab14631a346f6b5b07e0fc7269496" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 42271 } }
I20260812 08:02:09.367463 14848 sys_catalog.cc:565] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:02:09.367883 14853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 889ab14631a346f6b5b07e0fc7269496. Latest consensus state: current_term: 2 leader_uuid: "889ab14631a346f6b5b07e0fc7269496" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "889ab14631a346f6b5b07e0fc7269496" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 42271 } } }
I20260812 08:02:09.368053 14853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:09.368254 14854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "889ab14631a346f6b5b07e0fc7269496" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "889ab14631a346f6b5b07e0fc7269496" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 42271 } } }
I20260812 08:02:09.368366 14854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496 [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:09.368494 14857 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:02:09.372571 14857 catalog_manager.cc:679] Loaded metadata for table TestTable [id=25e36a2dfacf409b82aea43e952dcb03]
I20260812 08:02:09.374193 14857 tablet_loader.cc:96] loaded metadata for tablet 5f11401241104137928a1f9de1b19521 (table TestTable [id=25e36a2dfacf409b82aea43e952dcb03])
I20260812 08:02:09.374517 14857 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:02:09.375253 14857 catalog_manager.cc:1295] Loaded cluster ID: 9736a7eb7359475cb2f735ff876e476e
I20260812 08:02:09.375308 14857 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:02:09.377038 14857 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:02:09.377790 14857 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 889ab14631a346f6b5b07e0fc7269496: Loaded TSK: 0
I20260812 08:02:09.378592 14857 catalog_manager.cc:1550] Initializing in-progress tserver states...
W20260812 08:02:09.470793 14850 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:09.471102 14850 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:09.471177 14850 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:09.474843 14850 flags.cc:432] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20260812 08:02:09.474939 14850 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:09.475044 14850 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:02:09.479389 14850 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:40929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:44551
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=40735
--heartbeat_incompatible_replica_management_is_fatal=false
--heartbeat_interval_ms=500
--tserver_master_addrs=127.0.186.126:42271
--never_fsync=true
--heap_profile_path=/tmp/kudu.14850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:09.480754 14850 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:09.482115 14850 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:09.489308 14884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:09.489526 14850 server_base.cc:1061] running on GCE node
W20260812 08:02:09.489341 14881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:09.489708 14882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:09.490151 14850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:09.490778 14850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:09.491943 14850 hybrid_clock.cc:648] HybridClock initialized: now 1786521729491918 us; error 49 us; skew 500 ppm
I20260812 08:02:09.494047 14850 webserver.cc:533] Webserver started at http://127.0.186.65:40735/ using document root <none> and password file <none>
I20260812 08:02:09.494709 14850 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:09.494801 14850 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:09.498730 14850 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:02:09.500847 14893 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:09.502025 14850 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:09.502152 14850 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "622fc79708ae4a65b254cce1c881f0c1"
format_stamp: "Formatted at 2026-08-12 08:02:09 on dist-test-slave-49h3"
I20260812 08:02:09.502579 14850 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:09.523373 14850 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:09.524041 14850 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:09.524230 14850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:09.524822 14850 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:09.526156 14901 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:02:09.529534 14850 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:02:09.529592 14850 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.001s	sys 0.000s
I20260812 08:02:09.529644 14850 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:02:09.531114 14850 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:02:09.531178 14850 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:02:09.531513 14901 tablet_bootstrap.cc:492] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1: Bootstrap starting.
I20260812 08:02:09.542727 14850 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:44551
I20260812 08:02:09.542892 15031 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:44551 every 8 connection(s)
I20260812 08:02:09.544111 14850 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:02:09.547578   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 14850
I20260812 08:02:09.548426 14901 log.cc:826] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:09.551553 14901 tablet_bootstrap.cc:492] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:02:09.551925 14901 tablet_bootstrap.cc:492] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1: Bootstrap complete.
I20260812 08:02:09.552920 14901 ts_tablet_manager.cc:1403] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.008s
I20260812 08:02:09.555384 14901 raft_consensus.cc:359] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622fc79708ae4a65b254cce1c881f0c1" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44551 } }
I20260812 08:02:09.555694 14901 raft_consensus.cc:740] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 622fc79708ae4a65b254cce1c881f0c1, State: Initialized, Role: FOLLOWER
I20260812 08:02:09.556077 14901 consensus_queue.cc:260] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622fc79708ae4a65b254cce1c881f0c1" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44551 } }
I20260812 08:02:09.556186 14901 raft_consensus.cc:399] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:09.556231 14901 raft_consensus.cc:493] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:09.556308 14901 raft_consensus.cc:3060] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:02:09.557861 14901 raft_consensus.cc:515] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622fc79708ae4a65b254cce1c881f0c1" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44551 } }
I20260812 08:02:09.558167 14901 leader_election.cc:304] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 622fc79708ae4a65b254cce1c881f0c1; no voters: 
I20260812 08:02:09.558419 14901 leader_election.cc:290] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [CANDIDATE]: Term 2 election: Requested vote from peers 
I20260812 08:02:09.558571 15037 raft_consensus.cc:2804] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:02:09.558776 15037 raft_consensus.cc:697] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [term 2 LEADER]: Becoming Leader. State: Replica: 622fc79708ae4a65b254cce1c881f0c1, State: Running, Role: LEADER
I20260812 08:02:09.559062 15037 consensus_queue.cc:237] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622fc79708ae4a65b254cce1c881f0c1" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44551 } }
I20260812 08:02:09.559226 14901 ts_tablet_manager.cc:1434] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1: Time spent starting tablet: real 0.006s	user 0.009s	sys 0.000s
I20260812 08:02:09.570271 15032 heartbeater.cc:344] Connected to a master server at 127.0.186.126:42271
I20260812 08:02:09.570531 15032 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:09.571005 15032 heartbeater.cc:507] Master 127.0.186.126:42271 requested a full tablet report, sending...
W20260812 08:02:09.573673 14806 master_service.cc:413] Configuration error: replica replacement scheme (EVICT_FIRST) of the tablet server 622fc79708ae4a65b254cce1c881f0c1 at 127.0.186.65:56133 differs from the catalog manager's (PREPARE_REPLACEMENT_BEFORE_EVICTION); they must be run with the same scheme (controlled by the --raft_prepare_replacement_before_eviction flag)
W20260812 08:02:09.574721 15032 heartbeater.cc:646] Failed to heartbeat to 127.0.186.126:42271 (0 consecutive failures): Configuration error: replica replacement scheme (EVICT_FIRST) of the tablet server 622fc79708ae4a65b254cce1c881f0c1 at 127.0.186.65:56133 differs from the catalog manager's (PREPARE_REPLACEMENT_BEFORE_EVICTION); they must be run with the same scheme (controlled by the --raft_prepare_replacement_before_eviction flag)
E20260812 08:02:09.574803 15032 heartbeater.cc:687] master detected incompatibility: Configuration error: replica replacement scheme (EVICT_FIRST) of the tablet server 622fc79708ae4a65b254cce1c881f0c1 at 127.0.186.65:56133 differs from the catalog manager's (PREPARE_REPLACEMENT_BEFORE_EVICTION); they must be run with the same scheme (controlled by the --raft_prepare_replacement_before_eviction flag)
I20260812 08:02:09.574878 15032 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:09.575137 15032 heartbeater.cc:507] Master 127.0.186.126:42271 requested a full tablet report, sending...
W20260812 08:02:09.576031 14806 master_service.cc:413] Configuration error: replica replacement scheme (EVICT_FIRST) of the tablet server 622fc79708ae4a65b254cce1c881f0c1 at 127.0.186.65:56133 differs from the catalog manager's (PREPARE_REPLACEMENT_BEFORE_EVICTION); they must be run with the same scheme (controlled by the --raft_prepare_replacement_before_eviction flag)
I20260812 08:02:09.576479 15032 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:09.576682 15032 heartbeater.cc:507] Master 127.0.186.126:42271 requested a full tablet report, sending...
W20260812 08:02:09.577579 14806 master_service.cc:413] Configuration error: replica replacement scheme (EVICT_FIRST) of the tablet server 622fc79708ae4a65b254cce1c881f0c1 at 127.0.186.65:56133 differs from the catalog manager's (PREPARE_REPLACEMENT_BEFORE_EVICTION); they must be run with the same scheme (controlled by the --raft_prepare_replacement_before_eviction flag)
W20260812 08:02:09.577947 15032 heartbeater.cc:412] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 08:02:10.078190 15032 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:10.078552 15032 heartbeater.cc:507] Master 127.0.186.126:42271 requested a full tablet report, sending...
W20260812 08:02:10.079955 14806 master_service.cc:413] Configuration error: replica replacement scheme (EVICT_FIRST) of the tablet server 622fc79708ae4a65b254cce1c881f0c1 at 127.0.186.65:56133 differs from the catalog manager's (PREPARE_REPLACEMENT_BEFORE_EVICTION); they must be run with the same scheme (controlled by the --raft_prepare_replacement_before_eviction flag)
I20260812 08:02:10.580804 15032 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:10.581189 15032 heartbeater.cc:507] Master 127.0.186.126:42271 requested a full tablet report, sending...
W20260812 08:02:10.582453 14806 master_service.cc:413] Configuration error: replica replacement scheme (EVICT_FIRST) of the tablet server 622fc79708ae4a65b254cce1c881f0c1 at 127.0.186.65:56133 differs from the catalog manager's (PREPARE_REPLACEMENT_BEFORE_EVICTION); they must be run with the same scheme (controlled by the --raft_prepare_replacement_before_eviction flag)
I20260812 08:02:11.052762   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 14850
I20260812 08:02:11.063261   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:44551
--local_ip_for_outbound_sockets=127.0.186.65
--tserver_master_addrs=127.0.186.126:42271
--webserver_port=40735
--webserver_interface=127.0.186.65
--builtin_ntp_servers=127.0.186.84:40929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_incompatible_replica_management_is_fatal=false
--heartbeat_interval_ms=500
--raft_prepare_replacement_before_eviction=true
--heartbeat_inject_required_feature_flag=999 with env {}
W20260812 08:02:11.170667 15050 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:11.170886 15050 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:11.170933 15050 flags.cc:432] Enabled unsafe flag: --heartbeat_inject_required_feature_flag=999
W20260812 08:02:11.170970 15050 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:11.174319 15050 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:11.174459 15050 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:02:11.178401 15050 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:40929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:44551
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=40735
--heartbeat_incompatible_replica_management_is_fatal=false
--heartbeat_inject_required_feature_flag=999
--heartbeat_interval_ms=500
--tserver_master_addrs=127.0.186.126:42271
--never_fsync=true
--heap_profile_path=/tmp/kudu.15050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:11.179742 15050 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:11.180966 15050 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:11.187917 15058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:11.187919 15056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:11.188011 15050 server_base.cc:1061] running on GCE node
W20260812 08:02:11.187950 15060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:11.188557 15050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:11.189162 15050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:11.190337 15050 hybrid_clock.cc:648] HybridClock initialized: now 1786521731190323 us; error 29 us; skew 500 ppm
I20260812 08:02:11.192381 15050 webserver.cc:533] Webserver started at http://127.0.186.65:40735/ using document root <none> and password file <none>
I20260812 08:02:11.193002 15050 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:11.193089 15050 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:11.197038 15050 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.002s	sys 0.000s
I20260812 08:02:11.199173 15066 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:11.200269 15050 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:02:11.200397 15050 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "622fc79708ae4a65b254cce1c881f0c1"
format_stamp: "Formatted at 2026-08-12 08:02:09 on dist-test-slave-49h3"
I20260812 08:02:11.200845 15050 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:11.221594 15050 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:11.222311 15050 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:11.222491 15050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:11.223095 15050 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:11.224320 15073 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:02:11.227690 15050 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:02:11.227747 15050 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:02:11.227821 15050 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:02:11.229261 15050 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:02:11.229305 15050 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
I20260812 08:02:11.229662 15073 tablet_bootstrap.cc:492] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1: Bootstrap starting.
I20260812 08:02:11.240243 15050 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:44551
I20260812 08:02:11.240293 15199 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:44551 every 8 connection(s)
I20260812 08:02:11.241325 15050 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:02:11.243567 15073 log.cc:826] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:11.246459 15073 tablet_bootstrap.cc:492] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:02:11.246831 15073 tablet_bootstrap.cc:492] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1: Bootstrap complete.
I20260812 08:02:11.247921 15073 ts_tablet_manager.cc:1403] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.004s
I20260812 08:02:11.249862 15201 heartbeater.cc:344] Connected to a master server at 127.0.186.126:42271
I20260812 08:02:11.249998   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 15050
I20260812 08:02:11.250142 15201 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:11.250728 15201 heartbeater.cc:507] Master 127.0.186.126:42271 requested a full tablet report, sending...
I20260812 08:02:11.250962 15073 raft_consensus.cc:359] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622fc79708ae4a65b254cce1c881f0c1" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44551 } }
I20260812 08:02:11.251312 15073 raft_consensus.cc:740] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 622fc79708ae4a65b254cce1c881f0c1, State: Initialized, Role: FOLLOWER
I20260812 08:02:11.251744 15073 consensus_queue.cc:260] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622fc79708ae4a65b254cce1c881f0c1" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44551 } }
I20260812 08:02:11.251873 15073 raft_consensus.cc:399] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:11.252031 15073 raft_consensus.cc:493] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:11.252152 15073 raft_consensus.cc:3060] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [term 2 FOLLOWER]: Advancing to term 3
W20260812 08:02:11.253856 14788 messenger.cc:376] Unable to handle RPC call: Not implemented: call requires unsupported application feature flags: 999
I20260812 08:02:11.253857 15073 raft_consensus.cc:515] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622fc79708ae4a65b254cce1c881f0c1" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44551 } }
I20260812 08:02:11.254166 15073 leader_election.cc:304] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 622fc79708ae4a65b254cce1c881f0c1; no voters: 
I20260812 08:02:11.254462 15073 leader_election.cc:290] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [CANDIDATE]: Term 3 election: Requested vote from peers 
W20260812 08:02:11.254464 15201 heartbeater.cc:646] Failed to heartbeat to 127.0.186.126:42271 (0 consecutive failures): Remote error: Failed to send heartbeat to master: unsupported feature flags
I20260812 08:02:11.254575 15206 raft_consensus.cc:2804] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 08:02:11.254763 15206 raft_consensus.cc:697] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [term 3 LEADER]: Becoming Leader. State: Replica: 622fc79708ae4a65b254cce1c881f0c1, State: Running, Role: LEADER
I20260812 08:02:11.255065 15206 consensus_queue.cc:237] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622fc79708ae4a65b254cce1c881f0c1" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44551 } }
I20260812 08:02:11.255359 15073 ts_tablet_manager.cc:1434] T 5f11401241104137928a1f9de1b19521 P 622fc79708ae4a65b254cce1c881f0c1: Time spent starting tablet: real 0.007s	user 0.009s	sys 0.000s
E20260812 08:02:11.255941 15201 heartbeater.cc:687] master does not support required feature flag(s) : Remote error: Failed to send heartbeat to master: unsupported feature flags
I20260812 08:02:11.256039 15201 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:11.256247 15201 heartbeater.cc:507] Master 127.0.186.126:42271 requested a full tablet report, sending...
W20260812 08:02:11.257236 14788 messenger.cc:376] Unable to handle RPC call: Not implemented: call requires unsupported application feature flags: 999
I20260812 08:02:11.257548 15201 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:11.257735 15201 heartbeater.cc:507] Master 127.0.186.126:42271 requested a full tablet report, sending...
W20260812 08:02:11.258282 14788 messenger.cc:376] Unable to handle RPC call: Not implemented: call requires unsupported application feature flags: 999
W20260812 08:02:11.258533 15201 heartbeater.cc:412] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 08:02:11.758730 15201 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:11.759064 15201 heartbeater.cc:507] Master 127.0.186.126:42271 requested a full tablet report, sending...
W20260812 08:02:11.759992 14788 messenger.cc:376] Unable to handle RPC call: Not implemented: call requires unsupported application feature flags: 999
I20260812 08:02:12.260555 15201 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:12.260910 15201 heartbeater.cc:507] Master 127.0.186.126:42271 requested a full tablet report, sending...
W20260812 08:02:12.261886 14788 messenger.cc:376] Unable to handle RPC call: Not implemented: call requires unsupported application feature flags: 999
I20260812 08:02:12.751984   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 15050
I20260812 08:02:12.762661   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 14770
2026-08-12T08:02:12Z chronyd exiting
[       OK ] IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig/1 (4136 ms)
[ RUN      ] IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig/2
I20260812 08:02:12.785868   745 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:02:12.785917   745 ts_itest-base.cc:116] --------------
I20260812 08:02:12.785928   745 ts_itest-base.cc:117] 1 tablet servers
I20260812 08:02:12.785938   745 ts_itest-base.cc:118] 1 replicas per TS
I20260812 08:02:12.785946   745 ts_itest-base.cc:119] --------------
2026-08-12T08:02:12Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:02:12Z Disabled control of system clock
I20260812 08:02:12.794878   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:43271
--webserver_interface=127.0.186.126
--webserver_port=0
--builtin_ntp_servers=127.0.186.84:39289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:43271
--raft_prepare_replacement_before_eviction=false with env {}
W20260812 08:02:12.900494 15220 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:12.900792 15220 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:12.900856 15220 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:12.904460 15220 flags.cc:432] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20260812 08:02:12.904541 15220 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:02:12.904569 15220 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:12.904595 15220 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:02:12.904634 15220 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:02:12.908891 15220 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:39289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:43271
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:43271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.15220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:12.910245 15220 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:12.911412 15220 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:12.916790 15229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:12.916800 15226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:12.916875 15225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:12.917032 15220 server_base.cc:1061] running on GCE node
I20260812 08:02:12.917570 15220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:12.918416 15220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:12.919600 15220 hybrid_clock.cc:648] HybridClock initialized: now 1786521732919580 us; error 33 us; skew 500 ppm
I20260812 08:02:12.921423 15220 webserver.cc:533] Webserver started at http://127.0.186.126:35589/ using document root <none> and password file <none>
I20260812 08:02:12.922040 15220 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:12.922104 15220 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:12.922299 15220 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:12.923885 15220 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "eec39b81d6a14002abefe7647b10ac3d"
format_stamp: "Formatted at 2026-08-12 08:02:12 on dist-test-slave-49h3"
I20260812 08:02:12.924394 15220 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "eec39b81d6a14002abefe7647b10ac3d"
format_stamp: "Formatted at 2026-08-12 08:02:12 on dist-test-slave-49h3"
I20260812 08:02:12.927867 15220 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.002s
I20260812 08:02:12.930136 15235 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:12.931107 15220 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 08:02:12.931222 15220 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "eec39b81d6a14002abefe7647b10ac3d"
format_stamp: "Formatted at 2026-08-12 08:02:12 on dist-test-slave-49h3"
I20260812 08:02:12.931318 15220 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:12.960388 15220 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:12.961102 15220 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:12.961289 15220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:12.968587 15300 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:43271 every 8 connection(s)
I20260812 08:02:12.968588 15220 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:43271
I20260812 08:02:12.969686 15220 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:02:12.970297   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 15220
I20260812 08:02:12.970422   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:02:12.972906 15301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:12.978106 15301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d: Bootstrap starting.
I20260812 08:02:12.980552 15301 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:12.981259 15301 log.cc:826] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:12.983206 15301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d: No bootstrap required, opened a new log
I20260812 08:02:12.985944 15301 raft_consensus.cc:359] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eec39b81d6a14002abefe7647b10ac3d" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 43271 } }
I20260812 08:02:12.986229 15301 raft_consensus.cc:385] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:12.986295 15301 raft_consensus.cc:740] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eec39b81d6a14002abefe7647b10ac3d, State: Initialized, Role: FOLLOWER
I20260812 08:02:12.986790 15301 consensus_queue.cc:260] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eec39b81d6a14002abefe7647b10ac3d" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 43271 } }
I20260812 08:02:12.986948 15301 raft_consensus.cc:399] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:12.987025 15301 raft_consensus.cc:493] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:12.987167 15301 raft_consensus.cc:3060] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:12.988103 15301 raft_consensus.cc:515] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eec39b81d6a14002abefe7647b10ac3d" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 43271 } }
I20260812 08:02:12.988548 15301 leader_election.cc:304] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eec39b81d6a14002abefe7647b10ac3d; no voters: 
I20260812 08:02:12.988910 15301 leader_election.cc:290] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:02:12.989080 15307 raft_consensus.cc:2804] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:12.989317 15307 raft_consensus.cc:697] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [term 1 LEADER]: Becoming Leader. State: Replica: eec39b81d6a14002abefe7647b10ac3d, State: Running, Role: LEADER
I20260812 08:02:12.989804 15307 consensus_queue.cc:237] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eec39b81d6a14002abefe7647b10ac3d" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 43271 } }
I20260812 08:02:12.990556 15301 sys_catalog.cc:565] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:02:12.991789 15309 sys_catalog.cc:455] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader eec39b81d6a14002abefe7647b10ac3d. Latest consensus state: current_term: 1 leader_uuid: "eec39b81d6a14002abefe7647b10ac3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eec39b81d6a14002abefe7647b10ac3d" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 43271 } } }
I20260812 08:02:12.991952 15309 sys_catalog.cc:458] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:12.992136 15308 sys_catalog.cc:455] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eec39b81d6a14002abefe7647b10ac3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eec39b81d6a14002abefe7647b10ac3d" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 43271 } } }
I20260812 08:02:12.992283 15308 sys_catalog.cc:458] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:12.992518 15321 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:02:12.995723 15321 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:02:13.002090 15321 catalog_manager.cc:1383] Generated new cluster ID: 534f5d2854ca4bf18feff7e238ca324d
I20260812 08:02:13.002187 15321 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:02:13.011193 15321 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:02:13.012295 15321 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:02:13.022274 15321 catalog_manager.cc:6092] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d: Generated new TSK 0
I20260812 08:02:13.022907 15321 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:02:13.037329   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:0
--local_ip_for_outbound_sockets=127.0.186.65
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:43271
--builtin_ntp_servers=127.0.186.84:39289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_incompatible_replica_management_is_fatal=true
--heartbeat_interval_ms=500
--raft_prepare_replacement_before_eviction=false with env {}
W20260812 08:02:13.147836 15332 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:13.148044 15332 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:13.148108 15332 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:13.151371 15332 flags.cc:432] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20260812 08:02:13.151435 15332 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:13.151520 15332 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:02:13.155293 15332 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:39289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=0
--heartbeat_interval_ms=500
--tserver_master_addrs=127.0.186.126:43271
--never_fsync=true
--heap_profile_path=/tmp/kudu.15332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:13.156340 15332 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:13.157436 15332 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:13.163582 15341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:13.163565 15340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:13.163674 15332 server_base.cc:1061] running on GCE node
W20260812 08:02:13.163697 15343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:13.164194 15332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:13.164666 15332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:13.165807 15332 hybrid_clock.cc:648] HybridClock initialized: now 1786521733165790 us; error 27 us; skew 500 ppm
I20260812 08:02:13.167382 15332 webserver.cc:533] Webserver started at http://127.0.186.65:40095/ using document root <none> and password file <none>
I20260812 08:02:13.167901 15332 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:13.167951 15332 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:13.168130 15332 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:13.169603 15332 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "323e932d2055472b819c0ba45ec9418e"
format_stamp: "Formatted at 2026-08-12 08:02:13 on dist-test-slave-49h3"
I20260812 08:02:13.170048 15332 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "323e932d2055472b819c0ba45ec9418e"
format_stamp: "Formatted at 2026-08-12 08:02:13 on dist-test-slave-49h3"
I20260812 08:02:13.173075 15332 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.000s
I20260812 08:02:13.175330 15350 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:13.176311 15332 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:13.176419 15332 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "323e932d2055472b819c0ba45ec9418e"
format_stamp: "Formatted at 2026-08-12 08:02:13 on dist-test-slave-49h3"
I20260812 08:02:13.176506 15332 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:13.189963 15332 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:13.190725 15332 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:13.190914 15332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:13.191606 15332 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:13.192885 15332 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:13.192993 15332 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:13.193056 15332 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:13.193097 15332 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:13.201984 15332 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:44125
I20260812 08:02:13.202013 15493 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:44125 every 8 connection(s)
I20260812 08:02:13.202944 15332 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:02:13.204043   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 15332
I20260812 08:02:13.204170   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:02:13.211045 15494 heartbeater.cc:344] Connected to a master server at 127.0.186.126:43271
I20260812 08:02:13.211247 15494 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:13.211635 15494 heartbeater.cc:507] Master 127.0.186.126:43271 requested a full tablet report, sending...
I20260812 08:02:13.212966 15254 ts_manager.cc:194] Registered new tserver with Master: 323e932d2055472b819c0ba45ec9418e (127.0.186.65:44125)
I20260812 08:02:13.214155 15254 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.65:43163
I20260812 08:02:13.218343   745 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260812 08:02:13.226462 15254 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:32768:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
I20260812 08:02:13.242736 15407 tablet_service.cc:1511] Processing CreateTablet for tablet 22d277c73b10411187ca0e7f283a81bd (DEFAULT_TABLE table=TestTable [id=e218fe6b632b48f3b4f15671d688ad72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:13.243794 15407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22d277c73b10411187ca0e7f283a81bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:13.248986 15506 tablet_bootstrap.cc:492] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e: Bootstrap starting.
I20260812 08:02:13.251132 15506 tablet_bootstrap.cc:654] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:13.251962 15506 log.cc:826] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:13.254042 15506 tablet_bootstrap.cc:492] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e: No bootstrap required, opened a new log
I20260812 08:02:13.254276 15506 ts_tablet_manager.cc:1403] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:02:13.256841 15506 raft_consensus.cc:359] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "323e932d2055472b819c0ba45ec9418e" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44125 } }
I20260812 08:02:13.257055 15506 raft_consensus.cc:385] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:13.257109 15506 raft_consensus.cc:740] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 323e932d2055472b819c0ba45ec9418e, State: Initialized, Role: FOLLOWER
I20260812 08:02:13.257607 15506 consensus_queue.cc:260] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "323e932d2055472b819c0ba45ec9418e" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44125 } }
I20260812 08:02:13.257745 15506 raft_consensus.cc:399] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:13.257818 15506 raft_consensus.cc:493] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:13.257926 15506 raft_consensus.cc:3060] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:13.259044 15506 raft_consensus.cc:515] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "323e932d2055472b819c0ba45ec9418e" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44125 } }
I20260812 08:02:13.259416 15506 leader_election.cc:304] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 323e932d2055472b819c0ba45ec9418e; no voters: 
I20260812 08:02:13.259788 15506 leader_election.cc:290] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:02:13.259876 15510 raft_consensus.cc:2804] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:13.260061 15510 raft_consensus.cc:697] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [term 1 LEADER]: Becoming Leader. State: Replica: 323e932d2055472b819c0ba45ec9418e, State: Running, Role: LEADER
I20260812 08:02:13.260406 15510 consensus_queue.cc:237] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "323e932d2055472b819c0ba45ec9418e" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44125 } }
I20260812 08:02:13.260509 15494 heartbeater.cc:499] Master 127.0.186.126:43271 was elected leader, sending a full tablet report...
I20260812 08:02:13.260857 15506 ts_tablet_manager.cc:1434] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260812 08:02:13.264499 15254 catalog_manager.cc:5719] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e reported cstate change: term changed from 0 to 1, leader changed from <none> to 323e932d2055472b819c0ba45ec9418e (127.0.186.65). New cstate: current_term: 1 leader_uuid: "323e932d2055472b819c0ba45ec9418e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "323e932d2055472b819c0ba45ec9418e" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44125 } } }
I20260812 08:02:13.278081   745 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260812 08:02:13.279505   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 323e932d2055472b819c0ba45ec9418e to finish bootstrapping
I20260812 08:02:13.285332   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 15332
I20260812 08:02:13.297103   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 15220
I20260812 08:02:13.306094   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:43271
--webserver_interface=127.0.186.126
--webserver_port=35589
--builtin_ntp_servers=127.0.186.84:39289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:43271
--raft_prepare_replacement_before_eviction=false with env {}
W20260812 08:02:13.412619 15516 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:13.412843 15516 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:13.412884 15516 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:13.416179 15516 flags.cc:432] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20260812 08:02:13.416239 15516 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:02:13.416257 15516 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:13.416275 15516 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:02:13.416293 15516 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:02:13.420077 15516 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:39289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:43271
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:43271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=35589
--never_fsync=true
--heap_profile_path=/tmp/kudu.15516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:13.421192 15516 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:13.422322 15516 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:13.427582 15524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:13.427584 15525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:13.427673 15527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:13.427803 15516 server_base.cc:1061] running on GCE node
I20260812 08:02:13.428313 15516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:13.429138 15516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:13.430287 15516 hybrid_clock.cc:648] HybridClock initialized: now 1786521733430283 us; error 30 us; skew 500 ppm
I20260812 08:02:13.431849 15516 webserver.cc:533] Webserver started at http://127.0.186.126:35589/ using document root <none> and password file <none>
I20260812 08:02:13.432394 15516 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:13.432469 15516 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:13.435611 15516 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:02:13.437330 15533 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:13.438238 15516 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:13.438351 15516 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "eec39b81d6a14002abefe7647b10ac3d"
format_stamp: "Formatted at 2026-08-12 08:02:12 on dist-test-slave-49h3"
I20260812 08:02:13.438697 15516 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:13.446610 15516 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:13.447172 15516 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:13.447324 15516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:13.454223 15594 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:43271 every 8 connection(s)
I20260812 08:02:13.454233 15516 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:43271
I20260812 08:02:13.455251 15516 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:02:13.459161 15596 sys_catalog.cc:263] Verifying existing consensus state
I20260812 08:02:13.460721 15596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d: Bootstrap starting.
I20260812 08:02:13.461952   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 15516
I20260812 08:02:13.462385   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:44125
--local_ip_for_outbound_sockets=127.0.186.65
--tserver_master_addrs=127.0.186.126:43271
--webserver_port=40095
--webserver_interface=127.0.186.65
--builtin_ntp_servers=127.0.186.84:39289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_incompatible_replica_management_is_fatal=true
--heartbeat_interval_ms=500
--raft_prepare_replacement_before_eviction=false
--raft_prepare_replacement_before_eviction=true with env {}
I20260812 08:02:13.475620 15596 log.cc:826] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:13.484409 15596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=5 ignored=0} mutations{seen=2 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:02:13.484771 15596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d: Bootstrap complete.
I20260812 08:02:13.488325 15596 raft_consensus.cc:359] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eec39b81d6a14002abefe7647b10ac3d" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 43271 } }
I20260812 08:02:13.488641 15596 raft_consensus.cc:740] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eec39b81d6a14002abefe7647b10ac3d, State: Initialized, Role: FOLLOWER
I20260812 08:02:13.489123 15596 consensus_queue.cc:260] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eec39b81d6a14002abefe7647b10ac3d" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 43271 } }
I20260812 08:02:13.489245 15596 raft_consensus.cc:399] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:13.489318 15596 raft_consensus.cc:493] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:13.489423 15596 raft_consensus.cc:3060] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:02:13.490864 15596 raft_consensus.cc:515] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eec39b81d6a14002abefe7647b10ac3d" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 43271 } }
I20260812 08:02:13.491181 15596 leader_election.cc:304] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eec39b81d6a14002abefe7647b10ac3d; no voters: 
I20260812 08:02:13.491469 15596 leader_election.cc:290] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [CANDIDATE]: Term 2 election: Requested vote from peers 
I20260812 08:02:13.491585 15600 raft_consensus.cc:2804] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:02:13.491783 15600 raft_consensus.cc:697] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [term 2 LEADER]: Becoming Leader. State: Replica: eec39b81d6a14002abefe7647b10ac3d, State: Running, Role: LEADER
I20260812 08:02:13.492123 15600 consensus_queue.cc:237] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eec39b81d6a14002abefe7647b10ac3d" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 43271 } }
I20260812 08:02:13.492650 15596 sys_catalog.cc:565] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:02:13.493026 15600 sys_catalog.cc:455] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader eec39b81d6a14002abefe7647b10ac3d. Latest consensus state: current_term: 2 leader_uuid: "eec39b81d6a14002abefe7647b10ac3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eec39b81d6a14002abefe7647b10ac3d" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 43271 } } }
I20260812 08:02:13.493199 15600 sys_catalog.cc:458] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:13.493386 15601 sys_catalog.cc:455] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "eec39b81d6a14002abefe7647b10ac3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eec39b81d6a14002abefe7647b10ac3d" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 43271 } } }
I20260812 08:02:13.493571 15601 sys_catalog.cc:458] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:13.493654 15604 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:02:13.497354 15604 catalog_manager.cc:679] Loaded metadata for table TestTable [id=e218fe6b632b48f3b4f15671d688ad72]
I20260812 08:02:13.498900 15604 tablet_loader.cc:96] loaded metadata for tablet 22d277c73b10411187ca0e7f283a81bd (table TestTable [id=e218fe6b632b48f3b4f15671d688ad72])
I20260812 08:02:13.499092 15604 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:02:13.499742 15604 catalog_manager.cc:1295] Loaded cluster ID: 534f5d2854ca4bf18feff7e238ca324d
I20260812 08:02:13.499817 15604 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:02:13.502197 15604 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:02:13.502935 15604 catalog_manager.cc:6103] T 00000000000000000000000000000000 P eec39b81d6a14002abefe7647b10ac3d: Loaded TSK: 0
I20260812 08:02:13.503872 15604 catalog_manager.cc:1550] Initializing in-progress tserver states...
W20260812 08:02:13.571882 15598 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:13.572079 15598 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:13.572152 15598 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:13.575372 15598 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:13.575493 15598 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:02:13.579219 15598 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:39289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:44125
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=40095
--heartbeat_interval_ms=500
--tserver_master_addrs=127.0.186.126:43271
--never_fsync=true
--heap_profile_path=/tmp/kudu.15598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:13.580204 15598 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:13.581288 15598 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:13.587894 15627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:13.587838 15629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:13.587832 15626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:13.588284 15598 server_base.cc:1061] running on GCE node
I20260812 08:02:13.588678 15598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:13.589262 15598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:13.590426 15598 hybrid_clock.cc:648] HybridClock initialized: now 1786521733590401 us; error 41 us; skew 500 ppm
I20260812 08:02:13.592157 15598 webserver.cc:533] Webserver started at http://127.0.186.65:40095/ using document root <none> and password file <none>
I20260812 08:02:13.592720 15598 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:13.592787 15598 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:13.596210 15598 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:13.598100 15638 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:13.599072 15598 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:13.599187 15598 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "323e932d2055472b819c0ba45ec9418e"
format_stamp: "Formatted at 2026-08-12 08:02:13 on dist-test-slave-49h3"
I20260812 08:02:13.599550 15598 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:13.608590 15598 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:13.609140 15598 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:13.609293 15598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:13.609864 15598 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:13.611019 15645 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:02:13.614177 15598 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:02:13.614233 15598 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.003s	user 0.000s	sys 0.000s
I20260812 08:02:13.614277 15598 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:02:13.615628 15598 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:02:13.615682 15598 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
I20260812 08:02:13.616016 15645 tablet_bootstrap.cc:492] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e: Bootstrap starting.
I20260812 08:02:13.627130 15598 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:44125
I20260812 08:02:13.627316 15776 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:44125 every 8 connection(s)
I20260812 08:02:13.628697 15598 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:02:13.633040 15645 log.cc:826] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:13.636607 15645 tablet_bootstrap.cc:492] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:02:13.636997 15645 tablet_bootstrap.cc:492] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e: Bootstrap complete.
I20260812 08:02:13.638430 15645 ts_tablet_manager.cc:1403] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e: Time spent bootstrapping tablet: real 0.023s	user 0.005s	sys 0.016s
I20260812 08:02:13.639008   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 15598
I20260812 08:02:13.641366 15645 raft_consensus.cc:359] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "323e932d2055472b819c0ba45ec9418e" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44125 } }
I20260812 08:02:13.641775 15645 raft_consensus.cc:740] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 323e932d2055472b819c0ba45ec9418e, State: Initialized, Role: FOLLOWER
I20260812 08:02:13.642273 15645 consensus_queue.cc:260] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "323e932d2055472b819c0ba45ec9418e" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44125 } }
I20260812 08:02:13.642426 15645 raft_consensus.cc:399] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:13.642488 15645 raft_consensus.cc:493] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:13.642590 15645 raft_consensus.cc:3060] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:02:13.643304 15777 heartbeater.cc:344] Connected to a master server at 127.0.186.126:43271
I20260812 08:02:13.643554 15777 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:13.644166 15777 heartbeater.cc:507] Master 127.0.186.126:43271 requested a full tablet report, sending...
I20260812 08:02:13.644441 15645 raft_consensus.cc:515] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "323e932d2055472b819c0ba45ec9418e" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44125 } }
I20260812 08:02:13.644770 15645 leader_election.cc:304] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 323e932d2055472b819c0ba45ec9418e; no voters: 
I20260812 08:02:13.645056 15645 leader_election.cc:290] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [CANDIDATE]: Term 2 election: Requested vote from peers 
I20260812 08:02:13.645179 15783 raft_consensus.cc:2804] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:02:13.645370 15783 raft_consensus.cc:697] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [term 2 LEADER]: Becoming Leader. State: Replica: 323e932d2055472b819c0ba45ec9418e, State: Running, Role: LEADER
I20260812 08:02:13.645710 15783 consensus_queue.cc:237] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "323e932d2055472b819c0ba45ec9418e" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44125 } }
I20260812 08:02:13.646103 15645 ts_tablet_manager.cc:1434] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
W20260812 08:02:13.647109 15554 master_service.cc:413] Configuration error: replica replacement scheme (PREPARE_REPLACEMENT_BEFORE_EVICTION) of the tablet server 323e932d2055472b819c0ba45ec9418e at 127.0.186.65:48349 differs from the catalog manager's (EVICT_FIRST); they must be run with the same scheme (controlled by the --raft_prepare_replacement_before_eviction flag)
W20260812 08:02:13.648232 15777 heartbeater.cc:646] Failed to heartbeat to 127.0.186.126:43271 (0 consecutive failures): Configuration error: replica replacement scheme (PREPARE_REPLACEMENT_BEFORE_EVICTION) of the tablet server 323e932d2055472b819c0ba45ec9418e at 127.0.186.65:48349 differs from the catalog manager's (EVICT_FIRST); they must be run with the same scheme (controlled by the --raft_prepare_replacement_before_eviction flag)
F20260812 08:02:13.648322 15777 heartbeater.cc:685] master detected incompatibility: Configuration error: replica replacement scheme (PREPARE_REPLACEMENT_BEFORE_EVICTION) of the tablet server 323e932d2055472b819c0ba45ec9418e at 127.0.186.65:48349 differs from the catalog manager's (EVICT_FIRST); they must be run with the same scheme (controlled by the --raft_prepare_replacement_before_eviction flag)
*** Check failure stack trace: ***
    @     0x7f2251066d5d  google::LogMessage::Fail() at ??:0
    @     0x7f225106ab23  google::LogMessage::SendToLog() at ??:0
    @     0x7f225106675c  google::LogMessage::Flush() at ??:0
    @     0x7f2251067ee9  google::LogMessageFatal::~LogMessageFatal() at ??:0
    @     0x7f22555e9b55  kudu::tserver::Heartbeater::Thread::RunThread() at ??:0
    @     0x7f22555ea157  _ZZN4kudu7tserver11Heartbeater6Thread5StartEvENKUlvE_clEv at ??:0
    @     0x7f22555ebc3a  _ZNSt17_Function_handlerIFvvEZN4kudu7tserver11Heartbeater6Thread5StartEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
    @     0x5606e74b344e  std::function<>::operator()() at ??:0
    @     0x7f2252536712  kudu::Thread::SuperviseThread() at ??:0
    @     0x7f2252eb56db  start_thread at ??:0
    @     0x7f225021d61f  clone at ??:0
I20260812 08:02:15.431692   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:44125
--local_ip_for_outbound_sockets=127.0.186.65
--tserver_master_addrs=127.0.186.126:43271
--webserver_port=40095
--webserver_interface=127.0.186.65
--builtin_ntp_servers=127.0.186.84:39289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_incompatible_replica_management_is_fatal=true
--heartbeat_interval_ms=500
--raft_prepare_replacement_before_eviction=false
--heartbeat_inject_required_feature_flag=999 with env {}
W20260812 08:02:15.540519 15802 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:15.540716 15802 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:15.540756 15802 flags.cc:432] Enabled unsafe flag: --heartbeat_inject_required_feature_flag=999
W20260812 08:02:15.540778 15802 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:15.544075 15802 flags.cc:432] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20260812 08:02:15.544139 15802 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:15.544211 15802 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:02:15.547971 15802 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:39289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:44125
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=40095
--heartbeat_inject_required_feature_flag=999
--heartbeat_interval_ms=500
--tserver_master_addrs=127.0.186.126:43271
--never_fsync=true
--heap_profile_path=/tmp/kudu.15802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:15.548966 15802 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:15.550115 15802 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:15.556143 15808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:15.556146 15812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:15.556140 15807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:15.556329 15802 server_base.cc:1061] running on GCE node
I20260812 08:02:15.556766 15802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:15.557284 15802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:15.558451 15802 hybrid_clock.cc:648] HybridClock initialized: now 1786521735558435 us; error 31 us; skew 500 ppm
I20260812 08:02:15.560052 15802 webserver.cc:533] Webserver started at http://127.0.186.65:40095/ using document root <none> and password file <none>
I20260812 08:02:15.560561 15802 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:15.560611 15802 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:15.563758 15802 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:02:15.565593 15823 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:15.566509 15802 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:15.566609 15802 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "323e932d2055472b819c0ba45ec9418e"
format_stamp: "Formatted at 2026-08-12 08:02:13 on dist-test-slave-49h3"
I20260812 08:02:15.566953 15802 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:15.578972 15802 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:15.579499 15802 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:15.579648 15802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:15.580118 15802 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:15.581219 15830 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:02:15.584336 15802 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:02:15.584399 15802 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.003s	user 0.000s	sys 0.000s
I20260812 08:02:15.584461 15802 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:02:15.586000 15802 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:02:15.586062 15802 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.001s
I20260812 08:02:15.586344 15830 tablet_bootstrap.cc:492] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e: Bootstrap starting.
I20260812 08:02:15.596750 15802 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:44125
I20260812 08:02:15.596792 15959 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:44125 every 8 connection(s)
I20260812 08:02:15.597817 15802 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:02:15.598016   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 15802
I20260812 08:02:15.600525 15830 log.cc:826] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:15.603109 15830 tablet_bootstrap.cc:492] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:02:15.603412 15830 tablet_bootstrap.cc:492] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e: Bootstrap complete.
I20260812 08:02:15.604647 15830 ts_tablet_manager.cc:1403] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.006s
I20260812 08:02:15.606405 15960 heartbeater.cc:344] Connected to a master server at 127.0.186.126:43271
I20260812 08:02:15.606632 15960 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:15.607122 15960 heartbeater.cc:507] Master 127.0.186.126:43271 requested a full tablet report, sending...
I20260812 08:02:15.607239 15830 raft_consensus.cc:359] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "323e932d2055472b819c0ba45ec9418e" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44125 } }
I20260812 08:02:15.607549 15830 raft_consensus.cc:740] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 323e932d2055472b819c0ba45ec9418e, State: Initialized, Role: FOLLOWER
I20260812 08:02:15.607945 15830 consensus_queue.cc:260] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "323e932d2055472b819c0ba45ec9418e" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44125 } }
I20260812 08:02:15.608069 15830 raft_consensus.cc:399] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:15.608165 15830 raft_consensus.cc:493] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:15.608276 15830 raft_consensus.cc:3060] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [term 2 FOLLOWER]: Advancing to term 3
W20260812 08:02:15.609603 15536 messenger.cc:376] Unable to handle RPC call: Not implemented: call requires unsupported application feature flags: 999
I20260812 08:02:15.609920 15830 raft_consensus.cc:515] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "323e932d2055472b819c0ba45ec9418e" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44125 } }
W20260812 08:02:15.610164 15960 heartbeater.cc:646] Failed to heartbeat to 127.0.186.126:43271 (0 consecutive failures): Remote error: Failed to send heartbeat to master: unsupported feature flags
I20260812 08:02:15.610224 15830 leader_election.cc:304] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 323e932d2055472b819c0ba45ec9418e; no voters: 
I20260812 08:02:15.610508 15830 leader_election.cc:290] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [CANDIDATE]: Term 3 election: Requested vote from peers 
I20260812 08:02:15.610612 15964 raft_consensus.cc:2804] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [term 3 FOLLOWER]: Leader election won for term 3
I20260812 08:02:15.610785 15964 raft_consensus.cc:697] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [term 3 LEADER]: Becoming Leader. State: Replica: 323e932d2055472b819c0ba45ec9418e, State: Running, Role: LEADER
I20260812 08:02:15.611107 15964 consensus_queue.cc:237] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "323e932d2055472b819c0ba45ec9418e" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44125 } }
I20260812 08:02:15.611334 15830 ts_tablet_manager.cc:1434] T 22d277c73b10411187ca0e7f283a81bd P 323e932d2055472b819c0ba45ec9418e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
F20260812 08:02:15.611691 15960 heartbeater.cc:685] master does not support required feature flag(s) : Remote error: Failed to send heartbeat to master: unsupported feature flags
*** Check failure stack trace: ***
    @     0x7f31993add5d  google::LogMessage::Fail() at ??:0
    @     0x7f31993b1b23  google::LogMessage::SendToLog() at ??:0
    @     0x7f31993ad75c  google::LogMessage::Flush() at ??:0
    @     0x7f31993aeee9  google::LogMessageFatal::~LogMessageFatal() at ??:0
    @     0x7f319d930b55  kudu::tserver::Heartbeater::Thread::RunThread() at ??:0
    @     0x7f319d931157  _ZZN4kudu7tserver11Heartbeater6Thread5StartEvENKUlvE_clEv at ??:0
    @     0x7f319d932c3a  _ZNSt17_Function_handlerIFvvEZN4kudu7tserver11Heartbeater6Thread5StartEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
    @     0x55f2e73db44e  std::function<>::operator()() at ??:0
    @     0x7f319a87d712  kudu::Thread::SuperviseThread() at ??:0
    @     0x7f319b1fc6db  start_thread at ??:0
    @     0x7f319856461f  clone at ??:0
I20260812 08:02:17.310966   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 15516
2026-08-12T08:02:17Z chronyd exiting
[       OK ] IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig/2 (4551 ms)
[ RUN      ] IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig/3
I20260812 08:02:17.337016   745 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:02:17.337067   745 ts_itest-base.cc:116] --------------
I20260812 08:02:17.337078   745 ts_itest-base.cc:117] 1 tablet servers
I20260812 08:02:17.337090   745 ts_itest-base.cc:118] 1 replicas per TS
I20260812 08:02:17.337100   745 ts_itest-base.cc:119] --------------
2026-08-12T08:02:17Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:02:17Z Disabled control of system clock
I20260812 08:02:17.345880   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:46295
--webserver_interface=127.0.186.126
--webserver_port=0
--builtin_ntp_servers=127.0.186.84:45913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:46295
--raft_prepare_replacement_before_eviction=true with env {}
W20260812 08:02:17.447954 15992 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:17.448159 15992 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:17.448195 15992 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:17.451390 15992 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:02:17.451443 15992 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:17.451462 15992 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:02:17.451478 15992 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:02:17.455209 15992 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:45913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:46295
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:46295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.15992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:17.456224 15992 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:17.457296 15992 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:17.462646 15999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:17.462615 16002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:17.462620 15998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:17.462781 15992 server_base.cc:1061] running on GCE node
I20260812 08:02:17.463377 15992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:17.464254 15992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:17.465412 15992 hybrid_clock.cc:648] HybridClock initialized: now 1786521737465395 us; error 30 us; skew 500 ppm
I20260812 08:02:17.467134 15992 webserver.cc:533] Webserver started at http://127.0.186.126:40809/ using document root <none> and password file <none>
I20260812 08:02:17.467685 15992 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:17.467743 15992 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:17.467926 15992 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:17.469436 15992 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "0b83ffedad8c4cfb8de5ab60ba166120"
format_stamp: "Formatted at 2026-08-12 08:02:17 on dist-test-slave-49h3"
I20260812 08:02:17.469969 15992 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "0b83ffedad8c4cfb8de5ab60ba166120"
format_stamp: "Formatted at 2026-08-12 08:02:17 on dist-test-slave-49h3"
I20260812 08:02:17.473090 15992 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:02:17.475136 16008 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:17.476043 15992 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:02:17.476152 15992 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "0b83ffedad8c4cfb8de5ab60ba166120"
format_stamp: "Formatted at 2026-08-12 08:02:17 on dist-test-slave-49h3"
I20260812 08:02:17.476233 15992 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:17.504165 15992 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:17.504875 15992 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:17.505048 15992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:17.512490 15992 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:46295
I20260812 08:02:17.512547 16071 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:46295 every 8 connection(s)
I20260812 08:02:17.513521 15992 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:02:17.516361 16073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:17.521018   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 15992
I20260812 08:02:17.521157   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:02:17.521427 16073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120: Bootstrap starting.
I20260812 08:02:17.523852 16073 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:17.524524 16073 log.cc:826] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:17.526489 16073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120: No bootstrap required, opened a new log
I20260812 08:02:17.529440 16073 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b83ffedad8c4cfb8de5ab60ba166120" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 46295 } }
I20260812 08:02:17.529702 16073 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:17.529764 16073 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b83ffedad8c4cfb8de5ab60ba166120, State: Initialized, Role: FOLLOWER
I20260812 08:02:17.530201 16073 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b83ffedad8c4cfb8de5ab60ba166120" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 46295 } }
I20260812 08:02:17.530351 16073 raft_consensus.cc:399] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:17.530411 16073 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:17.530526 16073 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:17.531268 16073 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b83ffedad8c4cfb8de5ab60ba166120" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 46295 } }
I20260812 08:02:17.531592 16073 leader_election.cc:304] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b83ffedad8c4cfb8de5ab60ba166120; no voters: 
I20260812 08:02:17.531854 16073 leader_election.cc:290] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:02:17.531966 16078 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:17.532176 16078 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [term 1 LEADER]: Becoming Leader. State: Replica: 0b83ffedad8c4cfb8de5ab60ba166120, State: Running, Role: LEADER
I20260812 08:02:17.532495 16078 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b83ffedad8c4cfb8de5ab60ba166120" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 46295 } }
I20260812 08:02:17.532932 16073 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:02:17.534085 16079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b83ffedad8c4cfb8de5ab60ba166120" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b83ffedad8c4cfb8de5ab60ba166120" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 46295 } } }
I20260812 08:02:17.534276 16079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:17.534076 16081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b83ffedad8c4cfb8de5ab60ba166120. Latest consensus state: current_term: 1 leader_uuid: "0b83ffedad8c4cfb8de5ab60ba166120" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b83ffedad8c4cfb8de5ab60ba166120" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 46295 } } }
I20260812 08:02:17.534581 16081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:17.535267 16090 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:02:17.538143 16090 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:02:17.542753 16090 catalog_manager.cc:1383] Generated new cluster ID: dce9b307a55940af84bb3a99968a6a84
I20260812 08:02:17.542867 16090 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:02:17.569073 16090 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:02:17.569964 16090 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:02:17.582711 16090 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120: Generated new TSK 0
I20260812 08:02:17.583300 16090 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:02:17.592980   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:0
--local_ip_for_outbound_sockets=127.0.186.65
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:46295
--builtin_ntp_servers=127.0.186.84:45913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_incompatible_replica_management_is_fatal=true
--heartbeat_interval_ms=500
--raft_prepare_replacement_before_eviction=true with env {}
W20260812 08:02:17.704864 16103 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:17.705107 16103 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:17.705153 16103 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:17.708454 16103 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:17.708560 16103 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:02:17.712344 16103 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:45913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=0
--heartbeat_interval_ms=500
--tserver_master_addrs=127.0.186.126:46295
--never_fsync=true
--heap_profile_path=/tmp/kudu.16103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:17.713351 16103 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:17.714461 16103 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:17.720661 16109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:17.720757 16103 server_base.cc:1061] running on GCE node
W20260812 08:02:17.720737 16111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:17.720677 16113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:17.721277 16103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:17.721800 16103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:17.722941 16103 hybrid_clock.cc:648] HybridClock initialized: now 1786521737722927 us; error 29 us; skew 500 ppm
I20260812 08:02:17.724498 16103 webserver.cc:533] Webserver started at http://127.0.186.65:43663/ using document root <none> and password file <none>
I20260812 08:02:17.725015 16103 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:17.725068 16103 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:17.725266 16103 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:17.726778 16103 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "3d7455d7d86742448de07fcdd398b531"
format_stamp: "Formatted at 2026-08-12 08:02:17 on dist-test-slave-49h3"
I20260812 08:02:17.727242 16103 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "3d7455d7d86742448de07fcdd398b531"
format_stamp: "Formatted at 2026-08-12 08:02:17 on dist-test-slave-49h3"
I20260812 08:02:17.730358 16103 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:02:17.732316 16119 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:17.733215 16103 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:17.733312 16103 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "3d7455d7d86742448de07fcdd398b531"
format_stamp: "Formatted at 2026-08-12 08:02:17 on dist-test-slave-49h3"
I20260812 08:02:17.733397 16103 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:17.770221 16103 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:17.770933 16103 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:17.771132 16103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:17.771742 16103 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:17.772794 16103 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:17.772843 16103 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:17.772886 16103 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:17.772913 16103 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:17.781785 16103 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:45837
I20260812 08:02:17.781885 16256 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:45837 every 8 connection(s)
I20260812 08:02:17.782763 16103 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:02:17.790530   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 16103
I20260812 08:02:17.790666   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:02:17.790675 16257 heartbeater.cc:344] Connected to a master server at 127.0.186.126:46295
I20260812 08:02:17.790908 16257 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:17.791486 16257 heartbeater.cc:507] Master 127.0.186.126:46295 requested a full tablet report, sending...
I20260812 08:02:17.793052 16028 ts_manager.cc:194] Registered new tserver with Master: 3d7455d7d86742448de07fcdd398b531 (127.0.186.65:45837)
I20260812 08:02:17.794263 16028 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.65:35755
I20260812 08:02:17.804740   745 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260812 08:02:17.813493 16028 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45790:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
I20260812 08:02:17.829905 16172 tablet_service.cc:1511] Processing CreateTablet for tablet 18d7aa5029824d33a58f3c28211367f6 (DEFAULT_TABLE table=TestTable [id=96ff30b1839448f7b412f67c98aa5bff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:17.830931 16172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18d7aa5029824d33a58f3c28211367f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:17.835923 16270 tablet_bootstrap.cc:492] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531: Bootstrap starting.
I20260812 08:02:17.837914 16270 tablet_bootstrap.cc:654] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:17.838650 16270 log.cc:826] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:17.840416 16270 tablet_bootstrap.cc:492] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531: No bootstrap required, opened a new log
I20260812 08:02:17.840616 16270 ts_tablet_manager.cc:1403] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.000s
I20260812 08:02:17.843024 16270 raft_consensus.cc:359] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7455d7d86742448de07fcdd398b531" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45837 } }
I20260812 08:02:17.843214 16270 raft_consensus.cc:385] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:17.843273 16270 raft_consensus.cc:740] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d7455d7d86742448de07fcdd398b531, State: Initialized, Role: FOLLOWER
I20260812 08:02:17.843660 16270 consensus_queue.cc:260] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7455d7d86742448de07fcdd398b531" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45837 } }
I20260812 08:02:17.843782 16270 raft_consensus.cc:399] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:17.843847 16270 raft_consensus.cc:493] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:17.843957 16270 raft_consensus.cc:3060] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:17.844764 16270 raft_consensus.cc:515] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7455d7d86742448de07fcdd398b531" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45837 } }
I20260812 08:02:17.845116 16270 leader_election.cc:304] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d7455d7d86742448de07fcdd398b531; no voters: 
I20260812 08:02:17.845516 16270 leader_election.cc:290] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:02:17.845621 16272 raft_consensus.cc:2804] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:17.845853 16272 raft_consensus.cc:697] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [term 1 LEADER]: Becoming Leader. State: Replica: 3d7455d7d86742448de07fcdd398b531, State: Running, Role: LEADER
I20260812 08:02:17.846194 16257 heartbeater.cc:499] Master 127.0.186.126:46295 was elected leader, sending a full tablet report...
I20260812 08:02:17.846217 16272 consensus_queue.cc:237] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7455d7d86742448de07fcdd398b531" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45837 } }
I20260812 08:02:17.846815 16270 ts_tablet_manager.cc:1434] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260812 08:02:17.849887 16028 catalog_manager.cc:5719] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d7455d7d86742448de07fcdd398b531 (127.0.186.65). New cstate: current_term: 1 leader_uuid: "3d7455d7d86742448de07fcdd398b531" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7455d7d86742448de07fcdd398b531" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45837 } health_report { overall_health: HEALTHY } } }
I20260812 08:02:17.855840   745 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260812 08:02:17.857122   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 3d7455d7d86742448de07fcdd398b531 to finish bootstrapping
I20260812 08:02:17.862811   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 16103
I20260812 08:02:17.875800   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 15992
I20260812 08:02:17.885169   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:46295
--webserver_interface=127.0.186.126
--webserver_port=40809
--builtin_ntp_servers=127.0.186.84:45913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:46295
--raft_prepare_replacement_before_eviction=true with env {}
W20260812 08:02:17.996023 16277 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:17.996246 16277 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:17.996287 16277 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:17.999505 16277 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:02:17.999558 16277 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:17.999576 16277 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:02:17.999593 16277 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:02:18.003367 16277 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:45913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:46295
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:46295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=40809
--never_fsync=true
--heap_profile_path=/tmp/kudu.16277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:18.004400 16277 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:18.005467 16277 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:18.010641 16294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:18.010653 16289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:18.010658 16288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:18.010974 16277 server_base.cc:1061] running on GCE node
I20260812 08:02:18.011449 16277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:18.012282 16277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:18.013434 16277 hybrid_clock.cc:648] HybridClock initialized: now 1786521738013417 us; error 30 us; skew 500 ppm
I20260812 08:02:18.015046 16277 webserver.cc:533] Webserver started at http://127.0.186.126:40809/ using document root <none> and password file <none>
I20260812 08:02:18.015578 16277 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:18.015663 16277 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:18.018924 16277 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.001s	sys 0.002s
I20260812 08:02:18.020849 16303 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:18.021899 16277 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:18.022004 16277 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "0b83ffedad8c4cfb8de5ab60ba166120"
format_stamp: "Formatted at 2026-08-12 08:02:17 on dist-test-slave-49h3"
I20260812 08:02:18.022370 16277 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:18.046001 16277 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:18.046680 16277 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:18.046845 16277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:18.053812 16370 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:46295 every 8 connection(s)
I20260812 08:02:18.053823 16277 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:46295
I20260812 08:02:18.054869 16277 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:02:18.058785 16371 sys_catalog.cc:263] Verifying existing consensus state
I20260812 08:02:18.060243 16371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120: Bootstrap starting.
I20260812 08:02:18.061206   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 16277
I20260812 08:02:18.061643   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:45837
--local_ip_for_outbound_sockets=127.0.186.65
--tserver_master_addrs=127.0.186.126:46295
--webserver_port=43663
--webserver_interface=127.0.186.65
--builtin_ntp_servers=127.0.186.84:45913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_incompatible_replica_management_is_fatal=true
--heartbeat_interval_ms=500
--raft_prepare_replacement_before_eviction=true
--raft_prepare_replacement_before_eviction=false with env {}
I20260812 08:02:18.078118 16371 log.cc:826] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:18.087661 16371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=5 ignored=0} mutations{seen=2 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:02:18.088066 16371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120: Bootstrap complete.
I20260812 08:02:18.092691 16371 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b83ffedad8c4cfb8de5ab60ba166120" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 46295 } }
I20260812 08:02:18.093093 16371 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b83ffedad8c4cfb8de5ab60ba166120, State: Initialized, Role: FOLLOWER
I20260812 08:02:18.093706 16371 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b83ffedad8c4cfb8de5ab60ba166120" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 46295 } }
I20260812 08:02:18.093861 16371 raft_consensus.cc:399] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:18.093924 16371 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:18.094026 16371 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:02:18.095919 16371 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b83ffedad8c4cfb8de5ab60ba166120" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 46295 } }
I20260812 08:02:18.096292 16371 leader_election.cc:304] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b83ffedad8c4cfb8de5ab60ba166120; no voters: 
I20260812 08:02:18.096647 16371 leader_election.cc:290] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [CANDIDATE]: Term 2 election: Requested vote from peers 
I20260812 08:02:18.096773 16375 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:02:18.096992 16375 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [term 2 LEADER]: Becoming Leader. State: Replica: 0b83ffedad8c4cfb8de5ab60ba166120, State: Running, Role: LEADER
I20260812 08:02:18.097332 16375 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b83ffedad8c4cfb8de5ab60ba166120" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 46295 } }
I20260812 08:02:18.097867 16371 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:02:18.098637 16377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0b83ffedad8c4cfb8de5ab60ba166120" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b83ffedad8c4cfb8de5ab60ba166120" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 46295 } } }
I20260812 08:02:18.098749 16377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:18.099061 16378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b83ffedad8c4cfb8de5ab60ba166120. Latest consensus state: current_term: 2 leader_uuid: "0b83ffedad8c4cfb8de5ab60ba166120" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b83ffedad8c4cfb8de5ab60ba166120" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 46295 } } }
I20260812 08:02:18.099207 16378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120 [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:18.099514 16383 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:02:18.103240 16383 catalog_manager.cc:679] Loaded metadata for table TestTable [id=96ff30b1839448f7b412f67c98aa5bff]
I20260812 08:02:18.104722 16383 tablet_loader.cc:96] loaded metadata for tablet 18d7aa5029824d33a58f3c28211367f6 (table TestTable [id=96ff30b1839448f7b412f67c98aa5bff])
I20260812 08:02:18.104889 16383 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:02:18.105768 16383 catalog_manager.cc:1295] Loaded cluster ID: dce9b307a55940af84bb3a99968a6a84
I20260812 08:02:18.105830 16383 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:02:18.108280 16383 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:02:18.109031 16383 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 0b83ffedad8c4cfb8de5ab60ba166120: Loaded TSK: 0
I20260812 08:02:18.110036 16383 catalog_manager.cc:1550] Initializing in-progress tserver states...
W20260812 08:02:18.186151 16373 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:18.186369 16373 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:18.186431 16373 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:18.189774 16373 flags.cc:432] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20260812 08:02:18.189846 16373 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:18.189934 16373 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:02:18.193733 16373 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:45913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:45837
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=43663
--heartbeat_interval_ms=500
--tserver_master_addrs=127.0.186.126:46295
--never_fsync=true
--heap_profile_path=/tmp/kudu.16373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:18.194821 16373 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:18.195936 16373 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:18.202530 16408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:18.202592 16404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:18.202817 16373 server_base.cc:1061] running on GCE node
W20260812 08:02:18.202550 16406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:18.203358 16373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:18.204002 16373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:18.205193 16373 hybrid_clock.cc:648] HybridClock initialized: now 1786521738205164 us; error 40 us; skew 500 ppm
I20260812 08:02:18.207324 16373 webserver.cc:533] Webserver started at http://127.0.186.65:43663/ using document root <none> and password file <none>
I20260812 08:02:18.208036 16373 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:18.208142 16373 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:18.212702 16373 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.001s	sys 0.001s
I20260812 08:02:18.215361 16415 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:18.216637 16373 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260812 08:02:18.216771 16373 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "3d7455d7d86742448de07fcdd398b531"
format_stamp: "Formatted at 2026-08-12 08:02:17 on dist-test-slave-49h3"
I20260812 08:02:18.217300 16373 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:18.232856 16373 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:18.233711 16373 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:18.233915 16373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:18.234638 16373 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:18.236173 16423 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:02:18.239624 16373 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:02:18.239701 16373 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.001s	sys 0.000s
I20260812 08:02:18.239777 16373 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:02:18.241724 16373 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:02:18.241784 16373 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:18.242151 16423 tablet_bootstrap.cc:492] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531: Bootstrap starting.
I20260812 08:02:18.253791 16373 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:45837
I20260812 08:02:18.253832 16555 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:45837 every 8 connection(s)
I20260812 08:02:18.255064 16373 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:02:18.257341 16423 log.cc:826] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:18.259138   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 16373
I20260812 08:02:18.259953 16423 tablet_bootstrap.cc:492] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:02:18.260316 16423 tablet_bootstrap.cc:492] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531: Bootstrap complete.
I20260812 08:02:18.261598 16423 ts_tablet_manager.cc:1403] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531: Time spent bootstrapping tablet: real 0.020s	user 0.004s	sys 0.015s
I20260812 08:02:18.264293 16423 raft_consensus.cc:359] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7455d7d86742448de07fcdd398b531" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45837 } }
I20260812 08:02:18.264614 16423 raft_consensus.cc:740] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d7455d7d86742448de07fcdd398b531, State: Initialized, Role: FOLLOWER
I20260812 08:02:18.265108 16423 consensus_queue.cc:260] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7455d7d86742448de07fcdd398b531" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45837 } }
I20260812 08:02:18.265254 16423 raft_consensus.cc:399] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:18.265314 16423 raft_consensus.cc:493] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:18.265426 16423 raft_consensus.cc:3060] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:02:18.267534 16423 raft_consensus.cc:515] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7455d7d86742448de07fcdd398b531" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45837 } }
I20260812 08:02:18.267892 16423 leader_election.cc:304] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d7455d7d86742448de07fcdd398b531; no voters: 
I20260812 08:02:18.268260 16423 leader_election.cc:290] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [CANDIDATE]: Term 2 election: Requested vote from peers 
I20260812 08:02:18.268396 16564 raft_consensus.cc:2804] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:02:18.268647 16564 raft_consensus.cc:697] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [term 2 LEADER]: Becoming Leader. State: Replica: 3d7455d7d86742448de07fcdd398b531, State: Running, Role: LEADER
I20260812 08:02:18.269012 16564 consensus_queue.cc:237] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7455d7d86742448de07fcdd398b531" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45837 } }
I20260812 08:02:18.269106 16423 ts_tablet_manager.cc:1434] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260812 08:02:18.270138 16556 heartbeater.cc:344] Connected to a master server at 127.0.186.126:46295
I20260812 08:02:18.270373 16556 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:18.270880 16556 heartbeater.cc:507] Master 127.0.186.126:46295 requested a full tablet report, sending...
W20260812 08:02:18.273007 16323 master_service.cc:413] Configuration error: replica replacement scheme (EVICT_FIRST) of the tablet server 3d7455d7d86742448de07fcdd398b531 at 127.0.186.65:52873 differs from the catalog manager's (PREPARE_REPLACEMENT_BEFORE_EVICTION); they must be run with the same scheme (controlled by the --raft_prepare_replacement_before_eviction flag)
W20260812 08:02:18.273830 16556 heartbeater.cc:646] Failed to heartbeat to 127.0.186.126:46295 (0 consecutive failures): Configuration error: replica replacement scheme (EVICT_FIRST) of the tablet server 3d7455d7d86742448de07fcdd398b531 at 127.0.186.65:52873 differs from the catalog manager's (PREPARE_REPLACEMENT_BEFORE_EVICTION); they must be run with the same scheme (controlled by the --raft_prepare_replacement_before_eviction flag)
F20260812 08:02:18.273895 16556 heartbeater.cc:685] master detected incompatibility: Configuration error: replica replacement scheme (EVICT_FIRST) of the tablet server 3d7455d7d86742448de07fcdd398b531 at 127.0.186.65:52873 differs from the catalog manager's (PREPARE_REPLACEMENT_BEFORE_EVICTION); they must be run with the same scheme (controlled by the --raft_prepare_replacement_before_eviction flag)
*** Check failure stack trace: ***
    @     0x7fc2a6fead5d  google::LogMessage::Fail() at ??:0
    @     0x7fc2a6feeb23  google::LogMessage::SendToLog() at ??:0
    @     0x7fc2a6fea75c  google::LogMessage::Flush() at ??:0
    @     0x7fc2a6febee9  google::LogMessageFatal::~LogMessageFatal() at ??:0
    @     0x7fc2ab56db55  kudu::tserver::Heartbeater::Thread::RunThread() at ??:0
    @     0x7fc2ab56e157  _ZZN4kudu7tserver11Heartbeater6Thread5StartEvENKUlvE_clEv at ??:0
    @     0x7fc2ab56fc3a  _ZNSt17_Function_handlerIFvvEZN4kudu7tserver11Heartbeater6Thread5StartEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
    @     0x55883fac644e  std::function<>::operator()() at ??:0
    @     0x7fc2a84ba712  kudu::Thread::SuperviseThread() at ??:0
    @     0x7fc2a8e396db  start_thread at ??:0
    @     0x7fc2a61a161f  clone at ??:0
I20260812 08:02:19.975777   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:45837
--local_ip_for_outbound_sockets=127.0.186.65
--tserver_master_addrs=127.0.186.126:46295
--webserver_port=43663
--webserver_interface=127.0.186.65
--builtin_ntp_servers=127.0.186.84:45913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--heartbeat_incompatible_replica_management_is_fatal=true
--heartbeat_interval_ms=500
--raft_prepare_replacement_before_eviction=true
--heartbeat_inject_required_feature_flag=999 with env {}
W20260812 08:02:20.080886 16584 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:20.081085 16584 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:20.081126 16584 flags.cc:432] Enabled unsafe flag: --heartbeat_inject_required_feature_flag=999
W20260812 08:02:20.081176 16584 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:20.084378 16584 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:20.084491 16584 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:02:20.088176 16584 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:45913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:45837
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=43663
--heartbeat_inject_required_feature_flag=999
--heartbeat_interval_ms=500
--tserver_master_addrs=127.0.186.126:46295
--never_fsync=true
--heap_profile_path=/tmp/kudu.16584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:20.089154 16584 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:20.090267 16584 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:20.096129 16592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:20.096194 16590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:20.096345 16584 server_base.cc:1061] running on GCE node
W20260812 08:02:20.096140 16589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:20.096726 16584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:20.097229 16584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:20.098373 16584 hybrid_clock.cc:648] HybridClock initialized: now 1786521740098346 us; error 40 us; skew 500 ppm
I20260812 08:02:20.100144 16584 webserver.cc:533] Webserver started at http://127.0.186.65:43663/ using document root <none> and password file <none>
I20260812 08:02:20.100682 16584 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:20.100734 16584 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:20.104171 16584 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:20.106113 16600 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:20.107115 16584 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:20.107208 16584 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "3d7455d7d86742448de07fcdd398b531"
format_stamp: "Formatted at 2026-08-12 08:02:17 on dist-test-slave-49h3"
I20260812 08:02:20.107564 16584 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:20.118057 16584 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:20.118597 16584 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:20.118736 16584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:20.119234 16584 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:20.120365 16609 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:02:20.123615 16584 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:02:20.123677 16584 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.003s	user 0.000s	sys 0.000s
I20260812 08:02:20.123723 16584 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:02:20.125134 16584 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:02:20.125178 16584 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
I20260812 08:02:20.125535 16609 tablet_bootstrap.cc:492] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531: Bootstrap starting.
I20260812 08:02:20.135530 16584 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:45837
I20260812 08:02:20.135583 16751 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:45837 every 8 connection(s)
I20260812 08:02:20.136533 16584 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:02:20.139963 16609 log.cc:826] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:20.142031   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 16584
I20260812 08:02:20.142853 16609 tablet_bootstrap.cc:492] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:02:20.143247 16609 tablet_bootstrap.cc:492] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531: Bootstrap complete.
I20260812 08:02:20.144264 16609 ts_tablet_manager.cc:1403] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.004s
I20260812 08:02:20.145109 16752 heartbeater.cc:344] Connected to a master server at 127.0.186.126:46295
I20260812 08:02:20.145340 16752 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:20.145897 16752 heartbeater.cc:507] Master 127.0.186.126:46295 requested a full tablet report, sending...
I20260812 08:02:20.146842 16609 raft_consensus.cc:359] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7455d7d86742448de07fcdd398b531" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45837 } }
I20260812 08:02:20.147184 16609 raft_consensus.cc:740] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d7455d7d86742448de07fcdd398b531, State: Initialized, Role: FOLLOWER
W20260812 08:02:20.147558 16305 messenger.cc:376] Unable to handle RPC call: Not implemented: call requires unsupported application feature flags: 999
I20260812 08:02:20.147588 16609 consensus_queue.cc:260] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7455d7d86742448de07fcdd398b531" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45837 } }
I20260812 08:02:20.147764 16609 raft_consensus.cc:399] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:20.147826 16609 raft_consensus.cc:493] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:20.147935 16609 raft_consensus.cc:3060] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [term 2 FOLLOWER]: Advancing to term 3
W20260812 08:02:20.148130 16752 heartbeater.cc:646] Failed to heartbeat to 127.0.186.126:46295 (0 consecutive failures): Remote error: Failed to send heartbeat to master: unsupported feature flags
F20260812 08:02:20.149501 16752 heartbeater.cc:685] master does not support required feature flag(s) : Remote error: Failed to send heartbeat to master: unsupported feature flags
*** Check failure stack trace: ***
I20260812 08:02:20.150584 16609 raft_consensus.cc:515] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7455d7d86742448de07fcdd398b531" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45837 } }
I20260812 08:02:20.150945 16609 leader_election.cc:304] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d7455d7d86742448de07fcdd398b531; no voters: 
I20260812 08:02:20.151262 16609 leader_election.cc:290] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [CANDIDATE]: Term 3 election: Requested vote from peers 
I20260812 08:02:20.151397 16759 raft_consensus.cc:2804] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 08:02:20.151674 16759 raft_consensus.cc:697] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [term 3 LEADER]: Becoming Leader. State: Replica: 3d7455d7d86742448de07fcdd398b531, State: Running, Role: LEADER
I20260812 08:02:20.152087 16759 consensus_queue.cc:237] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7455d7d86742448de07fcdd398b531" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 45837 } }
I20260812 08:02:20.152290 16609 ts_tablet_manager.cc:1434] T 18d7aa5029824d33a58f3c28211367f6 P 3d7455d7d86742448de07fcdd398b531: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
    @     0x7fcadb677d5d  google::LogMessage::Fail() at ??:0
    @     0x7fcadb67bb23  google::LogMessage::SendToLog() at ??:0
    @     0x7fcadb67775c  google::LogMessage::Flush() at ??:0
    @     0x7fcadb678ee9  google::LogMessageFatal::~LogMessageFatal() at ??:0
    @     0x7fcadfbfab55  kudu::tserver::Heartbeater::Thread::RunThread() at ??:0
    @     0x7fcadfbfb157  _ZZN4kudu7tserver11Heartbeater6Thread5StartEvENKUlvE_clEv at ??:0
    @     0x7fcadfbfcc3a  _ZNSt17_Function_handlerIFvvEZN4kudu7tserver11Heartbeater6Thread5StartEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
    @     0x55f1829f944e  std::function<>::operator()() at ??:0
    @     0x7fcadcb47712  kudu::Thread::SuperviseThread() at ??:0
    @     0x7fcadd4c66db  start_thread at ??:0
    @     0x7fcada82e61f  clone at ??:0
I20260812 08:02:21.855264   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 16277
2026-08-12T08:02:21Z chronyd exiting
[       OK ] IncompatibleReplicaReplacementSchemesITest.MasterAndTserverMisconfig/3 (4544 ms)
[----------] 4 tests from IncompatibleReplicaReplacementSchemesITest (17396 ms total)

[----------] 6 tests from ReplicaBehindWalGcThresholdITest
[ RUN      ] ReplicaBehindWalGcThresholdITest.ReplicaReplacement/0
I20260812 08:02:21.881352   745 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:02:21.881402   745 ts_itest-base.cc:116] --------------
I20260812 08:02:21.881413   745 ts_itest-base.cc:117] 3 tablet servers
I20260812 08:02:21.881422   745 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:02:21.881431   745 ts_itest-base.cc:119] --------------
2026-08-12T08:02:21Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:02:21Z Disabled control of system clock
I20260812 08:02:21.890547   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:44369
--webserver_interface=127.0.186.126
--webserver_port=0
--builtin_ntp_servers=127.0.186.84:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:44369
--raft_prepare_replacement_before_eviction=true
--master_add_server_when_underreplicated=false with env {}
W20260812 08:02:21.996784 16792 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:21.996996 16792 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:21.997049 16792 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:22.000285 16792 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:02:22.000342 16792 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:22.000370 16792 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:02:22.000393 16792 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:02:22.004141 16792 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--master_add_server_when_underreplicated=false
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:44369
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:44369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.16792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:22.005223 16792 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:22.006376 16792 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:22.011698 16802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:22.011696 16798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:22.011684 16799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:22.012086 16792 server_base.cc:1061] running on GCE node
I20260812 08:02:22.012576 16792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:22.013422 16792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:22.014598 16792 hybrid_clock.cc:648] HybridClock initialized: now 1786521742014587 us; error 24 us; skew 500 ppm
I20260812 08:02:22.016309 16792 webserver.cc:533] Webserver started at http://127.0.186.126:46419/ using document root <none> and password file <none>
I20260812 08:02:22.016844 16792 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:22.016897 16792 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:22.017084 16792 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:22.018605 16792 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "b163af25a36743799e4f67ab23f0b8cf"
format_stamp: "Formatted at 2026-08-12 08:02:22 on dist-test-slave-49h3"
I20260812 08:02:22.019070 16792 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "b163af25a36743799e4f67ab23f0b8cf"
format_stamp: "Formatted at 2026-08-12 08:02:22 on dist-test-slave-49h3"
I20260812 08:02:22.022218 16792 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:02:22.024295 16808 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:22.025277 16792 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:22.025386 16792 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "b163af25a36743799e4f67ab23f0b8cf"
format_stamp: "Formatted at 2026-08-12 08:02:22 on dist-test-slave-49h3"
I20260812 08:02:22.025499 16792 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:22.050633 16792 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:22.051339 16792 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:22.051528 16792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:22.058936 16792 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:44369
I20260812 08:02:22.058940 16872 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:44369 every 8 connection(s)
I20260812 08:02:22.059950 16792 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:02:22.062837 16873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:22.065681   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 16792
I20260812 08:02:22.065814   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:02:22.068270 16873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf: Bootstrap starting.
I20260812 08:02:22.070381 16873 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:22.071056 16873 log.cc:826] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:22.072781 16873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf: No bootstrap required, opened a new log
I20260812 08:02:22.075171 16873 raft_consensus.cc:359] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b163af25a36743799e4f67ab23f0b8cf" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44369 } }
I20260812 08:02:22.075349 16873 raft_consensus.cc:385] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:22.075407 16873 raft_consensus.cc:740] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b163af25a36743799e4f67ab23f0b8cf, State: Initialized, Role: FOLLOWER
I20260812 08:02:22.075865 16873 consensus_queue.cc:260] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b163af25a36743799e4f67ab23f0b8cf" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44369 } }
I20260812 08:02:22.076017 16873 raft_consensus.cc:399] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:22.076092 16873 raft_consensus.cc:493] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:22.076202 16873 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:22.077136 16873 raft_consensus.cc:515] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b163af25a36743799e4f67ab23f0b8cf" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44369 } }
I20260812 08:02:22.077500 16873 leader_election.cc:304] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b163af25a36743799e4f67ab23f0b8cf; no voters: 
I20260812 08:02:22.077775 16873 leader_election.cc:290] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:02:22.077872 16879 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:22.078023 16879 raft_consensus.cc:697] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [term 1 LEADER]: Becoming Leader. State: Replica: b163af25a36743799e4f67ab23f0b8cf, State: Running, Role: LEADER
I20260812 08:02:22.078354 16879 consensus_queue.cc:237] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b163af25a36743799e4f67ab23f0b8cf" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44369 } }
I20260812 08:02:22.078924 16873 sys_catalog.cc:565] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:02:22.079497 16881 sys_catalog.cc:455] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader b163af25a36743799e4f67ab23f0b8cf. Latest consensus state: current_term: 1 leader_uuid: "b163af25a36743799e4f67ab23f0b8cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b163af25a36743799e4f67ab23f0b8cf" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44369 } } }
I20260812 08:02:22.079661 16881 sys_catalog.cc:458] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:22.079941 16880 sys_catalog.cc:455] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b163af25a36743799e4f67ab23f0b8cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b163af25a36743799e4f67ab23f0b8cf" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44369 } } }
I20260812 08:02:22.080104 16880 sys_catalog.cc:458] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:22.080106 16884 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:02:22.083462 16884 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:02:22.088505 16884 catalog_manager.cc:1383] Generated new cluster ID: 5fd11bed16b64cc19da7201e7d13fedb
I20260812 08:02:22.088569 16884 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:02:22.096721 16884 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:02:22.097780 16884 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:02:22.109097 16884 catalog_manager.cc:6092] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf: Generated new TSK 0
I20260812 08:02:22.109673 16884 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:02:22.118400   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:0
--local_ip_for_outbound_sockets=127.0.186.65
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:44369
--builtin_ntp_servers=127.0.186.84:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20260812 08:02:22.228271 16902 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:02:22.228511 16902 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:22.228533 16902 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:22.228590 16902 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:22.231979 16902 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:02:22.232053 16902 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:02:22.232087 16902 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:22.232157 16902 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:02:22.232197 16902 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:02:22.236195 16902 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:44369
--never_fsync=true
--heap_profile_path=/tmp/kudu.16902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:22.237288 16902 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:22.238441 16902 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:22.244655 16911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:22.244753 16909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:22.244660 16908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:22.245098 16902 server_base.cc:1061] running on GCE node
I20260812 08:02:22.245498 16902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:22.246147 16902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:22.247326 16902 hybrid_clock.cc:648] HybridClock initialized: now 1786521742247313 us; error 31 us; skew 500 ppm
I20260812 08:02:22.248917 16902 webserver.cc:533] Webserver started at http://127.0.186.65:46475/ using document root <none> and password file <none>
I20260812 08:02:22.249485 16902 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:22.249547 16902 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:22.249727 16902 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:22.251220 16902 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "a06397457dab4d5fa7f14e27026e675c"
format_stamp: "Formatted at 2026-08-12 08:02:22 on dist-test-slave-49h3"
I20260812 08:02:22.251647 16902 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "a06397457dab4d5fa7f14e27026e675c"
format_stamp: "Formatted at 2026-08-12 08:02:22 on dist-test-slave-49h3"
I20260812 08:02:22.254846 16902 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:02:22.257216 16920 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:22.258224 16902 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:22.258337 16902 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "a06397457dab4d5fa7f14e27026e675c"
format_stamp: "Formatted at 2026-08-12 08:02:22 on dist-test-slave-49h3"
I20260812 08:02:22.258430 16902 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:22.272773 16902 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:22.273360 16902 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:22.273564 16902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:22.274119 16902 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:22.275115 16902 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:22.275166 16902 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:22.275216 16902 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:22.275245 16902 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:22.283849 16902 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:32871
I20260812 08:02:22.283898 17052 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:32871 every 8 connection(s)
I20260812 08:02:22.284758 16902 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:02:22.285315   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 16902
I20260812 08:02:22.285440   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:02:22.288190   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.66:0
--local_ip_for_outbound_sockets=127.0.186.66
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:44369
--builtin_ntp_servers=127.0.186.84:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20260812 08:02:22.294272 17053 heartbeater.cc:344] Connected to a master server at 127.0.186.126:44369
I20260812 08:02:22.294490 17053 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:22.294961 17053 heartbeater.cc:507] Master 127.0.186.126:44369 requested a full tablet report, sending...
I20260812 08:02:22.296370 16828 ts_manager.cc:194] Registered new tserver with Master: a06397457dab4d5fa7f14e27026e675c (127.0.186.65:32871)
I20260812 08:02:22.297519 16828 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.65:56089
W20260812 08:02:22.392421 17056 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:02:22.392668 17056 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:22.392709 17056 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:22.392773 17056 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:22.395992 17056 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:02:22.396052 17056 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:02:22.396099 17056 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:22.396181 17056 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:02:22.396215 17056 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.66
I20260812 08:02:22.400027 17056 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:44369
--never_fsync=true
--heap_profile_path=/tmp/kudu.17056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:22.401081 17056 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:22.402215 17056 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 08:02:22.408500 17056 server_base.cc:1061] running on GCE node
W20260812 08:02:22.408447 17065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:22.408516 17063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:22.408425 17067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:22.408944 17056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:22.409495 17056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:22.410682 17056 hybrid_clock.cc:648] HybridClock initialized: now 1786521742410665 us; error 31 us; skew 500 ppm
I20260812 08:02:22.412403 17056 webserver.cc:533] Webserver started at http://127.0.186.66:34613/ using document root <none> and password file <none>
I20260812 08:02:22.412943 17056 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:22.412997 17056 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:22.413180 17056 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:22.414750 17056 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "cf9d4d161b86467d95a853b11bb103d9"
format_stamp: "Formatted at 2026-08-12 08:02:22 on dist-test-slave-49h3"
I20260812 08:02:22.415211 17056 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "cf9d4d161b86467d95a853b11bb103d9"
format_stamp: "Formatted at 2026-08-12 08:02:22 on dist-test-slave-49h3"
I20260812 08:02:22.418489 17056 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.001s
I20260812 08:02:22.420444 17075 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:22.421351 17056 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 08:02:22.421475 17056 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "cf9d4d161b86467d95a853b11bb103d9"
format_stamp: "Formatted at 2026-08-12 08:02:22 on dist-test-slave-49h3"
I20260812 08:02:22.421557 17056 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:22.448668 17056 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:22.449349 17056 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:22.449524 17056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:22.450107 17056 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:22.451145 17056 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:22.451195 17056 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:22.451248 17056 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:22.451275 17056 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:22.460129 17056 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.66:33619
I20260812 08:02:22.460160 17215 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.66:33619 every 8 connection(s)
I20260812 08:02:22.461130 17056 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:02:22.464957   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 17056
I20260812 08:02:22.465087   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:02:22.467716   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.67:0
--local_ip_for_outbound_sockets=127.0.186.67
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:44369
--builtin_ntp_servers=127.0.186.84:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20260812 08:02:22.470639 17216 heartbeater.cc:344] Connected to a master server at 127.0.186.126:44369
I20260812 08:02:22.470919 17216 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:22.471514 17216 heartbeater.cc:507] Master 127.0.186.126:44369 requested a full tablet report, sending...
I20260812 08:02:22.472648 16828 ts_manager.cc:194] Registered new tserver with Master: cf9d4d161b86467d95a853b11bb103d9 (127.0.186.66:33619)
I20260812 08:02:22.473213 16828 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.66:44729
W20260812 08:02:22.574240 17219 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:02:22.574484 17219 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:22.574522 17219 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:22.574579 17219 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:22.577852 17219 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:02:22.577903 17219 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:02:22.577939 17219 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:22.578025 17219 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:02:22.578058 17219 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.67
I20260812 08:02:22.581965 17219 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:44369
--never_fsync=true
--heap_profile_path=/tmp/kudu.17219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:22.583029 17219 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:22.584189 17219 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:22.590428 17229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:22.590386 17225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:22.590467 17227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:22.590719 17219 server_base.cc:1061] running on GCE node
I20260812 08:02:22.591071 17219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:22.591562 17219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:22.592724 17219 hybrid_clock.cc:648] HybridClock initialized: now 1786521742592705 us; error 27 us; skew 500 ppm
I20260812 08:02:22.594336 17219 webserver.cc:533] Webserver started at http://127.0.186.67:36173/ using document root <none> and password file <none>
I20260812 08:02:22.594861 17219 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:22.594919 17219 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:22.595091 17219 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:22.596534 17219 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "1d06c98078b5474a96ddc860047b8662"
format_stamp: "Formatted at 2026-08-12 08:02:22 on dist-test-slave-49h3"
I20260812 08:02:22.596994 17219 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "1d06c98078b5474a96ddc860047b8662"
format_stamp: "Formatted at 2026-08-12 08:02:22 on dist-test-slave-49h3"
I20260812 08:02:22.600107 17219 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:02:22.602092 17236 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:22.603029 17219 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:22.603132 17219 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "1d06c98078b5474a96ddc860047b8662"
format_stamp: "Formatted at 2026-08-12 08:02:22 on dist-test-slave-49h3"
I20260812 08:02:22.603214 17219 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:22.621941 17219 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:22.622545 17219 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:22.622710 17219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:22.623241 17219 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:22.624234 17219 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:22.624284 17219 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:22.624336 17219 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:22.624359 17219 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:22.633263 17219 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.67:42163
I20260812 08:02:22.633296 17369 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.67:42163 every 8 connection(s)
I20260812 08:02:22.634310 17219 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:02:22.634913   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 17219
I20260812 08:02:22.635013   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:02:22.642508 17370 heartbeater.cc:344] Connected to a master server at 127.0.186.126:44369
I20260812 08:02:22.642725 17370 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:22.643175 17370 heartbeater.cc:507] Master 127.0.186.126:44369 requested a full tablet report, sending...
I20260812 08:02:22.644121 16828 ts_manager.cc:194] Registered new tserver with Master: 1d06c98078b5474a96ddc860047b8662 (127.0.186.67:42163)
I20260812 08:02:22.644630 16828 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.67:39593
I20260812 08:02:22.649169   745 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:02:22.657164 16828 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56206:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
W20260812 08:02:22.659592 16828 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 08:02:22.675957 17136 tablet_service.cc:1511] Processing CreateTablet for tablet 679b4d04e0bd4be9a0707f328981fd47 (DEFAULT_TABLE table=TestTable [id=d92e5c0a80554ce3b7efbd68d45aae49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:22.675976 16979 tablet_service.cc:1511] Processing CreateTablet for tablet 679b4d04e0bd4be9a0707f328981fd47 (DEFAULT_TABLE table=TestTable [id=d92e5c0a80554ce3b7efbd68d45aae49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:22.675976 17295 tablet_service.cc:1511] Processing CreateTablet for tablet 679b4d04e0bd4be9a0707f328981fd47 (DEFAULT_TABLE table=TestTable [id=d92e5c0a80554ce3b7efbd68d45aae49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:22.676877 17136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 679b4d04e0bd4be9a0707f328981fd47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:22.676900 17295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 679b4d04e0bd4be9a0707f328981fd47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:22.676900 16979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 679b4d04e0bd4be9a0707f328981fd47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:22.682024 17388 tablet_bootstrap.cc:492] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9: Bootstrap starting.
I20260812 08:02:22.682946 17389 tablet_bootstrap.cc:492] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c: Bootstrap starting.
I20260812 08:02:22.683187 17390 tablet_bootstrap.cc:492] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662: Bootstrap starting.
I20260812 08:02:22.684623 17388 tablet_bootstrap.cc:654] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:22.685034 17390 tablet_bootstrap.cc:654] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:22.685034 17389 tablet_bootstrap.cc:654] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:22.685333 17388 log.cc:826] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:22.685849 17389 log.cc:826] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:22.685849 17390 log.cc:826] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:22.686918 17388 tablet_bootstrap.cc:492] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9: No bootstrap required, opened a new log
I20260812 08:02:22.687083 17388 ts_tablet_manager.cc:1403] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.000s
I20260812 08:02:22.687274 17389 tablet_bootstrap.cc:492] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c: No bootstrap required, opened a new log
I20260812 08:02:22.687438 17389 ts_tablet_manager.cc:1403] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.000s
I20260812 08:02:22.689505 17388 raft_consensus.cc:359] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 32871 } }
I20260812 08:02:22.689791 17388 raft_consensus.cc:385] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:22.689903 17388 raft_consensus.cc:740] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf9d4d161b86467d95a853b11bb103d9, State: Initialized, Role: FOLLOWER
I20260812 08:02:22.689817 17389 raft_consensus.cc:359] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 32871 } }
I20260812 08:02:22.689997 17389 raft_consensus.cc:385] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:22.690057 17389 raft_consensus.cc:740] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06397457dab4d5fa7f14e27026e675c, State: Initialized, Role: FOLLOWER
I20260812 08:02:22.690399 17388 consensus_queue.cc:260] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 32871 } }
I20260812 08:02:22.690519 17390 tablet_bootstrap.cc:492] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662: No bootstrap required, opened a new log
I20260812 08:02:22.690467 17389 consensus_queue.cc:260] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 32871 } }
I20260812 08:02:22.690765 17390 ts_tablet_manager.cc:1403] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662: Time spent bootstrapping tablet: real 0.008s	user 0.004s	sys 0.000s
I20260812 08:02:22.691052 17216 heartbeater.cc:499] Master 127.0.186.126:44369 was elected leader, sending a full tablet report...
I20260812 08:02:22.691252 17388 ts_tablet_manager.cc:1434] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:02:22.691277 17389 ts_tablet_manager.cc:1434] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260812 08:02:22.691598 17053 heartbeater.cc:499] Master 127.0.186.126:44369 was elected leader, sending a full tablet report...
I20260812 08:02:22.694058 17390 raft_consensus.cc:359] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 32871 } }
I20260812 08:02:22.694301 17390 raft_consensus.cc:385] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:22.694363 17390 raft_consensus.cc:740] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d06c98078b5474a96ddc860047b8662, State: Initialized, Role: FOLLOWER
I20260812 08:02:22.694788 17390 consensus_queue.cc:260] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 32871 } }
I20260812 08:02:22.695626 17390 ts_tablet_manager.cc:1434] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:02:22.695652 17370 heartbeater.cc:499] Master 127.0.186.126:44369 was elected leader, sending a full tablet report...
W20260812 08:02:22.736032 17371 tablet.cc:2406] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:02:22.762470 17217 tablet.cc:2406] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:02:22.786443 17054 tablet.cc:2406] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:02:22.798900 17396 raft_consensus.cc:493] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:02:22.799111 17396 raft_consensus.cc:515] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 32871 } }
I20260812 08:02:22.800293 17396 leader_election.cc:290] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d06c98078b5474a96ddc860047b8662 (127.0.186.67:42163), cf9d4d161b86467d95a853b11bb103d9 (127.0.186.66:33619)
I20260812 08:02:22.804970 17160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679b4d04e0bd4be9a0707f328981fd47" candidate_uuid: "a06397457dab4d5fa7f14e27026e675c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf9d4d161b86467d95a853b11bb103d9" is_pre_election: true
I20260812 08:02:22.805315 17160 raft_consensus.cc:2468] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06397457dab4d5fa7f14e27026e675c in term 0.
I20260812 08:02:22.805326 17318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679b4d04e0bd4be9a0707f328981fd47" candidate_uuid: "a06397457dab4d5fa7f14e27026e675c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d06c98078b5474a96ddc860047b8662" is_pre_election: true
I20260812 08:02:22.805603 17318 raft_consensus.cc:2468] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06397457dab4d5fa7f14e27026e675c in term 0.
I20260812 08:02:22.805868 16924 leader_election.cc:304] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06397457dab4d5fa7f14e27026e675c, cf9d4d161b86467d95a853b11bb103d9; no voters: 
I20260812 08:02:22.806115 17396 raft_consensus.cc:2804] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:02:22.806185 17396 raft_consensus.cc:493] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:02:22.806216 17396 raft_consensus.cc:3060] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:22.807150 17396 raft_consensus.cc:515] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 32871 } }
I20260812 08:02:22.807518 17396 leader_election.cc:290] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [CANDIDATE]: Term 1 election: Requested vote from peers 1d06c98078b5474a96ddc860047b8662 (127.0.186.67:42163), cf9d4d161b86467d95a853b11bb103d9 (127.0.186.66:33619)
I20260812 08:02:22.807830 17318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679b4d04e0bd4be9a0707f328981fd47" candidate_uuid: "a06397457dab4d5fa7f14e27026e675c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d06c98078b5474a96ddc860047b8662"
I20260812 08:02:22.807934 17160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679b4d04e0bd4be9a0707f328981fd47" candidate_uuid: "a06397457dab4d5fa7f14e27026e675c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf9d4d161b86467d95a853b11bb103d9"
I20260812 08:02:22.807984 17318 raft_consensus.cc:3060] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:22.808046 17160 raft_consensus.cc:3060] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:22.809041 17160 raft_consensus.cc:2468] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a06397457dab4d5fa7f14e27026e675c in term 1.
I20260812 08:02:22.809041 17318 raft_consensus.cc:2468] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a06397457dab4d5fa7f14e27026e675c in term 1.
I20260812 08:02:22.809351 16923 leader_election.cc:304] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d06c98078b5474a96ddc860047b8662, a06397457dab4d5fa7f14e27026e675c; no voters: 
I20260812 08:02:22.809526 17396 raft_consensus.cc:2804] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:22.809721 17396 raft_consensus.cc:697] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [term 1 LEADER]: Becoming Leader. State: Replica: a06397457dab4d5fa7f14e27026e675c, State: Running, Role: LEADER
I20260812 08:02:22.810009 17396 consensus_queue.cc:237] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 32871 } }
I20260812 08:02:22.812970 16828 catalog_manager.cc:5719] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c reported cstate change: term changed from 0 to 1, leader changed from <none> to a06397457dab4d5fa7f14e27026e675c (127.0.186.65). New cstate: current_term: 1 leader_uuid: "a06397457dab4d5fa7f14e27026e675c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 32871 } health_report { overall_health: HEALTHY } } }
I20260812 08:02:22.819653   745 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:02:22.820935   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver a06397457dab4d5fa7f14e27026e675c to finish bootstrapping
I20260812 08:02:22.825649   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver cf9d4d161b86467d95a853b11bb103d9 to finish bootstrapping
I20260812 08:02:22.829945   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 1d06c98078b5474a96ddc860047b8662 to finish bootstrapping
I20260812 08:02:22.844187   745 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 0.0 1.1
I20260812 08:02:22.947286   745 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 0.0 1.1
I20260812 08:02:23.150259   745 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 0.0 1.1
I20260812 08:02:23.259701 17396 consensus_queue.cc:1048] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:02:23.265406 17400 consensus_queue.cc:1048] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:02:24.765425 17416 raft_consensus.cc:493] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a06397457dab4d5fa7f14e27026e675c)
I20260812 08:02:24.765614 17416 raft_consensus.cc:515] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 32871 } }
I20260812 08:02:24.766744 17416 leader_election.cc:290] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1d06c98078b5474a96ddc860047b8662 (127.0.186.67:42163), a06397457dab4d5fa7f14e27026e675c (127.0.186.65:32871)
I20260812 08:02:24.767091 17418 raft_consensus.cc:493] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a06397457dab4d5fa7f14e27026e675c)
I20260812 08:02:24.767292 17418 raft_consensus.cc:515] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 32871 } }
I20260812 08:02:24.768421 17418 leader_election.cc:290] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cf9d4d161b86467d95a853b11bb103d9 (127.0.186.66:33619), a06397457dab4d5fa7f14e27026e675c (127.0.186.65:32871)
I20260812 08:02:24.770871 17318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679b4d04e0bd4be9a0707f328981fd47" candidate_uuid: "cf9d4d161b86467d95a853b11bb103d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1d06c98078b5474a96ddc860047b8662" is_pre_election: true
I20260812 08:02:24.771004 17318 raft_consensus.cc:2468] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf9d4d161b86467d95a853b11bb103d9 in term 1.
I20260812 08:02:24.771399 17078 leader_election.cc:304] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d06c98078b5474a96ddc860047b8662, cf9d4d161b86467d95a853b11bb103d9; no voters: 
I20260812 08:02:24.771658 17416 raft_consensus.cc:2804] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 08:02:24.771728 17416 raft_consensus.cc:493] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a06397457dab4d5fa7f14e27026e675c)
I20260812 08:02:24.771770 17416 raft_consensus.cc:3060] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:02:24.771873 17160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679b4d04e0bd4be9a0707f328981fd47" candidate_uuid: "1d06c98078b5474a96ddc860047b8662" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cf9d4d161b86467d95a853b11bb103d9" is_pre_election: true
I20260812 08:02:24.772573 17416 raft_consensus.cc:515] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 32871 } }
I20260812 08:02:24.772823 17160 raft_consensus.cc:2393] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1d06c98078b5474a96ddc860047b8662 in current term 2: Already voted for candidate cf9d4d161b86467d95a853b11bb103d9 in this term.
I20260812 08:02:24.772926 17416 leader_election.cc:290] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [CANDIDATE]: Term 2 election: Requested vote from peers 1d06c98078b5474a96ddc860047b8662 (127.0.186.67:42163), a06397457dab4d5fa7f14e27026e675c (127.0.186.65:32871)
I20260812 08:02:24.773253 17318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679b4d04e0bd4be9a0707f328981fd47" candidate_uuid: "cf9d4d161b86467d95a853b11bb103d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1d06c98078b5474a96ddc860047b8662"
I20260812 08:02:24.773371 17318 raft_consensus.cc:3060] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:02:24.774106 17318 raft_consensus.cc:2468] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf9d4d161b86467d95a853b11bb103d9 in term 2.
I20260812 08:02:24.774402 17078 leader_election.cc:304] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d06c98078b5474a96ddc860047b8662, cf9d4d161b86467d95a853b11bb103d9; no voters: 
I20260812 08:02:24.774595 17416 raft_consensus.cc:2804] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:02:24.774801 17416 raft_consensus.cc:697] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [term 2 LEADER]: Becoming Leader. State: Replica: cf9d4d161b86467d95a853b11bb103d9, State: Running, Role: LEADER
I20260812 08:02:24.775112 17416 consensus_queue.cc:237] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 32871 } }
I20260812 08:02:24.777652 16826 catalog_manager.cc:5719] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 reported cstate change: term changed from 1 to 2, leader changed from a06397457dab4d5fa7f14e27026e675c (127.0.186.65) to cf9d4d161b86467d95a853b11bb103d9 (127.0.186.66). New cstate: current_term: 2 leader_uuid: "cf9d4d161b86467d95a853b11bb103d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 32871 } health_report { overall_health: UNKNOWN } } }
I20260812 08:02:24.792719   745 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
I20260812 08:02:24.793042   745 raft_consensus-itest-base.cc:268] Waiting until we've written at least 4MB...
I20260812 08:02:24.807050 17318 raft_consensus.cc:1275] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662 [term 2 FOLLOWER]: Refusing update from remote peer cf9d4d161b86467d95a853b11bb103d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260812 08:02:24.807650 17416 consensus_queue.cc:1048] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:02:24.844058 17371 maintenance_manager.cc:419] P 1d06c98078b5474a96ddc860047b8662: Scheduling FlushMRSOp(679b4d04e0bd4be9a0707f328981fd47): 1577982 bytes log retention, and flush 2998537 bytes memory
I20260812 08:02:24.870162 17217 maintenance_manager.cc:419] P cf9d4d161b86467d95a853b11bb103d9: Scheduling FlushMRSOp(679b4d04e0bd4be9a0707f328981fd47): 2761438 bytes log retention, and flush 4038921 bytes memory
I20260812 08:02:24.891783 17244 maintenance_manager.cc:643] P 1d06c98078b5474a96ddc860047b8662: FlushMRSOp(679b4d04e0bd4be9a0707f328981fd47) complete. Timing: real 0.047s	user 0.036s	sys 0.000s Metrics: {"bytes_written":135803,"cfile_init":1,"compiler_manager_pool.queue_time_us":242,"drs_written":1,"lbm_read_time_us":70,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1117,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":16,"thread_start_us":250,"threads_started":2}
I20260812 08:02:24.892949 17371 maintenance_manager.cc:419] P 1d06c98078b5474a96ddc860047b8662: Scheduling LogGCOp(679b4d04e0bd4be9a0707f328981fd47): free 1577982 bytes of WAL
I20260812 08:02:24.893255 17244 log_reader.cc:385] T 679b4d04e0bd4be9a0707f328981fd47: removed 2 log segments from log reader
I20260812 08:02:24.893364 17244 log.cc:1079] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/679b4d04e0bd4be9a0707f328981fd47/wal-000000001 (ops 1-8)
I20260812 08:02:24.893486 17244 log.cc:1079] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/679b4d04e0bd4be9a0707f328981fd47/wal-000000002 (ops 9-14)
I20260812 08:02:24.894122 17244 maintenance_manager.cc:643] P 1d06c98078b5474a96ddc860047b8662: LogGCOp(679b4d04e0bd4be9a0707f328981fd47) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 08:02:24.895720 17371 maintenance_manager.cc:419] P 1d06c98078b5474a96ddc860047b8662: Scheduling FlushMRSOp(679b4d04e0bd4be9a0707f328981fd47): 2366751 bytes log retention, and flush 2998537 bytes memory
I20260812 08:02:24.913940   745 raft_consensus-itest-base.cc:274] Waiting for log GC on cf9d4d161b86467d95a853b11bb103d9
I20260812 08:02:24.929303 17244 maintenance_manager.cc:643] P 1d06c98078b5474a96ddc860047b8662: FlushMRSOp(679b4d04e0bd4be9a0707f328981fd47) complete. Timing: real 0.026s	user 0.024s	sys 0.000s Metrics: {"bytes_written":135869,"cfile_init":1,"dirs.queue_time_us":34,"dirs.run_cpu_time_us":171,"dirs.run_wall_time_us":1176,"drs_written":1,"lbm_read_time_us":50,"lbm_reads_lt_1ms":4,"lbm_write_time_us":604,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":18}
I20260812 08:02:24.930392 17371 maintenance_manager.cc:419] P 1d06c98078b5474a96ddc860047b8662: Scheduling LogGCOp(679b4d04e0bd4be9a0707f328981fd47): free 1577836 bytes of WAL
I20260812 08:02:24.930630 17244 log_reader.cc:385] T 679b4d04e0bd4be9a0707f328981fd47: removed 2 log segments from log reader
I20260812 08:02:24.930725 17244 log.cc:1079] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/679b4d04e0bd4be9a0707f328981fd47/wal-000000003 (ops 15-19)
I20260812 08:02:24.930811 17244 log.cc:1079] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/679b4d04e0bd4be9a0707f328981fd47/wal-000000004 (ops 20-26)
I20260812 08:02:24.931391 17244 maintenance_manager.cc:643] P 1d06c98078b5474a96ddc860047b8662: LogGCOp(679b4d04e0bd4be9a0707f328981fd47) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 08:02:24.931741 17371 maintenance_manager.cc:419] P 1d06c98078b5474a96ddc860047b8662: Scheduling UndoDeltaBlockGCOp(679b4d04e0bd4be9a0707f328981fd47): 655 bytes on disk
I20260812 08:02:24.932440 17244 maintenance_manager.cc:643] P 1d06c98078b5474a96ddc860047b8662: UndoDeltaBlockGCOp(679b4d04e0bd4be9a0707f328981fd47) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":114,"lbm_reads_lt_1ms":8}
I20260812 08:02:24.932934 17371 maintenance_manager.cc:419] P 1d06c98078b5474a96ddc860047b8662: Scheduling CompactRowSetsOp(679b4d04e0bd4be9a0707f328981fd47): perf score=0.996821
I20260812 08:02:24.936223 17084 maintenance_manager.cc:643] P cf9d4d161b86467d95a853b11bb103d9: FlushMRSOp(679b4d04e0bd4be9a0707f328981fd47) complete. Timing: real 0.066s	user 0.037s	sys 0.004s Metrics: {"bytes_written":135869,"cfile_init":1,"compiler_manager_pool.queue_time_us":29669,"drs_written":1,"lbm_read_time_us":68,"lbm_reads_lt_1ms":4,"lbm_write_time_us":628,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":24,"thread_start_us":275,"threads_started":2}
I20260812 08:02:24.937250 17217 maintenance_manager.cc:419] P cf9d4d161b86467d95a853b11bb103d9: Scheduling LogGCOp(679b4d04e0bd4be9a0707f328981fd47): free 2761438 bytes of WAL
I20260812 08:02:24.937557 17084 log_reader.cc:385] T 679b4d04e0bd4be9a0707f328981fd47: removed 3 log segments from log reader
I20260812 08:02:24.937665 17084 log.cc:1079] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/wals/679b4d04e0bd4be9a0707f328981fd47/wal-000000001 (ops 1-9)
I20260812 08:02:24.937767 17084 log.cc:1079] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/wals/679b4d04e0bd4be9a0707f328981fd47/wal-000000002 (ops 10-16)
I20260812 08:02:24.937839 17084 log.cc:1079] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/wals/679b4d04e0bd4be9a0707f328981fd47/wal-000000003 (ops 17-23)
I20260812 08:02:24.938768 17084 maintenance_manager.cc:643] P cf9d4d161b86467d95a853b11bb103d9: LogGCOp(679b4d04e0bd4be9a0707f328981fd47) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 08:02:24.939119 17217 maintenance_manager.cc:419] P cf9d4d161b86467d95a853b11bb103d9: Scheduling FlushMRSOp(679b4d04e0bd4be9a0707f328981fd47): 1840912 bytes log retention, and flush 2998537 bytes memory
I20260812 08:02:24.944770   745 raft_consensus-itest-base.cc:279] Log GC complete on cf9d4d161b86467d95a853b11bb103d9
I20260812 08:02:24.961522 17084 maintenance_manager.cc:643] P cf9d4d161b86467d95a853b11bb103d9: FlushMRSOp(679b4d04e0bd4be9a0707f328981fd47) complete. Timing: real 0.022s	user 0.022s	sys 0.000s Metrics: {"bytes_written":135803,"cfile_init":1,"dirs.queue_time_us":66,"dirs.run_cpu_time_us":227,"dirs.run_wall_time_us":1171,"drs_written":1,"lbm_read_time_us":58,"lbm_reads_lt_1ms":4,"lbm_write_time_us":677,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":14}
I20260812 08:02:24.962659 17217 maintenance_manager.cc:419] P cf9d4d161b86467d95a853b11bb103d9: Scheduling CompactRowSetsOp(679b4d04e0bd4be9a0707f328981fd47): perf score=0.996830
I20260812 08:02:24.984779 17244 maintenance_manager.cc:643] P 1d06c98078b5474a96ddc860047b8662: CompactRowSetsOp(679b4d04e0bd4be9a0707f328981fd47) complete. Timing: real 0.052s	user 0.050s	sys 0.000s Metrics: {"bytes_written":136001,"cfile_cache_hit":10,"cfile_cache_hit_bytes":262392,"cfile_cache_miss":26,"cfile_cache_miss_bytes":264250,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":84,"dirs.run_cpu_time_us":160,"dirs.run_wall_time_us":1065,"drs_written":1,"lbm_read_time_us":706,"lbm_reads_lt_1ms":46,"lbm_write_time_us":837,"lbm_writes_lt_1ms":23,"mutex_wait_us":21,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":34,"spinlock_wait_cycles":9088,"thread_start_us":351,"threads_started":5}
I20260812 08:02:24.985555 17371 maintenance_manager.cc:419] P 1d06c98078b5474a96ddc860047b8662: Scheduling UndoDeltaBlockGCOp(679b4d04e0bd4be9a0707f328981fd47): 516 bytes on disk
I20260812 08:02:24.985992 17244 maintenance_manager.cc:643] P 1d06c98078b5474a96ddc860047b8662: UndoDeltaBlockGCOp(679b4d04e0bd4be9a0707f328981fd47) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":55,"lbm_reads_lt_1ms":4}
I20260812 08:02:25.018402 17084 maintenance_manager.cc:643] P cf9d4d161b86467d95a853b11bb103d9: CompactRowSetsOp(679b4d04e0bd4be9a0707f328981fd47) complete. Timing: real 0.056s	user 0.055s	sys 0.000s Metrics: {"bytes_written":136001,"cfile_cache_hit":10,"cfile_cache_hit_bytes":262392,"cfile_cache_miss":26,"cfile_cache_miss_bytes":264336,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":1207,"dirs.run_cpu_time_us":206,"dirs.run_wall_time_us":1131,"drs_written":1,"lbm_read_time_us":806,"lbm_reads_lt_1ms":54,"lbm_write_time_us":648,"lbm_writes_lt_1ms":23,"mutex_wait_us":22,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":38,"spinlock_wait_cycles":896,"thread_start_us":525,"threads_started":5}
I20260812 08:02:25.019018 17217 maintenance_manager.cc:419] P cf9d4d161b86467d95a853b11bb103d9: Scheduling UndoDeltaBlockGCOp(679b4d04e0bd4be9a0707f328981fd47): 561 bytes on disk
I20260812 08:02:25.019511 17084 maintenance_manager.cc:643] P cf9d4d161b86467d95a853b11bb103d9: UndoDeltaBlockGCOp(679b4d04e0bd4be9a0707f328981fd47) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":60,"lbm_reads_lt_1ms":4}
W20260812 08:02:26.295514 17078 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: connection negotiation to 127.0.186.65:32871 for RPC RequestConsensusVote timed out after 1.527s (ON_OUTBOUND_QUEUE)
W20260812 08:02:26.877591 17239 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: connection negotiation to 127.0.186.65:32871 for RPC RequestConsensusVote timed out after 2.108s (ON_OUTBOUND_QUEUE)
I20260812 08:02:26.946333   745 raft_consensus-itest-base.cc:295] Servers converged with original term 2
I20260812 08:02:26.946408   745 raft_consensus-itest-base.cc:300] Resuming a06397457dab4d5fa7f14e27026e675c
I20260812 08:02:26.948624 17160 raft_consensus.cc:1240] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [term 2 LEADER]: Rejecting Update request from peer a06397457dab4d5fa7f14e27026e675c for earlier term 1. Current term is 2. Ops: []
I20260812 08:02:26.950394 17396 consensus_queue.cc:1059] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 40, Time since last communication: 0.000s
I20260812 08:02:26.950580 17396 consensus_queue.cc:1243] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [LEADER]: Peer cf9d4d161b86467d95a853b11bb103d9 log is divergent from this leader: its last log entry 2.40 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 40
I20260812 08:02:26.950645 17396 raft_consensus.cc:3055] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [term 1 LEADER]: Stepping down as leader of term 1
I20260812 08:02:26.950677 17396 raft_consensus.cc:740] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a06397457dab4d5fa7f14e27026e675c, State: Running, Role: LEADER
I20260812 08:02:26.950845 17396 consensus_queue.cc:260] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 32871 } }
I20260812 08:02:26.951143 17396 raft_consensus.cc:3060] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:02:26.954320 17317 raft_consensus.cc:1240] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662 [term 2 FOLLOWER]: Rejecting Update request from peer a06397457dab4d5fa7f14e27026e675c for earlier term 1. Current term is 2. Ops: []
I20260812 08:02:26.954550 17000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679b4d04e0bd4be9a0707f328981fd47" candidate_uuid: "cf9d4d161b86467d95a853b11bb103d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a06397457dab4d5fa7f14e27026e675c" is_pre_election: true
I20260812 08:02:26.954756 16999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679b4d04e0bd4be9a0707f328981fd47" candidate_uuid: "1d06c98078b5474a96ddc860047b8662" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a06397457dab4d5fa7f14e27026e675c" is_pre_election: true
I20260812 08:02:26.954864 16997 raft_consensus.cc:1275] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [term 2 FOLLOWER]: Refusing update from remote peer cf9d4d161b86467d95a853b11bb103d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260812 08:02:26.955138 16999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679b4d04e0bd4be9a0707f328981fd47" candidate_uuid: "cf9d4d161b86467d95a853b11bb103d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a06397457dab4d5fa7f14e27026e675c"
I20260812 08:02:26.955248 17239 leader_election.cc:304] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1d06c98078b5474a96ddc860047b8662; no voters: a06397457dab4d5fa7f14e27026e675c, cf9d4d161b86467d95a853b11bb103d9
I20260812 08:02:26.956079 17487 raft_consensus.cc:2749] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 08:02:26.956259 17479 consensus_queue.cc:1048] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 32871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:02:26.956374 16999 raft_consensus.cc:2468] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf9d4d161b86467d95a853b11bb103d9 in term 2.
I20260812 08:02:26.956632 17479 consensus_queue.cc:753] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [LEADER]: The logs necessary to catch up peer a06397457dab4d5fa7f14e27026e675c have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..33: Segment 1 which contained index 2 has been GCed)
I20260812 08:02:26.956715 17479 consensus_queue.cc:579] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [LEADER]: The logs necessary to catch up peer a06397457dab4d5fa7f14e27026e675c have been garbage collected. The replica will never be able to catch up
I20260812 08:02:26.959563 17160 consensus_queue.cc:237] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 2.40, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } }
I20260812 08:02:26.960757 17317 raft_consensus.cc:1275] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662 [term 2 FOLLOWER]: Refusing update from remote peer cf9d4d161b86467d95a853b11bb103d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 40. Preceding OpId from leader: term: 2 index: 41. (index mismatch)
I20260812 08:02:26.961139 17479 consensus_queue.cc:1048] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 41, Last known committed idx: 40, Time since last communication: 0.000s
I20260812 08:02:26.963053 17488 raft_consensus.cc:2955] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [term 2 LEADER]: Committing config change with OpId 2.41: config changed from index -1 to 41, VOTER a06397457dab4d5fa7f14e27026e675c (127.0.186.65) evicted. New config: { opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } } }
I20260812 08:02:26.963505 17317 raft_consensus.cc:2955] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662 [term 2 FOLLOWER]: Committing config change with OpId 2.41: config changed from index -1 to 41, VOTER a06397457dab4d5fa7f14e27026e675c (127.0.186.65) evicted. New config: { opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } } }
I20260812 08:02:26.964710 16811 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 679b4d04e0bd4be9a0707f328981fd47 with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:02:26.965521 16828 catalog_manager.cc:5719] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 reported cstate change: config changed from index -1 to 41, VOTER a06397457dab4d5fa7f14e27026e675c (127.0.186.65) evicted. New cstate: current_term: 2 leader_uuid: "cf9d4d161b86467d95a853b11bb103d9" committed_config { opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } health_report { overall_health: HEALTHY } } }
I20260812 08:02:26.968883 16979 tablet_service.cc:1558] Processing DeleteTablet for tablet 679b4d04e0bd4be9a0707f328981fd47 with delete_type TABLET_DATA_TOMBSTONED (TS a06397457dab4d5fa7f14e27026e675c not found in new config with opid_index 41) from {username='slave'} at 127.0.0.1:47142
I20260812 08:02:26.969411 17496 tablet_replica.cc:333] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c: stopping tablet replica
I20260812 08:02:26.969720 17496 raft_consensus.cc:2243] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 08:02:26.969928 17496 raft_consensus.cc:2272] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 08:02:26.970916 17496 ts_tablet_manager.cc:1916] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 08:02:26.972661 17496 ts_tablet_manager.cc:1929] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.1
I20260812 08:02:26.972735 17496 log.cc:1199] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c: Deleting WAL directory at /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/679b4d04e0bd4be9a0707f328981fd47
I20260812 08:02:26.973296 16810 catalog_manager.cc:5050] TS a06397457dab4d5fa7f14e27026e675c (127.0.186.65:32871): tablet 679b4d04e0bd4be9a0707f328981fd47 (table TestTable [id=d92e5c0a80554ce3b7efbd68d45aae49]) successfully deleted
I20260812 08:02:26.977303   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 16792
I20260812 08:02:26.988651   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:44369
--webserver_interface=127.0.186.126
--webserver_port=46419
--builtin_ntp_servers=127.0.186.84:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:44369
--raft_prepare_replacement_before_eviction=true
--master_add_server_when_underreplicated=false
--master_add_server_when_underreplicated=true with env {}
I20260812 08:02:26.994874 17371 maintenance_manager.cc:419] P 1d06c98078b5474a96ddc860047b8662: Scheduling LogGCOp(679b4d04e0bd4be9a0707f328981fd47): free 788915 bytes of WAL
I20260812 08:02:26.995141 17244 log_reader.cc:385] T 679b4d04e0bd4be9a0707f328981fd47: removed 1 log segments from log reader
I20260812 08:02:26.995187 17244 log.cc:1079] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/679b4d04e0bd4be9a0707f328981fd47/wal-000000005 (ops 27-32)
I20260812 08:02:26.995472 17244 maintenance_manager.cc:643] P 1d06c98078b5474a96ddc860047b8662: LogGCOp(679b4d04e0bd4be9a0707f328981fd47) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {}
I20260812 08:02:27.027505 17217 maintenance_manager.cc:419] P cf9d4d161b86467d95a853b11bb103d9: Scheduling LogGCOp(679b4d04e0bd4be9a0707f328981fd47): free 1840912 bytes of WAL
I20260812 08:02:27.027851 17084 log_reader.cc:385] T 679b4d04e0bd4be9a0707f328981fd47: removed 2 log segments from log reader
I20260812 08:02:27.027911 17084 log.cc:1079] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/wals/679b4d04e0bd4be9a0707f328981fd47/wal-000000004 (ops 24-30)
I20260812 08:02:27.027956 17084 log.cc:1079] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/wals/679b4d04e0bd4be9a0707f328981fd47/wal-000000005 (ops 31-37)
I20260812 08:02:27.028487 17084 maintenance_manager.cc:643] P cf9d4d161b86467d95a853b11bb103d9: LogGCOp(679b4d04e0bd4be9a0707f328981fd47) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
W20260812 08:02:27.096042 17497 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:27.096282 17497 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:27.096325 17497 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:27.099713 17497 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:02:27.099781 17497 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:27.099799 17497 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:02:27.099814 17497 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:02:27.103703 17497 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:44369
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:44369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=46419
--never_fsync=true
--heap_profile_path=/tmp/kudu.17497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:27.104838 17497 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:27.106060 17497 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:27.111795 17507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:27.111824 17503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:27.111795 17504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:27.112062 17497 server_base.cc:1061] running on GCE node
I20260812 08:02:27.112708 17497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:27.113832 17497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:27.115010 17497 hybrid_clock.cc:648] HybridClock initialized: now 1786521747114984 us; error 43 us; skew 500 ppm
I20260812 08:02:27.117069 17497 webserver.cc:533] Webserver started at http://127.0.186.126:46419/ using document root <none> and password file <none>
I20260812 08:02:27.117776 17497 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:27.117854 17497 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:27.121613 17497 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.004s	sys 0.000s
I20260812 08:02:27.123946 17514 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:27.125128 17497 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:27.125257 17497 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "b163af25a36743799e4f67ab23f0b8cf"
format_stamp: "Formatted at 2026-08-12 08:02:22 on dist-test-slave-49h3"
I20260812 08:02:27.125782 17497 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:27.139460 17497 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:27.140228 17497 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:27.140416 17497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:27.147907 17589 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:44369 every 8 connection(s)
I20260812 08:02:27.147912 17497 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:44369
I20260812 08:02:27.149014 17497 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:02:27.153082 17590 sys_catalog.cc:263] Verifying existing consensus state
I20260812 08:02:27.154670 17590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf: Bootstrap starting.
I20260812 08:02:27.154891   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 17497
I20260812 08:02:27.170383 17590 log.cc:826] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:27.178540 17590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:02:27.178925 17590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf: Bootstrap complete.
I20260812 08:02:27.182482 17590 raft_consensus.cc:359] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b163af25a36743799e4f67ab23f0b8cf" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44369 } }
I20260812 08:02:27.182843 17590 raft_consensus.cc:740] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b163af25a36743799e4f67ab23f0b8cf, State: Initialized, Role: FOLLOWER
I20260812 08:02:27.183301 17590 consensus_queue.cc:260] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b163af25a36743799e4f67ab23f0b8cf" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44369 } }
I20260812 08:02:27.183426 17590 raft_consensus.cc:399] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:27.183490 17590 raft_consensus.cc:493] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:27.183599 17590 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:02:27.185297 17590 raft_consensus.cc:515] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b163af25a36743799e4f67ab23f0b8cf" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44369 } }
I20260812 08:02:27.185683 17590 leader_election.cc:304] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b163af25a36743799e4f67ab23f0b8cf; no voters: 
I20260812 08:02:27.185999 17590 leader_election.cc:290] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [CANDIDATE]: Term 2 election: Requested vote from peers 
I20260812 08:02:27.186154 17595 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:02:27.186395 17595 raft_consensus.cc:697] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [term 2 LEADER]: Becoming Leader. State: Replica: b163af25a36743799e4f67ab23f0b8cf, State: Running, Role: LEADER
I20260812 08:02:27.186702 17595 consensus_queue.cc:237] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b163af25a36743799e4f67ab23f0b8cf" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44369 } }
I20260812 08:02:27.187191 17590 sys_catalog.cc:565] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:02:27.188305 17597 sys_catalog.cc:455] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader b163af25a36743799e4f67ab23f0b8cf. Latest consensus state: current_term: 2 leader_uuid: "b163af25a36743799e4f67ab23f0b8cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b163af25a36743799e4f67ab23f0b8cf" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44369 } } }
I20260812 08:02:27.188503 17597 sys_catalog.cc:458] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:27.188463 17596 sys_catalog.cc:455] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b163af25a36743799e4f67ab23f0b8cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b163af25a36743799e4f67ab23f0b8cf" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44369 } } }
I20260812 08:02:27.188565 17596 sys_catalog.cc:458] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:27.188966 17603 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:02:27.192408 17603 catalog_manager.cc:679] Loaded metadata for table TestTable [id=d92e5c0a80554ce3b7efbd68d45aae49]
I20260812 08:02:27.193600 17603 tablet_loader.cc:96] loaded metadata for tablet 679b4d04e0bd4be9a0707f328981fd47 (table TestTable [id=d92e5c0a80554ce3b7efbd68d45aae49])
I20260812 08:02:27.193753 17603 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:02:27.194404 17603 catalog_manager.cc:1295] Loaded cluster ID: 5fd11bed16b64cc19da7201e7d13fedb
I20260812 08:02:27.194458 17603 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:02:27.196077 17603 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:02:27.196720 17603 catalog_manager.cc:6103] T 00000000000000000000000000000000 P b163af25a36743799e4f67ab23f0b8cf: Loaded TSK: 0
I20260812 08:02:27.197578 17603 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:02:27.958585 17539 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" instance_seqno: 1786521742281509) as {username='slave'} at 127.0.186.65:36479; Asking this server to re-register.
I20260812 08:02:27.959188 17053 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:27.959388 17053 heartbeater.cc:507] Master 127.0.186.126:44369 requested a full tablet report, sending...
I20260812 08:02:27.960590 17539 ts_manager.cc:194] Registered new tserver with Master: a06397457dab4d5fa7f14e27026e675c (127.0.186.65:32871)
I20260812 08:02:27.973272 17539 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1d06c98078b5474a96ddc860047b8662" instance_seqno: 1786521742630837) as {username='slave'} at 127.0.186.67:39439; Asking this server to re-register.
I20260812 08:02:27.973791 17370 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:27.973976 17370 heartbeater.cc:507] Master 127.0.186.126:44369 requested a full tablet report, sending...
I20260812 08:02:27.974591 17539 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" instance_seqno: 1786521742457635) as {username='slave'} at 127.0.186.66:49617; Asking this server to re-register.
I20260812 08:02:27.974869 17538 ts_manager.cc:194] Registered new tserver with Master: 1d06c98078b5474a96ddc860047b8662 (127.0.186.67:42163)
I20260812 08:02:27.975010 17216 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:27.975167 17216 heartbeater.cc:507] Master 127.0.186.126:44369 requested a full tablet report, sending...
I20260812 08:02:27.975975 17538 ts_manager.cc:194] Registered new tserver with Master: cf9d4d161b86467d95a853b11bb103d9 (127.0.186.66:33619)
I20260812 08:02:27.981925 17160 consensus_queue.cc:237] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 41, Committed index: 41, Last appended: 2.41, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 32871 } attrs { promote: true } }
I20260812 08:02:27.983434 17317 raft_consensus.cc:1275] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662 [term 2 FOLLOWER]: Refusing update from remote peer cf9d4d161b86467d95a853b11bb103d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 41. Preceding OpId from leader: term: 2 index: 42. (index mismatch)
W20260812 08:02:27.983855 17078 consensus_peers.cc:597] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 -> Peer a06397457dab4d5fa7f14e27026e675c (127.0.186.65:32871): Couldn't send request to peer a06397457dab4d5fa7f14e27026e675c. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:02:27.984071 17626 consensus_queue.cc:1048] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 42, Last known committed idx: 41, Time since last communication: 0.000s
I20260812 08:02:27.985929 17615 raft_consensus.cc:2955] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [term 2 LEADER]: Committing config change with OpId 2.42: config changed from index 41 to 42, NON_VOTER a06397457dab4d5fa7f14e27026e675c (127.0.186.65) added. New config: { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 32871 } attrs { promote: true } } }
I20260812 08:02:27.986409 17317 raft_consensus.cc:2955] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662 [term 2 FOLLOWER]: Committing config change with OpId 2.42: config changed from index 41 to 42, NON_VOTER a06397457dab4d5fa7f14e27026e675c (127.0.186.65) added. New config: { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 32871 } attrs { promote: true } } }
I20260812 08:02:27.988564 17517 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 679b4d04e0bd4be9a0707f328981fd47 with cas_config_opid_index 41: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:02:27.988965 17538 catalog_manager.cc:5719] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 reported cstate change: config changed from index 41 to 42, NON_VOTER a06397457dab4d5fa7f14e27026e675c (127.0.186.65) added. New cstate: current_term: 2 leader_uuid: "cf9d4d161b86467d95a853b11bb103d9" committed_config { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 32871 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260812 08:02:28.415534 17635 ts_tablet_manager.cc:933] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c: Initiating tablet copy from peer cf9d4d161b86467d95a853b11bb103d9 (127.0.186.66:33619)
I20260812 08:02:28.416141 17635 tablet_copy_client.cc:323] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.186.66:33619
I20260812 08:02:28.416934 17184 tablet_copy_service.cc:140] P cf9d4d161b86467d95a853b11bb103d9: Received BeginTabletCopySession request for tablet 679b4d04e0bd4be9a0707f328981fd47 from peer a06397457dab4d5fa7f14e27026e675c ({username='slave'} at 127.0.186.65:36635)
I20260812 08:02:28.417088 17184 tablet_copy_service.cc:161] P cf9d4d161b86467d95a853b11bb103d9: Beginning new tablet copy session on tablet 679b4d04e0bd4be9a0707f328981fd47 from peer a06397457dab4d5fa7f14e27026e675c at {username='slave'} at 127.0.186.65:36635: session id = a06397457dab4d5fa7f14e27026e675c-679b4d04e0bd4be9a0707f328981fd47
I20260812 08:02:28.418602 17184 tablet_copy_source_session.cc:215] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9: Tablet Copy: opened 5 blocks and 1 log segments
I20260812 08:02:28.420244 17635 ts_tablet_manager.cc:1916] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260812 08:02:28.422266 17635 ts_tablet_manager.cc:1929] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.1
I20260812 08:02:28.422446 17635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 679b4d04e0bd4be9a0707f328981fd47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:28.423460 17635 tablet_copy_client.cc:806] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c: tablet copy: Starting download of 5 data blocks...
I20260812 08:02:28.430759 17635 tablet_copy_client.cc:670] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c: tablet copy: Starting download of 1 WAL segments...
I20260812 08:02:28.433602 17635 tablet_copy_client.cc:538] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 08:02:28.435312 17635 tablet_bootstrap.cc:492] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c: Bootstrap starting.
I20260812 08:02:28.444805 17635 tablet_bootstrap.cc:492] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=3} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=1. Pending: 0 replicates
I20260812 08:02:28.445050 17635 tablet_bootstrap.cc:492] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c: Bootstrap complete.
I20260812 08:02:28.445277 17635 ts_tablet_manager.cc:1403] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c: Time spent bootstrapping tablet: real 0.010s	user 0.008s	sys 0.000s
I20260812 08:02:28.445789 17635 raft_consensus.cc:359] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 32871 } attrs { promote: true } }
I20260812 08:02:28.445894 17635 raft_consensus.cc:740] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: a06397457dab4d5fa7f14e27026e675c, State: Initialized, Role: LEARNER
I20260812 08:02:28.445995 17635 consensus_queue.cc:260] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 42, Last appended: 2.42, Last appended by leader: 42, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: NON_VOTER last_known_addr { host: "127.0.186.65" port: 32871 } attrs { promote: true } }
I20260812 08:02:28.446352 17635 ts_tablet_manager.cc:1434] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c: Time spent starting tablet: real 0.001s	user 0.005s	sys 0.000s
I20260812 08:02:28.447002 17184 tablet_copy_service.cc:342] P cf9d4d161b86467d95a853b11bb103d9: Request end of tablet copy session a06397457dab4d5fa7f14e27026e675c-679b4d04e0bd4be9a0707f328981fd47 received from {username='slave'} at 127.0.186.65:36635
I20260812 08:02:28.447111 17184 tablet_copy_service.cc:434] P cf9d4d161b86467d95a853b11bb103d9: ending tablet copy session a06397457dab4d5fa7f14e27026e675c-679b4d04e0bd4be9a0707f328981fd47 on tablet 679b4d04e0bd4be9a0707f328981fd47 with peer a06397457dab4d5fa7f14e27026e675c
I20260812 08:02:28.450031 17054 maintenance_manager.cc:419] P a06397457dab4d5fa7f14e27026e675c: Scheduling UndoDeltaBlockGCOp(679b4d04e0bd4be9a0707f328981fd47): 561 bytes on disk
I20260812 08:02:28.450783 16927 maintenance_manager.cc:643] P a06397457dab4d5fa7f14e27026e675c: UndoDeltaBlockGCOp(679b4d04e0bd4be9a0707f328981fd47) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":47,"lbm_reads_lt_1ms":4}
I20260812 08:02:28.900836 16999 raft_consensus.cc:1217] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [term 2 LEARNER]: Deduplicated request from leader. Original: 2.41->[2.42-2.42]   Dedup: 2.42->[]
I20260812 08:02:29.503702 17642 raft_consensus.cc:1064] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9: attempting to promote NON_VOTER a06397457dab4d5fa7f14e27026e675c to VOTER
I20260812 08:02:29.504207 17642 consensus_queue.cc:237] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 42, Committed index: 42, Last appended: 2.42, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 32871 } attrs { promote: false } }
I20260812 08:02:29.505627 16999 raft_consensus.cc:1275] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [term 2 LEARNER]: Refusing update from remote peer cf9d4d161b86467d95a853b11bb103d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 42. Preceding OpId from leader: term: 2 index: 43. (index mismatch)
I20260812 08:02:29.505756 17317 raft_consensus.cc:1275] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662 [term 2 FOLLOWER]: Refusing update from remote peer cf9d4d161b86467d95a853b11bb103d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 42. Preceding OpId from leader: term: 2 index: 43. (index mismatch)
I20260812 08:02:29.506095 17642 consensus_queue.cc:1048] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 32871 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 43, Last known committed idx: 42, Time since last communication: 0.000s
I20260812 08:02:29.506313 17643 consensus_queue.cc:1048] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 43, Last known committed idx: 42, Time since last communication: 0.000s
I20260812 08:02:29.508350 17645 raft_consensus.cc:2955] T 679b4d04e0bd4be9a0707f328981fd47 P cf9d4d161b86467d95a853b11bb103d9 [term 2 LEADER]: Committing config change with OpId 2.43: config changed from index 42 to 43, a06397457dab4d5fa7f14e27026e675c (127.0.186.65) changed from NON_VOTER to VOTER. New config: { opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 32871 } attrs { promote: false } } }
I20260812 08:02:29.508767 17317 raft_consensus.cc:2955] T 679b4d04e0bd4be9a0707f328981fd47 P 1d06c98078b5474a96ddc860047b8662 [term 2 FOLLOWER]: Committing config change with OpId 2.43: config changed from index 42 to 43, a06397457dab4d5fa7f14e27026e675c (127.0.186.65) changed from NON_VOTER to VOTER. New config: { opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 32871 } attrs { promote: false } } }
I20260812 08:02:29.508863 16999 raft_consensus.cc:2955] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c [term 2 FOLLOWER]: Committing config change with OpId 2.43: config changed from index 42 to 43, a06397457dab4d5fa7f14e27026e675c (127.0.186.65) changed from NON_VOTER to VOTER. New config: { opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 32871 } attrs { promote: false } } }
I20260812 08:02:29.512144 17539 catalog_manager.cc:5719] T 679b4d04e0bd4be9a0707f328981fd47 P a06397457dab4d5fa7f14e27026e675c reported cstate change: config changed from index 42 to 43, a06397457dab4d5fa7f14e27026e675c (127.0.186.65) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "cf9d4d161b86467d95a853b11bb103d9" committed_config { opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "1d06c98078b5474a96ddc860047b8662" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 42163 } } peers { permanent_uuid: "cf9d4d161b86467d95a853b11bb103d9" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33619 } } peers { permanent_uuid: "a06397457dab4d5fa7f14e27026e675c" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 32871 } attrs { promote: false } } }
I20260812 08:02:29.587443 17295 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:02:29.587617 17136 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260812 08:02:29.588433 16979 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |       Address       | Status
----------------------------------+---------------------+---------
 b163af25a36743799e4f67ab23f0b8cf | 127.0.186.126:44369 | HEALTHY

Unusual flags for Master:
                   Flag                   |                                                                                               Value                                                                                                |         Tags          |         Master
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
 ipki_ca_key_size                         | 768                                                                                                                                                                                                | experimental          | all 1 server(s) checked
 ipki_server_key_size                     | 768                                                                                                                                                                                                | experimental          | all 1 server(s) checked
 master_add_server_when_underreplicated   | true                                                                                                                                                                                               | hidden                | all 1 server(s) checked
 never_fsync                              | true                                                                                                                                                                                               | unsafe,advanced       | all 1 server(s) checked
 openssl_security_level_override          | 0                                                                                                                                                                                                  | unsafe,hidden         | all 1 server(s) checked
 raft_prepare_replacement_before_eviction | true                                                                                                                                                                                               | experimental,advanced | all 1 server(s) checked
 rpc_reuseport                            | true                                                                                                                                                                                               | experimental          | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports         | true                                                                                                                                                                                               | unsafe                | all 1 server(s) checked
 server_dump_info_format                  | pb                                                                                                                                                                                                 | hidden                | all 1 server(s) checked
 server_dump_info_path                    | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden                | all 1 server(s) checked
 tsk_num_rsa_bits                         | 512                                                                                                                                                                                                | experimental          | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |       Value        |         Master
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.0.186.84:43723 | all 1 server(s) checked
 time_source         | builtin            | all 1 server(s) checked

Tablet Server Summary
               UUID               |      Address       | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
 1d06c98078b5474a96ddc860047b8662 | 127.0.186.67:42163 | HEALTHY | <none>   |       0        |       0
 a06397457dab4d5fa7f14e27026e675c | 127.0.186.65:32871 | HEALTHY | <none>   |       0        |       0
 cf9d4d161b86467d95a853b11bb103d9 | 127.0.186.66:33619 | HEALTHY | <none>   |       1        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
                   Flag                   |                                                                                             Value                                                                                              |             Tags              |      Tablet Server
------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------+-------------------------
 ipki_server_key_size                     | 768                                                                                                                                                                                            | experimental                  | all 3 server(s) checked
 local_ip_for_outbound_sockets            | 127.0.186.65                                                                                                                                                                                   | experimental                  | 127.0.186.65:32871
 local_ip_for_outbound_sockets            | 127.0.186.66                                                                                                                                                                                   | experimental                  | 127.0.186.66:33619
 local_ip_for_outbound_sockets            | 127.0.186.67                                                                                                                                                                                   | experimental                  | 127.0.186.67:42163
 log_compression_codec                    | no_compression                                                                                                                                                                                 | experimental                  | all 3 server(s) checked
 log_max_segments_to_retain               | 3                                                                                                                                                                                              | experimental,runtime,advanced | all 3 server(s) checked
 log_target_replay_size_mb                | 1                                                                                                                                                                                              | experimental                  | all 3 server(s) checked
 maintenance_manager_polling_interval_ms  | 100                                                                                                                                                                                            | hidden                        | all 3 server(s) checked
 max_cell_size_bytes                      | 1000000                                                                                                                                                                                        | unsafe                        | all 3 server(s) checked
 never_fsync                              | true                                                                                                                                                                                           | unsafe,advanced               | all 3 server(s) checked
 openssl_security_level_override          | 0                                                                                                                                                                                              | unsafe,hidden                 | all 3 server(s) checked
 raft_prepare_replacement_before_eviction | true                                                                                                                                                                                           | experimental,advanced         | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports         | true                                                                                                                                                                                           | unsafe                        | all 3 server(s) checked
 server_dump_info_format                  | pb                                                                                                                                                                                             | hidden                        | all 3 server(s) checked
 server_dump_info_path                    | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden                        | 127.0.186.65:32871
 server_dump_info_path                    | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden                        | 127.0.186.66:33619
 server_dump_info_path                    | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_0.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden                        | 127.0.186.67:42163

Flags of checked categories for Tablet Server:
        Flag         |       Value        |      Tablet Server
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.0.186.84:43723 | all 3 server(s) checked
 time_source         | builtin            | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
   Name    | RF | Status  | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-----------+----+---------+---------------+---------+------------+------------------+-------------
 TestTable | 3  | HEALTHY | 1             | 1       | 0          | 0                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

OK
I20260812 08:02:29.611193   745 log_verifier.cc:126] Checking tablet 679b4d04e0bd4be9a0707f328981fd47
I20260812 08:02:29.617764   745 log_verifier.cc:177] Verified matching terms for 13 ops in tablet 679b4d04e0bd4be9a0707f328981fd47
I20260812 08:02:29.618219   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 16902
I20260812 08:02:29.628341   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 17056
I20260812 08:02:29.639699   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 17219
I20260812 08:02:29.651147   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 17497
2026-08-12T08:02:29Z chronyd exiting
[       OK ] ReplicaBehindWalGcThresholdITest.ReplicaReplacement/0 (7795 ms)
[ RUN      ] ReplicaBehindWalGcThresholdITest.ReplicaReplacement/1
I20260812 08:02:29.676903   745 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:02:29.676950   745 ts_itest-base.cc:116] --------------
I20260812 08:02:29.676961   745 ts_itest-base.cc:117] 3 tablet servers
I20260812 08:02:29.676970   745 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:02:29.676980   745 ts_itest-base.cc:119] --------------
2026-08-12T08:02:29Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:02:29Z Disabled control of system clock
I20260812 08:02:29.686314   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:44527
--webserver_interface=127.0.186.126
--webserver_port=0
--builtin_ntp_servers=127.0.186.84:46879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:44527
--raft_prepare_replacement_before_eviction=true
--master_add_server_when_underreplicated=false with env {}
W20260812 08:02:29.792516 17696 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:29.792764 17696 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:29.792804 17696 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:29.796202 17696 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:02:29.796263 17696 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:29.796289 17696 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:02:29.796306 17696 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:02:29.800148 17696 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:46879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--master_add_server_when_underreplicated=false
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:44527
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:44527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.17696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:29.801218 17696 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:29.802377 17696 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:29.807705 17701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:29.807776 17702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:29.807705 17704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:29.807961 17696 server_base.cc:1061] running on GCE node
I20260812 08:02:29.808446 17696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:29.809281 17696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:29.810427 17696 hybrid_clock.cc:648] HybridClock initialized: now 1786521749810409 us; error 34 us; skew 500 ppm
I20260812 08:02:29.812055 17696 webserver.cc:533] Webserver started at http://127.0.186.126:33695/ using document root <none> and password file <none>
I20260812 08:02:29.812568 17696 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:29.812618 17696 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:29.812780 17696 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:29.814285 17696 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "76fa69a4f44344528ad0eb506177898b"
format_stamp: "Formatted at 2026-08-12 08:02:29 on dist-test-slave-49h3"
I20260812 08:02:29.814721 17696 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "76fa69a4f44344528ad0eb506177898b"
format_stamp: "Formatted at 2026-08-12 08:02:29 on dist-test-slave-49h3"
I20260812 08:02:29.817916 17696 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:02:29.820022 17712 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:29.821065 17696 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:29.821179 17696 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "76fa69a4f44344528ad0eb506177898b"
format_stamp: "Formatted at 2026-08-12 08:02:29 on dist-test-slave-49h3"
I20260812 08:02:29.821275 17696 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:29.837373 17696 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:29.838155 17696 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:29.838339 17696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:29.845963 17780 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:44527 every 8 connection(s)
I20260812 08:02:29.845959 17696 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:44527
I20260812 08:02:29.847075 17696 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:02:29.849977 17781 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:29.851503   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 17696
I20260812 08:02:29.851603   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:02:29.855033 17781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b: Bootstrap starting.
I20260812 08:02:29.857524 17781 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:29.858335 17781 log.cc:826] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:29.860110 17781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b: No bootstrap required, opened a new log
I20260812 08:02:29.862664 17781 raft_consensus.cc:359] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76fa69a4f44344528ad0eb506177898b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44527 } }
I20260812 08:02:29.862835 17781 raft_consensus.cc:385] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:29.862883 17781 raft_consensus.cc:740] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76fa69a4f44344528ad0eb506177898b, State: Initialized, Role: FOLLOWER
I20260812 08:02:29.863248 17781 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76fa69a4f44344528ad0eb506177898b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44527 } }
I20260812 08:02:29.863366 17781 raft_consensus.cc:399] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:29.863409 17781 raft_consensus.cc:493] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:29.863484 17781 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:29.864183 17781 raft_consensus.cc:515] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76fa69a4f44344528ad0eb506177898b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44527 } }
I20260812 08:02:29.864467 17781 leader_election.cc:304] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76fa69a4f44344528ad0eb506177898b; no voters: 
I20260812 08:02:29.864745 17781 leader_election.cc:290] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:02:29.864847 17788 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:29.865054 17788 raft_consensus.cc:697] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [term 1 LEADER]: Becoming Leader. State: Replica: 76fa69a4f44344528ad0eb506177898b, State: Running, Role: LEADER
I20260812 08:02:29.865398 17788 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76fa69a4f44344528ad0eb506177898b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44527 } }
I20260812 08:02:29.865880 17781 sys_catalog.cc:565] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:02:29.867027 17789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76fa69a4f44344528ad0eb506177898b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76fa69a4f44344528ad0eb506177898b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44527 } } }
I20260812 08:02:29.867024 17790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76fa69a4f44344528ad0eb506177898b. Latest consensus state: current_term: 1 leader_uuid: "76fa69a4f44344528ad0eb506177898b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76fa69a4f44344528ad0eb506177898b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44527 } } }
I20260812 08:02:29.867177 17789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:29.867225 17790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:29.868078 17798 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:02:29.871085 17798 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:02:29.877079 17798 catalog_manager.cc:1383] Generated new cluster ID: 8680ff2e259b4c0195556ed3cadc29d4
I20260812 08:02:29.877239 17798 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:02:29.889155 17798 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:02:29.890401 17798 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:02:29.915843 17798 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b: Generated new TSK 0
I20260812 08:02:29.916594 17798 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:02:29.933053   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:0
--local_ip_for_outbound_sockets=127.0.186.65
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:44527
--builtin_ntp_servers=127.0.186.84:46879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20260812 08:02:30.063156 17808 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:02:30.063421 17808 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:30.063442 17808 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:30.063495 17808 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:30.066954 17808 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:02:30.067014 17808 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:02:30.067051 17808 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:30.067143 17808 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:02:30.067176 17808 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:02:30.071105 17808 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:46879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:44527
--never_fsync=true
--heap_profile_path=/tmp/kudu.17808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:30.072213 17808 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:30.073367 17808 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:30.079741 17814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:30.079767 17813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:30.079777 17816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:30.080057 17808 server_base.cc:1061] running on GCE node
I20260812 08:02:30.080374 17808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:30.080916 17808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:30.082062 17808 hybrid_clock.cc:648] HybridClock initialized: now 1786521750082039 us; error 33 us; skew 500 ppm
I20260812 08:02:30.083636 17808 webserver.cc:533] Webserver started at http://127.0.186.65:44627/ using document root <none> and password file <none>
I20260812 08:02:30.084165 17808 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:30.084220 17808 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:30.084406 17808 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:30.086016 17808 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "72cb331ade34427dbf3147f1f60f7c67"
format_stamp: "Formatted at 2026-08-12 08:02:30 on dist-test-slave-49h3"
I20260812 08:02:30.086483 17808 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "72cb331ade34427dbf3147f1f60f7c67"
format_stamp: "Formatted at 2026-08-12 08:02:30 on dist-test-slave-49h3"
I20260812 08:02:30.089594 17808 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:02:30.091610 17822 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:30.092576 17808 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:30.092672 17808 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "72cb331ade34427dbf3147f1f60f7c67"
format_stamp: "Formatted at 2026-08-12 08:02:30 on dist-test-slave-49h3"
I20260812 08:02:30.092759 17808 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:30.117556 17808 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:30.118254 17808 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:30.118423 17808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:30.119029 17808 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:30.120105 17808 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:30.120150 17808 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:30.120195 17808 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:30.120223 17808 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:30.129156 17808 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:36365
I20260812 08:02:30.129190 17957 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:36365 every 8 connection(s)
I20260812 08:02:30.130151 17808 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:02:30.132663   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 17808
I20260812 08:02:30.132781   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:02:30.135288   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.66:0
--local_ip_for_outbound_sockets=127.0.186.66
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:44527
--builtin_ntp_servers=127.0.186.84:46879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20260812 08:02:30.139295 17958 heartbeater.cc:344] Connected to a master server at 127.0.186.126:44527
I20260812 08:02:30.139549 17958 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:30.140101 17958 heartbeater.cc:507] Master 127.0.186.126:44527 requested a full tablet report, sending...
I20260812 08:02:30.141669 17737 ts_manager.cc:194] Registered new tserver with Master: 72cb331ade34427dbf3147f1f60f7c67 (127.0.186.65:36365)
I20260812 08:02:30.142820 17737 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.65:42359
W20260812 08:02:30.240658 17962 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:02:30.240888 17962 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:30.240929 17962 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:30.240988 17962 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:30.244168 17962 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:02:30.244217 17962 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:02:30.244259 17962 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:30.244340 17962 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:02:30.244374 17962 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.66
I20260812 08:02:30.248114 17962 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:46879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:44527
--never_fsync=true
--heap_profile_path=/tmp/kudu.17962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:30.249167 17962 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:30.250290 17962 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:30.256345 17972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:30.256448 17962 server_base.cc:1061] running on GCE node
W20260812 08:02:30.256596 17969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:30.256352 17970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:30.257006 17962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:30.257560 17962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:30.258708 17962 hybrid_clock.cc:648] HybridClock initialized: now 1786521750258698 us; error 26 us; skew 500 ppm
I20260812 08:02:30.260342 17962 webserver.cc:533] Webserver started at http://127.0.186.66:40047/ using document root <none> and password file <none>
I20260812 08:02:30.260885 17962 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:30.260938 17962 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:30.261128 17962 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:30.262665 17962 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "4b9385e4bb9048af824071b2309c83ba"
format_stamp: "Formatted at 2026-08-12 08:02:30 on dist-test-slave-49h3"
I20260812 08:02:30.263109 17962 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "4b9385e4bb9048af824071b2309c83ba"
format_stamp: "Formatted at 2026-08-12 08:02:30 on dist-test-slave-49h3"
I20260812 08:02:30.266312 17962 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.002s
I20260812 08:02:30.268498 17979 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:30.269474 17962 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:02:30.269587 17962 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "4b9385e4bb9048af824071b2309c83ba"
format_stamp: "Formatted at 2026-08-12 08:02:30 on dist-test-slave-49h3"
I20260812 08:02:30.269680 17962 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:30.278999 17962 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:30.279640 17962 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:30.279798 17962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:30.280371 17962 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:30.281420 17962 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:30.281498 17962 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:30.281562 17962 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:30.281587 17962 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:30.290078 17962 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.66:37013
I20260812 08:02:30.290104 18116 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.66:37013 every 8 connection(s)
I20260812 08:02:30.290997 17962 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:02:30.292111   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 17962
I20260812 08:02:30.292212   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:02:30.294813   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.67:0
--local_ip_for_outbound_sockets=127.0.186.67
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:44527
--builtin_ntp_servers=127.0.186.84:46879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20260812 08:02:30.299909 18117 heartbeater.cc:344] Connected to a master server at 127.0.186.126:44527
I20260812 08:02:30.300158 18117 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:30.300665 18117 heartbeater.cc:507] Master 127.0.186.126:44527 requested a full tablet report, sending...
I20260812 08:02:30.301764 17737 ts_manager.cc:194] Registered new tserver with Master: 4b9385e4bb9048af824071b2309c83ba (127.0.186.66:37013)
I20260812 08:02:30.302311 17737 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.66:48703
W20260812 08:02:30.399825 18120 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:02:30.400058 18120 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:30.400107 18120 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:30.400169 18120 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:30.403407 18120 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:02:30.403462 18120 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:02:30.403506 18120 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:30.403586 18120 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:02:30.403621 18120 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.67
I20260812 08:02:30.407462 18120 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:46879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:44527
--never_fsync=true
--heap_profile_path=/tmp/kudu.18120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:30.408489 18120 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:30.409612 18120 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:30.415688 18125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:30.415692 18126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:30.415814 18120 server_base.cc:1061] running on GCE node
W20260812 08:02:30.415696 18128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:30.416374 18120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:30.416879 18120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:30.418028 18120 hybrid_clock.cc:648] HybridClock initialized: now 1786521750418008 us; error 29 us; skew 500 ppm
I20260812 08:02:30.419572 18120 webserver.cc:533] Webserver started at http://127.0.186.67:43375/ using document root <none> and password file <none>
I20260812 08:02:30.420097 18120 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:30.420152 18120 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:30.420351 18120 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:30.421923 18120 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110"
format_stamp: "Formatted at 2026-08-12 08:02:30 on dist-test-slave-49h3"
I20260812 08:02:30.422389 18120 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110"
format_stamp: "Formatted at 2026-08-12 08:02:30 on dist-test-slave-49h3"
I20260812 08:02:30.425536 18120 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:02:30.427634 18136 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:30.428548 18120 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:30.428651 18120 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110"
format_stamp: "Formatted at 2026-08-12 08:02:30 on dist-test-slave-49h3"
I20260812 08:02:30.428727 18120 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:30.446372 18120 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:30.446956 18120 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:30.447105 18120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:30.447609 18120 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:30.448599 18120 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:30.448647 18120 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:30.448685 18120 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:30.448705 18120 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:30.457176 18120 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.67:44489
I20260812 08:02:30.457211 18274 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.67:44489 every 8 connection(s)
I20260812 08:02:30.458101 18120 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:02:30.461325   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 18120
I20260812 08:02:30.461412   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:02:30.466295 18275 heartbeater.cc:344] Connected to a master server at 127.0.186.126:44527
I20260812 08:02:30.466511 18275 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:30.466996 18275 heartbeater.cc:507] Master 127.0.186.126:44527 requested a full tablet report, sending...
I20260812 08:02:30.467969 17737 ts_manager.cc:194] Registered new tserver with Master: 99e3ab4cbdbc49ce8ad92622b6e0e110 (127.0.186.67:44489)
I20260812 08:02:30.468539 17737 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.67:57225
I20260812 08:02:30.475207   745 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:02:30.483992 17737 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38952:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
W20260812 08:02:30.486474 17737 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 08:02:30.502727 17880 tablet_service.cc:1511] Processing CreateTablet for tablet df4dbd05f89d44fc8799d314a5dc6f27 (DEFAULT_TABLE table=TestTable [id=2f64cab8f6b84c97aeaefdbff7bf2c32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:30.503641 17880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df4dbd05f89d44fc8799d314a5dc6f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:30.503561 18034 tablet_service.cc:1511] Processing CreateTablet for tablet df4dbd05f89d44fc8799d314a5dc6f27 (DEFAULT_TABLE table=TestTable [id=2f64cab8f6b84c97aeaefdbff7bf2c32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:30.504355 18034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df4dbd05f89d44fc8799d314a5dc6f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:30.508531 18294 tablet_bootstrap.cc:492] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67: Bootstrap starting.
I20260812 08:02:30.508740 18295 tablet_bootstrap.cc:492] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba: Bootstrap starting.
I20260812 08:02:30.510739 18294 tablet_bootstrap.cc:654] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:30.510593 18190 tablet_service.cc:1511] Processing CreateTablet for tablet df4dbd05f89d44fc8799d314a5dc6f27 (DEFAULT_TABLE table=TestTable [id=2f64cab8f6b84c97aeaefdbff7bf2c32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:30.511425 18295 tablet_bootstrap.cc:654] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:30.511466 18294 log.cc:826] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:30.511529 18190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df4dbd05f89d44fc8799d314a5dc6f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:30.512187 18295 log.cc:826] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:30.512995 18294 tablet_bootstrap.cc:492] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67: No bootstrap required, opened a new log
I20260812 08:02:30.513173 18294 ts_tablet_manager.cc:1403] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.000s
I20260812 08:02:30.513906 18295 tablet_bootstrap.cc:492] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba: No bootstrap required, opened a new log
I20260812 08:02:30.514101 18295 ts_tablet_manager.cc:1403] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:02:30.515547 18294 raft_consensus.cc:359] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 37013 } }
I20260812 08:02:30.515738 18294 raft_consensus.cc:385] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:30.515830 18294 raft_consensus.cc:740] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72cb331ade34427dbf3147f1f60f7c67, State: Initialized, Role: FOLLOWER
I20260812 08:02:30.516383 18294 consensus_queue.cc:260] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 37013 } }
I20260812 08:02:30.517032 18295 raft_consensus.cc:359] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 37013 } }
I20260812 08:02:30.517233 18299 tablet_bootstrap.cc:492] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110: Bootstrap starting.
I20260812 08:02:30.517302 18295 raft_consensus.cc:385] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:30.517354 18294 ts_tablet_manager.cc:1434] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:02:30.517436 18295 raft_consensus.cc:740] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b9385e4bb9048af824071b2309c83ba, State: Initialized, Role: FOLLOWER
I20260812 08:02:30.517959 17958 heartbeater.cc:499] Master 127.0.186.126:44527 was elected leader, sending a full tablet report...
I20260812 08:02:30.517927 18295 consensus_queue.cc:260] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 37013 } }
I20260812 08:02:30.518522 18117 heartbeater.cc:499] Master 127.0.186.126:44527 was elected leader, sending a full tablet report...
I20260812 08:02:30.519495 18299 tablet_bootstrap.cc:654] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:30.519632 18295 ts_tablet_manager.cc:1434] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.003s
I20260812 08:02:30.520351 18299 log.cc:826] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:30.522070 18299 tablet_bootstrap.cc:492] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110: No bootstrap required, opened a new log
I20260812 08:02:30.522261 18299 ts_tablet_manager.cc:1403] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:02:30.524816 18299 raft_consensus.cc:359] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 37013 } }
I20260812 08:02:30.524976 18299 raft_consensus.cc:385] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:30.525009 18299 raft_consensus.cc:740] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99e3ab4cbdbc49ce8ad92622b6e0e110, State: Initialized, Role: FOLLOWER
I20260812 08:02:30.525393 18299 consensus_queue.cc:260] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 37013 } }
I20260812 08:02:30.526053 18275 heartbeater.cc:499] Master 127.0.186.126:44527 was elected leader, sending a full tablet report...
I20260812 08:02:30.526240 18299 ts_tablet_manager.cc:1434] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110: Time spent starting tablet: real 0.004s	user 0.003s	sys 0.000s
W20260812 08:02:30.531662 17959 tablet.cc:2406] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:02:30.558933 18276 tablet.cc:2406] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:02:30.592203 18118 tablet.cc:2406] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:02:30.604183 18300 raft_consensus.cc:493] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:02:30.604408 18300 raft_consensus.cc:515] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 37013 } }
I20260812 08:02:30.605664 18300 leader_election.cc:290] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99e3ab4cbdbc49ce8ad92622b6e0e110 (127.0.186.67:44489), 72cb331ade34427dbf3147f1f60f7c67 (127.0.186.65:36365)
I20260812 08:02:30.610452 17907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df4dbd05f89d44fc8799d314a5dc6f27" candidate_uuid: "4b9385e4bb9048af824071b2309c83ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72cb331ade34427dbf3147f1f60f7c67" is_pre_election: true
I20260812 08:02:30.610733 18220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df4dbd05f89d44fc8799d314a5dc6f27" candidate_uuid: "4b9385e4bb9048af824071b2309c83ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" is_pre_election: true
I20260812 08:02:30.610806 17907 raft_consensus.cc:2468] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b9385e4bb9048af824071b2309c83ba in term 0.
I20260812 08:02:30.611008 18220 raft_consensus.cc:2468] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b9385e4bb9048af824071b2309c83ba in term 0.
I20260812 08:02:30.611306 17981 leader_election.cc:304] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b9385e4bb9048af824071b2309c83ba, 72cb331ade34427dbf3147f1f60f7c67; no voters: 
I20260812 08:02:30.611555 18300 raft_consensus.cc:2804] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:02:30.611626 18300 raft_consensus.cc:493] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:02:30.611657 18300 raft_consensus.cc:3060] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:30.612520 18300 raft_consensus.cc:515] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 37013 } }
I20260812 08:02:30.612897 18300 leader_election.cc:290] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [CANDIDATE]: Term 1 election: Requested vote from peers 99e3ab4cbdbc49ce8ad92622b6e0e110 (127.0.186.67:44489), 72cb331ade34427dbf3147f1f60f7c67 (127.0.186.65:36365)
I20260812 08:02:30.613289 18220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df4dbd05f89d44fc8799d314a5dc6f27" candidate_uuid: "4b9385e4bb9048af824071b2309c83ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110"
I20260812 08:02:30.613310 17907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df4dbd05f89d44fc8799d314a5dc6f27" candidate_uuid: "4b9385e4bb9048af824071b2309c83ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72cb331ade34427dbf3147f1f60f7c67"
I20260812 08:02:30.613425 18220 raft_consensus.cc:3060] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:30.613425 17907 raft_consensus.cc:3060] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:30.614605 18220 raft_consensus.cc:2468] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b9385e4bb9048af824071b2309c83ba in term 1.
I20260812 08:02:30.614957 17982 leader_election.cc:304] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b9385e4bb9048af824071b2309c83ba, 99e3ab4cbdbc49ce8ad92622b6e0e110; no voters: 
I20260812 08:02:30.615041 17907 raft_consensus.cc:2468] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b9385e4bb9048af824071b2309c83ba in term 1.
I20260812 08:02:30.615141 18300 raft_consensus.cc:2804] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:30.615398 18300 raft_consensus.cc:697] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [term 1 LEADER]: Becoming Leader. State: Replica: 4b9385e4bb9048af824071b2309c83ba, State: Running, Role: LEADER
I20260812 08:02:30.615723 18300 consensus_queue.cc:237] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 37013 } }
I20260812 08:02:30.618806 17737 catalog_manager.cc:5719] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b9385e4bb9048af824071b2309c83ba (127.0.186.66). New cstate: current_term: 1 leader_uuid: "4b9385e4bb9048af824071b2309c83ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 37013 } health_report { overall_health: HEALTHY } } }
I20260812 08:02:30.647840   745 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:02:30.649430   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 72cb331ade34427dbf3147f1f60f7c67 to finish bootstrapping
I20260812 08:02:30.654917   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 4b9385e4bb9048af824071b2309c83ba to finish bootstrapping
I20260812 08:02:30.659941   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 99e3ab4cbdbc49ce8ad92622b6e0e110 to finish bootstrapping
I20260812 08:02:30.675879   745 cluster_itest_util.cc:258] Not converged past 1 yet: 1.1 0.0 0.0
I20260812 08:02:30.778920   745 cluster_itest_util.cc:258] Not converged past 1 yet: 1.1 0.0 0.0
I20260812 08:02:30.981858   745 cluster_itest_util.cc:258] Not converged past 1 yet: 1.1 0.0 0.0
I20260812 08:02:31.000392 18300 consensus_queue.cc:1048] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:02:31.005986 18311 consensus_queue.cc:1048] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:02:32.505532 18314 raft_consensus.cc:493] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b9385e4bb9048af824071b2309c83ba)
I20260812 08:02:32.505702 18314 raft_consensus.cc:515] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 37013 } }
I20260812 08:02:32.506821 18314 leader_election.cc:290] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99e3ab4cbdbc49ce8ad92622b6e0e110 (127.0.186.67:44489), 4b9385e4bb9048af824071b2309c83ba (127.0.186.66:37013)
I20260812 08:02:32.507517 18321 raft_consensus.cc:493] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b9385e4bb9048af824071b2309c83ba)
I20260812 08:02:32.507639 18321 raft_consensus.cc:515] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 37013 } }
I20260812 08:02:32.508764 18321 leader_election.cc:290] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 72cb331ade34427dbf3147f1f60f7c67 (127.0.186.65:36365), 4b9385e4bb9048af824071b2309c83ba (127.0.186.66:37013)
I20260812 08:02:32.511060 18220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df4dbd05f89d44fc8799d314a5dc6f27" candidate_uuid: "72cb331ade34427dbf3147f1f60f7c67" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" is_pre_election: true
I20260812 08:02:32.511201 18220 raft_consensus.cc:2468] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72cb331ade34427dbf3147f1f60f7c67 in term 1.
I20260812 08:02:32.511581 17826 leader_election.cc:304] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72cb331ade34427dbf3147f1f60f7c67, 99e3ab4cbdbc49ce8ad92622b6e0e110; no voters: 
I20260812 08:02:32.511829 18314 raft_consensus.cc:2804] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 08:02:32.511929 18314 raft_consensus.cc:493] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b9385e4bb9048af824071b2309c83ba)
I20260812 08:02:32.511987 18314 raft_consensus.cc:3060] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:02:32.512617 17907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df4dbd05f89d44fc8799d314a5dc6f27" candidate_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72cb331ade34427dbf3147f1f60f7c67" is_pre_election: true
I20260812 08:02:32.513046 18314 raft_consensus.cc:515] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 37013 } }
I20260812 08:02:32.513309 17907 raft_consensus.cc:2393] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99e3ab4cbdbc49ce8ad92622b6e0e110 in current term 2: Already voted for candidate 72cb331ade34427dbf3147f1f60f7c67 in this term.
I20260812 08:02:32.513648 18314 leader_election.cc:290] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [CANDIDATE]: Term 2 election: Requested vote from peers 99e3ab4cbdbc49ce8ad92622b6e0e110 (127.0.186.67:44489), 4b9385e4bb9048af824071b2309c83ba (127.0.186.66:37013)
I20260812 08:02:32.513734 18220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df4dbd05f89d44fc8799d314a5dc6f27" candidate_uuid: "72cb331ade34427dbf3147f1f60f7c67" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110"
I20260812 08:02:32.513847 18220 raft_consensus.cc:3060] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:02:32.514549 18220 raft_consensus.cc:2468] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72cb331ade34427dbf3147f1f60f7c67 in term 2.
I20260812 08:02:32.514837 17826 leader_election.cc:304] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72cb331ade34427dbf3147f1f60f7c67, 99e3ab4cbdbc49ce8ad92622b6e0e110; no voters: 
I20260812 08:02:32.515018 18314 raft_consensus.cc:2804] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:02:32.515239 18314 raft_consensus.cc:697] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [term 2 LEADER]: Becoming Leader. State: Replica: 72cb331ade34427dbf3147f1f60f7c67, State: Running, Role: LEADER
I20260812 08:02:32.515553 18314 consensus_queue.cc:237] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 37013 } }
I20260812 08:02:32.518010 17736 catalog_manager.cc:5719] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 reported cstate change: term changed from 1 to 2, leader changed from 4b9385e4bb9048af824071b2309c83ba (127.0.186.66) to 72cb331ade34427dbf3147f1f60f7c67 (127.0.186.65). New cstate: current_term: 2 leader_uuid: "72cb331ade34427dbf3147f1f60f7c67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 37013 } health_report { overall_health: UNKNOWN } } }
I20260812 08:02:33.120785 18220 raft_consensus.cc:1275] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110 [term 2 FOLLOWER]: Refusing update from remote peer 72cb331ade34427dbf3147f1f60f7c67: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 08:02:33.121374 18335 consensus_queue.cc:1048] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260812 08:02:34.049779 17827 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: connection negotiation to 127.0.186.66:37013 for RPC RequestConsensusVote timed out after 1.534s (ON_OUTBOUND_QUEUE)
W20260812 08:02:34.571949 18140 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: connection negotiation to 127.0.186.66:37013 for RPC RequestConsensusVote timed out after 2.062s (ON_OUTBOUND_QUEUE)
W20260812 08:02:35.586491 17827 leader_election.cc:336] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4b9385e4bb9048af824071b2309c83ba (127.0.186.66:37013): Timed out: connection negotiation to 127.0.186.66:37013 for RPC RequestConsensusVote timed out after 1.534s (ON_OUTBOUND_QUEUE)
W20260812 08:02:35.682305 18322 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:02:32.508972 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.186.66:37013 (local address 127.0.186.67:45239)
0812 08:02:32.509304 (+   332us) negotiation.cc:107] Waiting for socket to connect
0812 08:02:32.509316 (+    12us) client_negotiation.cc:175] Beginning negotiation
0812 08:02:32.509396 (+    80us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 08:02:35.681752 (+3172356us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.186.66:37013: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":83,"threads_started":1}
W20260812 08:02:35.682307 18319 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:02:32.507087 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.186.66:37013 (local address 127.0.186.65:41983)
0812 08:02:32.507581 (+   494us) negotiation.cc:107] Waiting for socket to connect
0812 08:02:32.507594 (+    13us) client_negotiation.cc:175] Beginning negotiation
0812 08:02:32.507665 (+    71us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 08:02:35.681752 (+3174087us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.186.66:37013: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":23,"thread_start_us":51,"threads_started":1}
W20260812 08:02:35.682565 18140 leader_election.cc:336] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b9385e4bb9048af824071b2309c83ba (127.0.186.66:37013): Timed out: Client connection negotiation failed: client connection to 127.0.186.66:37013: received 0 of 4 requested bytes
W20260812 08:02:35.682617 17827 leader_election.cc:336] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b9385e4bb9048af824071b2309c83ba (127.0.186.66:37013): Timed out: Client connection negotiation failed: client connection to 127.0.186.66:37013: received 0 of 4 requested bytes
I20260812 08:02:35.682668 18140 leader_election.cc:304] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99e3ab4cbdbc49ce8ad92622b6e0e110; no voters: 4b9385e4bb9048af824071b2309c83ba, 72cb331ade34427dbf3147f1f60f7c67
I20260812 08:02:35.683280 18350 raft_consensus.cc:2749] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 08:02:38.753827 18319 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:02:35.683430 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.186.66:37013 (local address 127.0.186.65:45041)
0812 08:02:35.683570 (+   140us) negotiation.cc:107] Waiting for socket to connect
0812 08:02:35.683584 (+    14us) client_negotiation.cc:175] Beginning negotiation
0812 08:02:35.683672 (+    88us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 08:02:38.753706 (+3070034us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.186.66:37013: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 08:02:38.754117 17827 consensus_peers.cc:597] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 -> Peer 4b9385e4bb9048af824071b2309c83ba (127.0.186.66:37013): Couldn't send request to peer 4b9385e4bb9048af824071b2309c83ba. Status: Timed out: Client connection negotiation failed: client connection to 127.0.186.66:37013: received 0 of 4 requested bytes. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:02:38.999338 18356 consensus_queue.cc:579] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [LEADER]: Leader has been unable to successfully communicate with peer 4b9385e4bb9048af824071b2309c83ba for more than 5 seconds (6.483s)
W20260812 08:02:42.081799 18319 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:02:39.000041 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.186.66:37013 (local address 127.0.186.65:37429)
0812 08:02:39.000199 (+   158us) negotiation.cc:107] Waiting for socket to connect
0812 08:02:39.000218 (+    19us) client_negotiation.cc:175] Beginning negotiation
0812 08:02:39.000329 (+   111us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 08:02:42.081681 (+3081352us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.186.66:37013: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 08:02:42.082003 17827 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: Client connection negotiation failed: client connection to 127.0.186.66:37013: received 0 of 4 requested bytes [suppressed 2 similar messages]
W20260812 08:02:45.153774 18319 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:02:42.082759 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.186.66:37013 (local address 127.0.186.65:42895)
0812 08:02:42.082877 (+   118us) negotiation.cc:107] Waiting for socket to connect
0812 08:02:42.082891 (+    14us) client_negotiation.cc:175] Beginning negotiation
0812 08:02:42.082990 (+    99us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 08:02:45.153663 (+3070673us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.186.66:37013: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 08:02:47.730441   745 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
I20260812 08:02:47.730775   745 raft_consensus-itest-base.cc:268] Waiting until we've written at least 4MB...
I20260812 08:02:47.793376 17959 maintenance_manager.cc:419] P 72cb331ade34427dbf3147f1f60f7c67: Scheduling FlushMRSOp(df4dbd05f89d44fc8799d314a5dc6f27): 2761400 bytes log retention, and flush 4038921 bytes memory
I20260812 08:02:47.820513 18276 maintenance_manager.cc:419] P 99e3ab4cbdbc49ce8ad92622b6e0e110: Scheduling FlushMRSOp(df4dbd05f89d44fc8799d314a5dc6f27): 3418830 bytes log retention, and flush 5079305 bytes memory
I20260812 08:02:47.839789   745 raft_consensus-itest-base.cc:274] Waiting for log GC on 72cb331ade34427dbf3147f1f60f7c67
I20260812 08:02:47.860476 17828 maintenance_manager.cc:643] P 72cb331ade34427dbf3147f1f60f7c67: FlushMRSOp(df4dbd05f89d44fc8799d314a5dc6f27) complete. Timing: real 0.067s	user 0.041s	sys 0.004s Metrics: {"bytes_written":135869,"cfile_init":1,"compiler_manager_pool.queue_time_us":229,"drs_written":1,"lbm_read_time_us":73,"lbm_reads_lt_1ms":4,"lbm_write_time_us":666,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":24,"thread_start_us":265,"threads_started":2}
I20260812 08:02:47.861430 17959 maintenance_manager.cc:419] P 72cb331ade34427dbf3147f1f60f7c67: Scheduling LogGCOp(df4dbd05f89d44fc8799d314a5dc6f27): free 2761400 bytes of WAL
I20260812 08:02:47.861794 17828 log_reader.cc:385] T df4dbd05f89d44fc8799d314a5dc6f27: removed 3 log segments from log reader
I20260812 08:02:47.861909 17828 log.cc:1079] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/df4dbd05f89d44fc8799d314a5dc6f27/wal-000000001 (ops 1-9)
I20260812 08:02:47.862017 17828 log.cc:1079] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/df4dbd05f89d44fc8799d314a5dc6f27/wal-000000002 (ops 10-16)
I20260812 08:02:47.862089 17828 log.cc:1079] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/df4dbd05f89d44fc8799d314a5dc6f27/wal-000000003 (ops 17-23)
I20260812 08:02:47.863087 17828 maintenance_manager.cc:643] P 72cb331ade34427dbf3147f1f60f7c67: LogGCOp(df4dbd05f89d44fc8799d314a5dc6f27) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 08:02:47.863539 17959 maintenance_manager.cc:419] P 72cb331ade34427dbf3147f1f60f7c67: Scheduling FlushMRSOp(df4dbd05f89d44fc8799d314a5dc6f27): 1840912 bytes log retention, and flush 2998537 bytes memory
I20260812 08:02:47.870723   745 raft_consensus-itest-base.cc:279] Log GC complete on 72cb331ade34427dbf3147f1f60f7c67
I20260812 08:02:47.887403 18141 maintenance_manager.cc:643] P 99e3ab4cbdbc49ce8ad92622b6e0e110: FlushMRSOp(df4dbd05f89d44fc8799d314a5dc6f27) complete. Timing: real 0.066s	user 0.044s	sys 0.009s Metrics: {"bytes_written":135937,"cfile_init":1,"compiler_manager_pool.queue_time_us":233,"drs_written":1,"lbm_read_time_us":85,"lbm_reads_lt_1ms":4,"lbm_write_time_us":806,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":30,"thread_start_us":260,"threads_started":2}
I20260812 08:02:47.887811 17828 maintenance_manager.cc:643] P 72cb331ade34427dbf3147f1f60f7c67: FlushMRSOp(df4dbd05f89d44fc8799d314a5dc6f27) complete. Timing: real 0.024s	user 0.021s	sys 0.000s Metrics: {"bytes_written":135803,"cfile_init":1,"dirs.queue_time_us":49,"dirs.run_cpu_time_us":196,"dirs.run_wall_time_us":1404,"drs_written":1,"lbm_read_time_us":101,"lbm_reads_lt_1ms":4,"lbm_write_time_us":602,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":15}
I20260812 08:02:47.888335 18276 maintenance_manager.cc:419] P 99e3ab4cbdbc49ce8ad92622b6e0e110: Scheduling LogGCOp(df4dbd05f89d44fc8799d314a5dc6f27): free 2629877 bytes of WAL
I20260812 08:02:47.888640 18141 log_reader.cc:385] T df4dbd05f89d44fc8799d314a5dc6f27: removed 3 log segments from log reader
I20260812 08:02:47.888751 18141 log.cc:1079] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/df4dbd05f89d44fc8799d314a5dc6f27/wal-000000001 (ops 1-9)
I20260812 08:02:47.888746 17959 maintenance_manager.cc:419] P 72cb331ade34427dbf3147f1f60f7c67: Scheduling UndoDeltaBlockGCOp(df4dbd05f89d44fc8799d314a5dc6f27): 704 bytes on disk
I20260812 08:02:47.888856 18141 log.cc:1079] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/df4dbd05f89d44fc8799d314a5dc6f27/wal-000000002 (ops 10-15)
I20260812 08:02:47.888895 18141 log.cc:1079] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/df4dbd05f89d44fc8799d314a5dc6f27/wal-000000003 (ops 16-22)
I20260812 08:02:47.889602 17828 maintenance_manager.cc:643] P 72cb331ade34427dbf3147f1f60f7c67: UndoDeltaBlockGCOp(df4dbd05f89d44fc8799d314a5dc6f27) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":142,"lbm_reads_lt_1ms":8}
I20260812 08:02:47.889825 18141 maintenance_manager.cc:643] P 99e3ab4cbdbc49ce8ad92622b6e0e110: LogGCOp(df4dbd05f89d44fc8799d314a5dc6f27) complete. Timing: real 0.001s	user 0.000s	sys 0.001s Metrics: {}
I20260812 08:02:47.890089 17959 maintenance_manager.cc:419] P 72cb331ade34427dbf3147f1f60f7c67: Scheduling CompactRowSetsOp(df4dbd05f89d44fc8799d314a5dc6f27): perf score=1.012920
I20260812 08:02:47.890128 18276 maintenance_manager.cc:419] P 99e3ab4cbdbc49ce8ad92622b6e0e110: Scheduling UndoDeltaBlockGCOp(df4dbd05f89d44fc8799d314a5dc6f27): 468 bytes on disk
I20260812 08:02:47.890609 18141 maintenance_manager.cc:643] P 99e3ab4cbdbc49ce8ad92622b6e0e110: UndoDeltaBlockGCOp(df4dbd05f89d44fc8799d314a5dc6f27) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":63,"lbm_reads_lt_1ms":4}
I20260812 08:02:47.946242 17828 maintenance_manager.cc:643] P 72cb331ade34427dbf3147f1f60f7c67: CompactRowSetsOp(df4dbd05f89d44fc8799d314a5dc6f27) complete. Timing: real 0.056s	user 0.056s	sys 0.000s Metrics: {"bytes_written":136001,"cfile_cache_hit":10,"cfile_cache_hit_bytes":262392,"cfile_cache_miss":26,"cfile_cache_miss_bytes":264348,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":675,"dirs.run_cpu_time_us":160,"dirs.run_wall_time_us":1245,"drs_written":1,"lbm_read_time_us":750,"lbm_reads_lt_1ms":46,"lbm_write_time_us":649,"lbm_writes_lt_1ms":23,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":39,"thread_start_us":365,"threads_started":5}
I20260812 08:02:47.946919 17959 maintenance_manager.cc:419] P 72cb331ade34427dbf3147f1f60f7c67: Scheduling UndoDeltaBlockGCOp(df4dbd05f89d44fc8799d314a5dc6f27): 570 bytes on disk
I20260812 08:02:47.947302 17828 maintenance_manager.cc:643] P 72cb331ade34427dbf3147f1f60f7c67: UndoDeltaBlockGCOp(df4dbd05f89d44fc8799d314a5dc6f27) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":61,"lbm_reads_lt_1ms":4}
W20260812 08:02:48.737874 18319 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:02:45.502105 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.186.66:37013 (local address 127.0.186.65:57501)
0812 08:02:45.502246 (+   141us) negotiation.cc:107] Waiting for socket to connect
0812 08:02:45.502261 (+    15us) client_negotiation.cc:175] Beginning negotiation
0812 08:02:45.502365 (+   104us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 08:02:48.737743 (+3235378us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.186.66:37013: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 08:02:48.738111 17827 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: Client connection negotiation failed: client connection to 127.0.186.66:37013: received 0 of 4 requested bytes
I20260812 08:02:49.872045   745 raft_consensus-itest-base.cc:295] Servers converged with original term 2
I20260812 08:02:49.872120   745 raft_consensus-itest-base.cc:300] Resuming 4b9385e4bb9048af824071b2309c83ba
I20260812 08:02:49.874311 18311 consensus_queue.cc:579] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [LEADER]: Leader has been unable to successfully communicate with peer 72cb331ade34427dbf3147f1f60f7c67 for more than 5 seconds (18.869s)
I20260812 08:02:49.874594 18311 consensus_queue.cc:579] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [LEADER]: Leader has been unable to successfully communicate with peer 99e3ab4cbdbc49ce8ad92622b6e0e110 for more than 5 seconds (18.866s)
I20260812 08:02:49.875239 17907 raft_consensus.cc:1240] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [term 2 LEADER]: Rejecting Update request from peer 4b9385e4bb9048af824071b2309c83ba for earlier term 1. Current term is 2. Ops: []
I20260812 08:02:49.876097 18300 consensus_queue.cc:1059] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 41, Time since last communication: 0.000s
I20260812 08:02:49.876271 18300 consensus_queue.cc:1243] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [LEADER]: Peer 72cb331ade34427dbf3147f1f60f7c67 log is divergent from this leader: its last log entry 2.41 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 41
I20260812 08:02:49.876346 18300 raft_consensus.cc:3055] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [term 1 LEADER]: Stepping down as leader of term 1
I20260812 08:02:49.876382 18300 raft_consensus.cc:740] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b9385e4bb9048af824071b2309c83ba, State: Running, Role: LEADER
I20260812 08:02:49.876559 18300 consensus_queue.cc:260] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 37013 } }
I20260812 08:02:49.876850 18300 raft_consensus.cc:3060] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [term 1 FOLLOWER]: Advancing to term 2
W20260812 08:02:49.877094 18414 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:02:49.875742 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.186.65:41983 (local address 127.0.186.66:37013)
0812 08:02:49.876261 (+   519us) server_negotiation.cc:207] Beginning negotiation
0812 08:02:49.876266 (+     5us) server_negotiation.cc:400] Waiting for connection header
0812 08:02:49.876281 (+    15us) server_negotiation.cc:408] Connection header received
0812 08:02:49.876326 (+    45us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 08:02:49.876329 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 08:02:49.876373 (+    44us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 08:02:49.876513 (+   140us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.65:41983: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":79,"threads_started":1}
I20260812 08:02:49.877703 18219 raft_consensus.cc:1240] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110 [term 2 FOLLOWER]: Rejecting Update request from peer 4b9385e4bb9048af824071b2309c83ba for earlier term 1. Current term is 2. Ops: []
W20260812 08:02:49.878535 18414 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:02:49.875887 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.186.65:45041 (local address 127.0.186.66:37013)
0812 08:02:49.878230 (+  2343us) server_negotiation.cc:207] Beginning negotiation
0812 08:02:49.878235 (+     5us) server_negotiation.cc:400] Waiting for connection header
0812 08:02:49.878248 (+    13us) server_negotiation.cc:408] Connection header received
0812 08:02:49.878286 (+    38us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 08:02:49.878289 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 08:02:49.878331 (+    42us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 08:02:49.878446 (+   115us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.65:45041: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":1357,"thread_start_us":36,"threads_started":1}
W20260812 08:02:49.878935 18417 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:02:49.875923 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.186.67:45239 (local address 127.0.186.66:37013)
0812 08:02:49.878602 (+  2679us) server_negotiation.cc:207] Beginning negotiation
0812 08:02:49.878606 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 08:02:49.878620 (+    14us) server_negotiation.cc:408] Connection header received
0812 08:02:49.878661 (+    41us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 08:02:49.878663 (+     2us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 08:02:49.878703 (+    40us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 08:02:49.878821 (+   118us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.67:45239: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":2608,"thread_start_us":64,"threads_started":1}
W20260812 08:02:49.879172 18414 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:02:49.875962 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.186.65:42895 (local address 127.0.186.66:37013)
0812 08:02:49.878809 (+  2847us) server_negotiation.cc:207] Beginning negotiation
0812 08:02:49.878813 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 08:02:49.878825 (+    12us) server_negotiation.cc:408] Connection header received
0812 08:02:49.878862 (+    37us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 08:02:49.878865 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 08:02:49.878905 (+    40us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 08:02:49.879086 (+   181us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.65:42895: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":2796,"thread_start_us":48,"threads_started":1}
W20260812 08:02:49.879639 18414 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:02:49.876054 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.186.65:37429 (local address 127.0.186.66:37013)
0812 08:02:49.879394 (+  3340us) server_negotiation.cc:207] Beginning negotiation
0812 08:02:49.879398 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 08:02:49.879409 (+    11us) server_negotiation.cc:408] Connection header received
0812 08:02:49.879443 (+    34us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 08:02:49.879446 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 08:02:49.879480 (+    34us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 08:02:49.879565 (+    85us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.65:37429: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":3289,"thread_start_us":81,"threads_started":1}
W20260812 08:02:49.879745 18420 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:02:49.876189 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.186.65:57501 (local address 127.0.186.66:37013)
0812 08:02:49.879452 (+  3263us) server_negotiation.cc:207] Beginning negotiation
0812 08:02:49.879455 (+     3us) server_negotiation.cc:400] Waiting for connection header
0812 08:02:49.879465 (+    10us) server_negotiation.cc:408] Connection header received
0812 08:02:49.879498 (+    33us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 08:02:49.879500 (+     2us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 08:02:49.879533 (+    33us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 08:02:49.879647 (+   114us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.65:57501: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":3205,"thread_start_us":72,"threads_started":1}
I20260812 08:02:49.884328 18413 consensus_queue.cc:753] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [LEADER]: The logs necessary to catch up peer 4b9385e4bb9048af824071b2309c83ba have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 3..34: Segment 1 which contained index 3 has been GCed)
I20260812 08:02:49.884403 18413 consensus_queue.cc:579] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [LEADER]: The logs necessary to catch up peer 4b9385e4bb9048af824071b2309c83ba have been garbage collected. The replica will never be able to catch up
I20260812 08:02:49.887125 17907 consensus_queue.cc:237] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 41, Committed index: 41, Last appended: 2.41, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } }
I20260812 08:02:49.888466 18219 raft_consensus.cc:1275] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110 [term 2 FOLLOWER]: Refusing update from remote peer 72cb331ade34427dbf3147f1f60f7c67: Log matching property violated. Preceding OpId in replica: term: 2 index: 41. Preceding OpId from leader: term: 2 index: 42. (index mismatch)
I20260812 08:02:49.888921 18413 consensus_queue.cc:1048] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 42, Last known committed idx: 41, Time since last communication: 0.000s
I20260812 08:02:49.891214 18413 raft_consensus.cc:2955] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [term 2 LEADER]: Committing config change with OpId 2.42: config changed from index -1 to 42, VOTER 4b9385e4bb9048af824071b2309c83ba (127.0.186.66) evicted. New config: { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } } }
I20260812 08:02:49.893095 17714 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet df4dbd05f89d44fc8799d314a5dc6f27 with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:02:49.894157 17736 catalog_manager.cc:5719] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 reported cstate change: config changed from index -1 to 42, VOTER 4b9385e4bb9048af824071b2309c83ba (127.0.186.66) evicted. New cstate: current_term: 2 leader_uuid: "72cb331ade34427dbf3147f1f60f7c67" committed_config { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } health_report { overall_health: HEALTHY } } }
I20260812 08:02:49.894121 18219 raft_consensus.cc:2955] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110 [term 2 FOLLOWER]: Committing config change with OpId 2.42: config changed from index -1 to 42, VOTER 4b9385e4bb9048af824071b2309c83ba (127.0.186.66) evicted. New config: { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } } }
I20260812 08:02:49.897830 18034 tablet_service.cc:1558] Processing DeleteTablet for tablet df4dbd05f89d44fc8799d314a5dc6f27 with delete_type TABLET_DATA_TOMBSTONED (TS 4b9385e4bb9048af824071b2309c83ba not found in new config with opid_index 42) from {username='slave'} at 127.0.0.1:41820
I20260812 08:02:49.898481 18433 tablet_replica.cc:333] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba: stopping tablet replica
I20260812 08:02:49.898795 18433 raft_consensus.cc:2243] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 08:02:49.899256 18276 maintenance_manager.cc:419] P 99e3ab4cbdbc49ce8ad92622b6e0e110: Scheduling LogGCOp(df4dbd05f89d44fc8799d314a5dc6f27): free 788953 bytes of WAL
I20260812 08:02:49.899449 18141 log_reader.cc:385] T df4dbd05f89d44fc8799d314a5dc6f27: removed 1 log segments from log reader
I20260812 08:02:49.899446 18433 raft_consensus.cc:2272] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 08:02:49.899490 18141 log.cc:1079] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/df4dbd05f89d44fc8799d314a5dc6f27/wal-000000004 (ops 23-28)
I20260812 08:02:49.899835 18141 maintenance_manager.cc:643] P 99e3ab4cbdbc49ce8ad92622b6e0e110: LogGCOp(df4dbd05f89d44fc8799d314a5dc6f27) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {}
I20260812 08:02:49.901278 18433 ts_tablet_manager.cc:1916] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 08:02:49.902139   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 17696
I20260812 08:02:49.906956 18433 ts_tablet_manager.cc:1929] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.2
I20260812 08:02:49.907140 18433 log.cc:1199] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba: Deleting WAL directory at /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/wals/df4dbd05f89d44fc8799d314a5dc6f27
W20260812 08:02:49.917028 17981 connection.cc:573] server connection from 127.0.0.1:41820 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 08:02:49.918004   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:44527
--webserver_interface=127.0.186.126
--webserver_port=33695
--builtin_ntp_servers=127.0.186.84:46879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:44527
--raft_prepare_replacement_before_eviction=true
--master_add_server_when_underreplicated=false
--master_add_server_when_underreplicated=true with env {}
I20260812 08:02:49.955605 17959 maintenance_manager.cc:419] P 72cb331ade34427dbf3147f1f60f7c67: Scheduling LogGCOp(df4dbd05f89d44fc8799d314a5dc6f27): free 1840912 bytes of WAL
I20260812 08:02:49.955891 17828 log_reader.cc:385] T df4dbd05f89d44fc8799d314a5dc6f27: removed 2 log segments from log reader
I20260812 08:02:49.955938 17828 log.cc:1079] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/df4dbd05f89d44fc8799d314a5dc6f27/wal-000000004 (ops 24-30)
I20260812 08:02:49.955978 17828 log.cc:1079] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/df4dbd05f89d44fc8799d314a5dc6f27/wal-000000005 (ops 31-37)
I20260812 08:02:49.956692 17828 maintenance_manager.cc:643] P 72cb331ade34427dbf3147f1f60f7c67: LogGCOp(df4dbd05f89d44fc8799d314a5dc6f27) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
W20260812 08:02:50.027238 18434 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:50.027444 18434 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:50.027501 18434 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:50.030759 18434 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:02:50.030813 18434 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:50.030841 18434 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:02:50.030865 18434 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:02:50.034638 18434 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:46879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:44527
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:44527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=33695
--never_fsync=true
--heap_profile_path=/tmp/kudu.18434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:50.035655 18434 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:50.036712 18434 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:50.041946 18439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:50.042059 18441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:50.041953 18443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:50.042935 18434 server_base.cc:1061] running on GCE node
I20260812 08:02:50.043422 18434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:50.044306 18434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:50.045490 18434 hybrid_clock.cc:648] HybridClock initialized: now 1786521770045461 us; error 41 us; skew 500 ppm
I20260812 08:02:50.047386 18434 webserver.cc:533] Webserver started at http://127.0.186.126:33695/ using document root <none> and password file <none>
I20260812 08:02:50.048010 18434 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:50.048097 18434 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:50.051692 18434 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:02:50.053690 18450 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:50.054800 18434 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:50.054911 18434 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "76fa69a4f44344528ad0eb506177898b"
format_stamp: "Formatted at 2026-08-12 08:02:29 on dist-test-slave-49h3"
I20260812 08:02:50.055307 18434 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:50.076057 18434 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:50.076764 18434 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:50.076936 18434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:50.085117 18521 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:44527 every 8 connection(s)
I20260812 08:02:50.085121 18434 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:44527
I20260812 08:02:50.086323 18434 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:02:50.090442 18522 sys_catalog.cc:263] Verifying existing consensus state
I20260812 08:02:50.091958 18522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b: Bootstrap starting.
I20260812 08:02:50.094779   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 18434
I20260812 08:02:50.107577 18522 log.cc:826] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:50.115840 18522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:02:50.116214 18522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b: Bootstrap complete.
I20260812 08:02:50.119477 18522 raft_consensus.cc:359] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76fa69a4f44344528ad0eb506177898b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44527 } }
I20260812 08:02:50.119791 18522 raft_consensus.cc:740] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76fa69a4f44344528ad0eb506177898b, State: Initialized, Role: FOLLOWER
I20260812 08:02:50.120215 18522 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76fa69a4f44344528ad0eb506177898b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44527 } }
I20260812 08:02:50.120344 18522 raft_consensus.cc:399] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:50.120409 18522 raft_consensus.cc:493] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:50.120502 18522 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:02:50.122059 18522 raft_consensus.cc:515] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76fa69a4f44344528ad0eb506177898b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44527 } }
I20260812 08:02:50.122368 18522 leader_election.cc:304] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76fa69a4f44344528ad0eb506177898b; no voters: 
I20260812 08:02:50.122665 18522 leader_election.cc:290] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [CANDIDATE]: Term 2 election: Requested vote from peers 
I20260812 08:02:50.122766 18527 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:02:50.122927 18527 raft_consensus.cc:697] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [term 2 LEADER]: Becoming Leader. State: Replica: 76fa69a4f44344528ad0eb506177898b, State: Running, Role: LEADER
I20260812 08:02:50.123205 18527 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76fa69a4f44344528ad0eb506177898b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44527 } }
I20260812 08:02:50.123805 18522 sys_catalog.cc:565] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:02:50.124615 18528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "76fa69a4f44344528ad0eb506177898b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76fa69a4f44344528ad0eb506177898b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44527 } } }
I20260812 08:02:50.124653 18529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76fa69a4f44344528ad0eb506177898b. Latest consensus state: current_term: 2 leader_uuid: "76fa69a4f44344528ad0eb506177898b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76fa69a4f44344528ad0eb506177898b" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 44527 } } }
I20260812 08:02:50.124820 18529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:50.124820 18528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:50.125403 18539 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:02:50.128235 18539 catalog_manager.cc:679] Loaded metadata for table TestTable [id=2f64cab8f6b84c97aeaefdbff7bf2c32]
I20260812 08:02:50.129257 18539 tablet_loader.cc:96] loaded metadata for tablet df4dbd05f89d44fc8799d314a5dc6f27 (table TestTable [id=2f64cab8f6b84c97aeaefdbff7bf2c32])
I20260812 08:02:50.129388 18539 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:02:50.130043 18539 catalog_manager.cc:1295] Loaded cluster ID: 8680ff2e259b4c0195556ed3cadc29d4
I20260812 08:02:50.130100 18539 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:02:50.131703 18539 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:02:50.132275 18539 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 76fa69a4f44344528ad0eb506177898b: Loaded TSK: 0
I20260812 08:02:50.133041 18539 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:02:50.890862 18470 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" instance_seqno: 1786521750287755) as {username='slave'} at 127.0.186.66:46987; Asking this server to re-register.
I20260812 08:02:50.891448 18117 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:50.891638 18117 heartbeater.cc:507] Master 127.0.186.126:44527 requested a full tablet report, sending...
I20260812 08:02:50.892759 18470 ts_manager.cc:194] Registered new tserver with Master: 4b9385e4bb9048af824071b2309c83ba (127.0.186.66:37013)
I20260812 08:02:50.901888 18469 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" instance_seqno: 1786521750454774) as {username='slave'} at 127.0.186.67:53829; Asking this server to re-register.
I20260812 08:02:50.901862 18470 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" instance_seqno: 1786521750126765) as {username='slave'} at 127.0.186.65:49885; Asking this server to re-register.
I20260812 08:02:50.902362 17958 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:50.902395 18275 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:50.902541 17958 heartbeater.cc:507] Master 127.0.186.126:44527 requested a full tablet report, sending...
I20260812 08:02:50.902552 18275 heartbeater.cc:507] Master 127.0.186.126:44527 requested a full tablet report, sending...
I20260812 08:02:50.903322 18469 ts_manager.cc:194] Registered new tserver with Master: 99e3ab4cbdbc49ce8ad92622b6e0e110 (127.0.186.67:44489)
I20260812 08:02:50.903383 18470 ts_manager.cc:194] Registered new tserver with Master: 72cb331ade34427dbf3147f1f60f7c67 (127.0.186.65:36365)
I20260812 08:02:50.909891 17907 consensus_queue.cc:237] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 42, Committed index: 42, Last appended: 2.42, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 37013 } attrs { promote: true } }
I20260812 08:02:50.911214 18219 raft_consensus.cc:1275] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110 [term 2 FOLLOWER]: Refusing update from remote peer 72cb331ade34427dbf3147f1f60f7c67: Log matching property violated. Preceding OpId in replica: term: 2 index: 42. Preceding OpId from leader: term: 2 index: 43. (index mismatch)
I20260812 08:02:50.911599 18563 consensus_queue.cc:1048] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 43, Last known committed idx: 42, Time since last communication: 0.000s
W20260812 08:02:50.911808 17827 consensus_peers.cc:597] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 -> Peer 4b9385e4bb9048af824071b2309c83ba (127.0.186.66:37013): Couldn't send request to peer 4b9385e4bb9048af824071b2309c83ba. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:02:50.913328 18549 raft_consensus.cc:2955] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [term 2 LEADER]: Committing config change with OpId 2.43: config changed from index 42 to 43, NON_VOTER 4b9385e4bb9048af824071b2309c83ba (127.0.186.66) added. New config: { opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 37013 } attrs { promote: true } } }
I20260812 08:02:50.913794 18219 raft_consensus.cc:2955] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110 [term 2 FOLLOWER]: Committing config change with OpId 2.43: config changed from index 42 to 43, NON_VOTER 4b9385e4bb9048af824071b2309c83ba (127.0.186.66) added. New config: { opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 37013 } attrs { promote: true } } }
I20260812 08:02:50.914915 18453 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet df4dbd05f89d44fc8799d314a5dc6f27 with cas_config_opid_index 42: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:02:50.915979 18469 catalog_manager.cc:5719] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110 reported cstate change: config changed from index 42 to 43, NON_VOTER 4b9385e4bb9048af824071b2309c83ba (127.0.186.66) added. New cstate: current_term: 2 leader_uuid: "72cb331ade34427dbf3147f1f60f7c67" committed_config { opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 37013 } attrs { promote: true } } }
I20260812 08:02:51.375239 18571 ts_tablet_manager.cc:933] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba: Initiating tablet copy from peer 72cb331ade34427dbf3147f1f60f7c67 (127.0.186.65:36365)
I20260812 08:02:51.375825 18571 tablet_copy_client.cc:323] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba: tablet copy: Beginning tablet copy session from remote peer at address 127.0.186.65:36365
I20260812 08:02:51.376632 17928 tablet_copy_service.cc:140] P 72cb331ade34427dbf3147f1f60f7c67: Received BeginTabletCopySession request for tablet df4dbd05f89d44fc8799d314a5dc6f27 from peer 4b9385e4bb9048af824071b2309c83ba ({username='slave'} at 127.0.186.66:54535)
I20260812 08:02:51.376812 17928 tablet_copy_service.cc:161] P 72cb331ade34427dbf3147f1f60f7c67: Beginning new tablet copy session on tablet df4dbd05f89d44fc8799d314a5dc6f27 from peer 4b9385e4bb9048af824071b2309c83ba at {username='slave'} at 127.0.186.66:54535: session id = 4b9385e4bb9048af824071b2309c83ba-df4dbd05f89d44fc8799d314a5dc6f27
I20260812 08:02:51.378381 17928 tablet_copy_source_session.cc:215] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67: Tablet Copy: opened 5 blocks and 1 log segments
I20260812 08:02:51.380069 18571 ts_tablet_manager.cc:1916] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260812 08:02:51.381966 18571 ts_tablet_manager.cc:1929] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 2.2
I20260812 08:02:51.382155 18571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df4dbd05f89d44fc8799d314a5dc6f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:51.383252 18571 tablet_copy_client.cc:806] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba: tablet copy: Starting download of 5 data blocks...
I20260812 08:02:51.390720 18571 tablet_copy_client.cc:670] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba: tablet copy: Starting download of 1 WAL segments...
I20260812 08:02:51.393967 18571 tablet_copy_client.cc:538] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 08:02:51.395751 18571 tablet_bootstrap.cc:492] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba: Bootstrap starting.
I20260812 08:02:51.405857 18571 tablet_bootstrap.cc:492] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=4} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=2. Pending: 0 replicates
I20260812 08:02:51.406118 18571 tablet_bootstrap.cc:492] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba: Bootstrap complete.
I20260812 08:02:51.406355 18571 ts_tablet_manager.cc:1403] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba: Time spent bootstrapping tablet: real 0.011s	user 0.012s	sys 0.000s
I20260812 08:02:51.406814 18571 raft_consensus.cc:359] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 37013 } attrs { promote: true } }
I20260812 08:02:51.406914 18571 raft_consensus.cc:740] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 4b9385e4bb9048af824071b2309c83ba, State: Initialized, Role: LEARNER
I20260812 08:02:51.407018 18571 consensus_queue.cc:260] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 43, Last appended: 2.43, Last appended by leader: 43, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 37013 } attrs { promote: true } }
I20260812 08:02:51.407298 18571 ts_tablet_manager.cc:1434] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba: Time spent starting tablet: real 0.001s	user 0.002s	sys 0.000s
I20260812 08:02:51.407917 17928 tablet_copy_service.cc:342] P 72cb331ade34427dbf3147f1f60f7c67: Request end of tablet copy session 4b9385e4bb9048af824071b2309c83ba-df4dbd05f89d44fc8799d314a5dc6f27 received from {username='slave'} at 127.0.186.66:54535
I20260812 08:02:51.408013 17928 tablet_copy_service.cc:434] P 72cb331ade34427dbf3147f1f60f7c67: ending tablet copy session 4b9385e4bb9048af824071b2309c83ba-df4dbd05f89d44fc8799d314a5dc6f27 on tablet df4dbd05f89d44fc8799d314a5dc6f27 with peer 4b9385e4bb9048af824071b2309c83ba
I20260812 08:02:51.477576 18118 maintenance_manager.cc:419] P 4b9385e4bb9048af824071b2309c83ba: Scheduling UndoDeltaBlockGCOp(df4dbd05f89d44fc8799d314a5dc6f27): 570 bytes on disk
I20260812 08:02:51.478576 17984 maintenance_manager.cc:643] P 4b9385e4bb9048af824071b2309c83ba: UndoDeltaBlockGCOp(df4dbd05f89d44fc8799d314a5dc6f27) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":76,"lbm_reads_lt_1ms":4}
I20260812 08:02:51.843673 18058 raft_consensus.cc:1217] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [term 2 LEARNER]: Deduplicated request from leader. Original: 2.42->[2.43-2.43]   Dedup: 2.43->[]
I20260812 08:02:52.223786 18576 raft_consensus.cc:1064] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67: attempting to promote NON_VOTER 4b9385e4bb9048af824071b2309c83ba to VOTER
I20260812 08:02:52.224289 18576 consensus_queue.cc:237] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 43, Committed index: 43, Last appended: 2.43, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 37013 } attrs { promote: false } }
I20260812 08:02:52.225709 18058 raft_consensus.cc:1275] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [term 2 LEARNER]: Refusing update from remote peer 72cb331ade34427dbf3147f1f60f7c67: Log matching property violated. Preceding OpId in replica: term: 2 index: 43. Preceding OpId from leader: term: 2 index: 44. (index mismatch)
I20260812 08:02:52.225770 18219 raft_consensus.cc:1275] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110 [term 2 FOLLOWER]: Refusing update from remote peer 72cb331ade34427dbf3147f1f60f7c67: Log matching property violated. Preceding OpId in replica: term: 2 index: 43. Preceding OpId from leader: term: 2 index: 44. (index mismatch)
I20260812 08:02:52.226171 18578 consensus_queue.cc:1048] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 37013 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 44, Last known committed idx: 43, Time since last communication: 0.000s
I20260812 08:02:52.226377 18576 consensus_queue.cc:1048] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 44, Last known committed idx: 43, Time since last communication: 0.000s
I20260812 08:02:52.228047 18578 raft_consensus.cc:2955] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 [term 2 LEADER]: Committing config change with OpId 2.44: config changed from index 43 to 44, 4b9385e4bb9048af824071b2309c83ba (127.0.186.66) changed from NON_VOTER to VOTER. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 37013 } attrs { promote: false } } }
I20260812 08:02:52.228506 18219 raft_consensus.cc:2955] T df4dbd05f89d44fc8799d314a5dc6f27 P 99e3ab4cbdbc49ce8ad92622b6e0e110 [term 2 FOLLOWER]: Committing config change with OpId 2.44: config changed from index 43 to 44, 4b9385e4bb9048af824071b2309c83ba (127.0.186.66) changed from NON_VOTER to VOTER. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 37013 } attrs { promote: false } } }
I20260812 08:02:52.228847 18058 raft_consensus.cc:2955] T df4dbd05f89d44fc8799d314a5dc6f27 P 4b9385e4bb9048af824071b2309c83ba [term 2 FOLLOWER]: Committing config change with OpId 2.44: config changed from index 43 to 44, 4b9385e4bb9048af824071b2309c83ba (127.0.186.66) changed from NON_VOTER to VOTER. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 37013 } attrs { promote: false } } }
I20260812 08:02:52.230713 18469 catalog_manager.cc:5719] T df4dbd05f89d44fc8799d314a5dc6f27 P 72cb331ade34427dbf3147f1f60f7c67 reported cstate change: config changed from index 43 to 44, 4b9385e4bb9048af824071b2309c83ba (127.0.186.66) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "72cb331ade34427dbf3147f1f60f7c67" committed_config { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "99e3ab4cbdbc49ce8ad92622b6e0e110" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 44489 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72cb331ade34427dbf3147f1f60f7c67" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 36365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b9385e4bb9048af824071b2309c83ba" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 37013 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 08:02:52.302868 17880 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260812 08:02:52.302894 18034 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:02:52.304786 18190 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |       Address       | Status
----------------------------------+---------------------+---------
 76fa69a4f44344528ad0eb506177898b | 127.0.186.126:44527 | HEALTHY

Unusual flags for Master:
                   Flag                   |                                                                                               Value                                                                                                |         Tags          |         Master
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
 ipki_ca_key_size                         | 768                                                                                                                                                                                                | experimental          | all 1 server(s) checked
 ipki_server_key_size                     | 768                                                                                                                                                                                                | experimental          | all 1 server(s) checked
 master_add_server_when_underreplicated   | true                                                                                                                                                                                               | hidden                | all 1 server(s) checked
 never_fsync                              | true                                                                                                                                                                                               | unsafe,advanced       | all 1 server(s) checked
 openssl_security_level_override          | 0                                                                                                                                                                                                  | unsafe,hidden         | all 1 server(s) checked
 raft_prepare_replacement_before_eviction | true                                                                                                                                                                                               | experimental,advanced | all 1 server(s) checked
 rpc_reuseport                            | true                                                                                                                                                                                               | experimental          | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports         | true                                                                                                                                                                                               | unsafe                | all 1 server(s) checked
 server_dump_info_format                  | pb                                                                                                                                                                                                 | hidden                | all 1 server(s) checked
 server_dump_info_path                    | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden                | all 1 server(s) checked
 tsk_num_rsa_bits                         | 512                                                                                                                                                                                                | experimental          | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |       Value        |         Master
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.0.186.84:46879 | all 1 server(s) checked
 time_source         | builtin            | all 1 server(s) checked

Tablet Server Summary
               UUID               |      Address       | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
 4b9385e4bb9048af824071b2309c83ba | 127.0.186.66:37013 | HEALTHY | <none>   |       0        |       0
 72cb331ade34427dbf3147f1f60f7c67 | 127.0.186.65:36365 | HEALTHY | <none>   |       1        |       0
 99e3ab4cbdbc49ce8ad92622b6e0e110 | 127.0.186.67:44489 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
                   Flag                   |                                                                                             Value                                                                                              |             Tags              |      Tablet Server
------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------+-------------------------
 ipki_server_key_size                     | 768                                                                                                                                                                                            | experimental                  | all 3 server(s) checked
 local_ip_for_outbound_sockets            | 127.0.186.65                                                                                                                                                                                   | experimental                  | 127.0.186.65:36365
 local_ip_for_outbound_sockets            | 127.0.186.66                                                                                                                                                                                   | experimental                  | 127.0.186.66:37013
 local_ip_for_outbound_sockets            | 127.0.186.67                                                                                                                                                                                   | experimental                  | 127.0.186.67:44489
 log_compression_codec                    | no_compression                                                                                                                                                                                 | experimental                  | all 3 server(s) checked
 log_max_segments_to_retain               | 3                                                                                                                                                                                              | experimental,runtime,advanced | all 3 server(s) checked
 log_target_replay_size_mb                | 1                                                                                                                                                                                              | experimental                  | all 3 server(s) checked
 maintenance_manager_polling_interval_ms  | 100                                                                                                                                                                                            | hidden                        | all 3 server(s) checked
 max_cell_size_bytes                      | 1000000                                                                                                                                                                                        | unsafe                        | all 3 server(s) checked
 never_fsync                              | true                                                                                                                                                                                           | unsafe,advanced               | all 3 server(s) checked
 openssl_security_level_override          | 0                                                                                                                                                                                              | unsafe,hidden                 | all 3 server(s) checked
 raft_prepare_replacement_before_eviction | true                                                                                                                                                                                           | experimental,advanced         | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports         | true                                                                                                                                                                                           | unsafe                        | all 3 server(s) checked
 server_dump_info_format                  | pb                                                                                                                                                                                             | hidden                        | all 3 server(s) checked
 server_dump_info_path                    | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden                        | 127.0.186.65:36365
 server_dump_info_path                    | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden                        | 127.0.186.66:37013
 server_dump_info_path                    | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_1.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden                        | 127.0.186.67:44489

Flags of checked categories for Tablet Server:
        Flag         |       Value        |      Tablet Server
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.0.186.84:46879 | all 3 server(s) checked
 time_source         | builtin            | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
   Name    | RF | Status  | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-----------+----+---------+---------------+---------+------------+------------------+-------------
 TestTable | 3  | HEALTHY | 1             | 1       | 0          | 0                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

OK
I20260812 08:02:52.328115   745 log_verifier.cc:126] Checking tablet df4dbd05f89d44fc8799d314a5dc6f27
I20260812 08:02:52.334666   745 log_verifier.cc:177] Verified matching terms for 16 ops in tablet df4dbd05f89d44fc8799d314a5dc6f27
I20260812 08:02:52.335052   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 17808
I20260812 08:02:52.351293   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 17962
I20260812 08:02:52.361703   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 18120
I20260812 08:02:52.371572   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 18434
2026-08-12T08:02:52Z chronyd exiting
[       OK ] ReplicaBehindWalGcThresholdITest.ReplicaReplacement/1 (22723 ms)
[ RUN      ] ReplicaBehindWalGcThresholdITest.ReplicaReplacement/2
I20260812 08:02:52.400182   745 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:02:52.400238   745 ts_itest-base.cc:116] --------------
I20260812 08:02:52.400247   745 ts_itest-base.cc:117] 3 tablet servers
I20260812 08:02:52.400257   745 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:02:52.400266   745 ts_itest-base.cc:119] --------------
2026-08-12T08:02:52Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:02:52Z Disabled control of system clock
I20260812 08:02:52.409586   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:41121
--webserver_interface=127.0.186.126
--webserver_port=0
--builtin_ntp_servers=127.0.186.84:39373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:41121
--raft_prepare_replacement_before_eviction=true
--master_add_server_when_underreplicated=false with env {}
W20260812 08:02:52.513051 18635 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:52.513286 18635 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:52.513342 18635 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:52.516845 18635 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:02:52.516908 18635 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:52.516937 18635 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:02:52.516959 18635 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:02:52.520773 18635 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:39373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--master_add_server_when_underreplicated=false
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:41121
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:41121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.18635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:52.521894 18635 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:52.523067 18635 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:52.528682 18644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:52.528678 18646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:52.528720 18642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:52.528995 18635 server_base.cc:1061] running on GCE node
I20260812 08:02:52.529517 18635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:52.530409 18635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:52.531586 18635 hybrid_clock.cc:648] HybridClock initialized: now 1786521772531567 us; error 30 us; skew 500 ppm
I20260812 08:02:52.533385 18635 webserver.cc:533] Webserver started at http://127.0.186.126:36313/ using document root <none> and password file <none>
I20260812 08:02:52.533977 18635 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:52.534035 18635 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:52.534245 18635 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:52.535753 18635 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "faba2ef34b794bfcb5ef62a91dadc899"
format_stamp: "Formatted at 2026-08-12 08:02:52 on dist-test-slave-49h3"
I20260812 08:02:52.536202 18635 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "faba2ef34b794bfcb5ef62a91dadc899"
format_stamp: "Formatted at 2026-08-12 08:02:52 on dist-test-slave-49h3"
I20260812 08:02:52.539426 18635 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:02:52.541421 18655 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:52.542404 18635 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:52.542506 18635 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "faba2ef34b794bfcb5ef62a91dadc899"
format_stamp: "Formatted at 2026-08-12 08:02:52 on dist-test-slave-49h3"
I20260812 08:02:52.542595 18635 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:52.552742 18635 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:52.553356 18635 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:52.553563 18635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:52.560497 18635 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:41121
I20260812 08:02:52.560528 18722 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:41121 every 8 connection(s)
I20260812 08:02:52.561534 18635 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:02:52.564355 18724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:52.564713   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 18635
I20260812 08:02:52.564812   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:02:52.569900 18724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899: Bootstrap starting.
I20260812 08:02:52.572144 18724 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:52.572923 18724 log.cc:826] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:52.574667 18724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899: No bootstrap required, opened a new log
I20260812 08:02:52.576996 18724 raft_consensus.cc:359] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faba2ef34b794bfcb5ef62a91dadc899" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41121 } }
I20260812 08:02:52.577172 18724 raft_consensus.cc:385] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:52.577212 18724 raft_consensus.cc:740] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faba2ef34b794bfcb5ef62a91dadc899, State: Initialized, Role: FOLLOWER
I20260812 08:02:52.577579 18724 consensus_queue.cc:260] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faba2ef34b794bfcb5ef62a91dadc899" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41121 } }
I20260812 08:02:52.577706 18724 raft_consensus.cc:399] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:52.577756 18724 raft_consensus.cc:493] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:52.577828 18724 raft_consensus.cc:3060] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:52.578732 18724 raft_consensus.cc:515] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faba2ef34b794bfcb5ef62a91dadc899" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41121 } }
I20260812 08:02:52.579049 18724 leader_election.cc:304] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: faba2ef34b794bfcb5ef62a91dadc899; no voters: 
I20260812 08:02:52.579291 18724 leader_election.cc:290] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:02:52.579424 18729 raft_consensus.cc:2804] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:52.579609 18729 raft_consensus.cc:697] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [term 1 LEADER]: Becoming Leader. State: Replica: faba2ef34b794bfcb5ef62a91dadc899, State: Running, Role: LEADER
I20260812 08:02:52.579945 18729 consensus_queue.cc:237] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faba2ef34b794bfcb5ef62a91dadc899" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41121 } }
I20260812 08:02:52.580315 18724 sys_catalog.cc:565] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:02:52.582585 18731 sys_catalog.cc:455] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [sys.catalog]: SysCatalogTable state changed. Reason: New leader faba2ef34b794bfcb5ef62a91dadc899. Latest consensus state: current_term: 1 leader_uuid: "faba2ef34b794bfcb5ef62a91dadc899" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faba2ef34b794bfcb5ef62a91dadc899" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41121 } } }
I20260812 08:02:52.582942 18731 sys_catalog.cc:458] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:52.582588 18730 sys_catalog.cc:455] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "faba2ef34b794bfcb5ef62a91dadc899" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faba2ef34b794bfcb5ef62a91dadc899" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41121 } } }
I20260812 08:02:52.583142 18730 sys_catalog.cc:458] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [sys.catalog]: This master's current role is: LEADER
W20260812 08:02:52.584895 18748 catalog_manager.cc:1594] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 08:02:52.584959 18748 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 08:02:52.585048 18749 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:02:52.586015 18749 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:02:52.591032 18749 catalog_manager.cc:1383] Generated new cluster ID: 38b8a2fceb2d434ba7df4750d631641e
I20260812 08:02:52.591109 18749 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:02:52.622840 18749 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:02:52.623761 18749 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:02:52.634024 18749 catalog_manager.cc:6092] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899: Generated new TSK 0
I20260812 08:02:52.634616 18749 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:02:52.639374   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:0
--local_ip_for_outbound_sockets=127.0.186.65
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:41121
--builtin_ntp_servers=127.0.186.84:39373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20260812 08:02:52.746897 18753 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:02:52.747152 18753 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:52.747191 18753 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:52.747264 18753 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:52.750614 18753 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:02:52.750674 18753 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:02:52.750720 18753 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:52.750799 18753 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:02:52.750833 18753 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:02:52.754712 18753 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:39373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:41121
--never_fsync=true
--heap_profile_path=/tmp/kudu.18753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:52.755793 18753 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:52.756939 18753 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:52.763092 18758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:52.763103 18759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:52.763094 18761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:52.763362 18753 server_base.cc:1061] running on GCE node
I20260812 08:02:52.763721 18753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:52.764245 18753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:52.765349 18753 hybrid_clock.cc:648] HybridClock initialized: now 1786521772765342 us; error 25 us; skew 500 ppm
I20260812 08:02:52.767002 18753 webserver.cc:533] Webserver started at http://127.0.186.65:38197/ using document root <none> and password file <none>
I20260812 08:02:52.767539 18753 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:52.767593 18753 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:52.767791 18753 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:52.769287 18753 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "2eada00db5434f33816ab72fa82e096f"
format_stamp: "Formatted at 2026-08-12 08:02:52 on dist-test-slave-49h3"
I20260812 08:02:52.769793 18753 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "2eada00db5434f33816ab72fa82e096f"
format_stamp: "Formatted at 2026-08-12 08:02:52 on dist-test-slave-49h3"
I20260812 08:02:52.772890 18753 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:02:52.774919 18768 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:52.775884 18753 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:02:52.775993 18753 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "2eada00db5434f33816ab72fa82e096f"
format_stamp: "Formatted at 2026-08-12 08:02:52 on dist-test-slave-49h3"
I20260812 08:02:52.776124 18753 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:52.790212 18753 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:52.790900 18753 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:52.791072 18753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:52.791636 18753 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:52.792689 18753 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:52.792747 18753 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:52.792802 18753 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:52.792827 18753 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:52.801723 18753 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:44541
I20260812 08:02:52.801751 18910 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:44541 every 8 connection(s)
I20260812 08:02:52.802701 18753 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:02:52.806078   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 18753
I20260812 08:02:52.806177   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:02:52.808313   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.66:0
--local_ip_for_outbound_sockets=127.0.186.66
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:41121
--builtin_ntp_servers=127.0.186.84:39373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20260812 08:02:52.815683 18911 heartbeater.cc:344] Connected to a master server at 127.0.186.126:41121
I20260812 08:02:52.815935 18911 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:52.816457 18911 heartbeater.cc:507] Master 127.0.186.126:41121 requested a full tablet report, sending...
I20260812 08:02:52.818032 18678 ts_manager.cc:194] Registered new tserver with Master: 2eada00db5434f33816ab72fa82e096f (127.0.186.65:44541)
I20260812 08:02:52.819167 18678 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.65:56025
W20260812 08:02:52.913288 18914 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:02:52.913533 18914 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:52.913570 18914 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:52.913631 18914 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:52.916796 18914 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:02:52.916848 18914 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:02:52.916882 18914 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:52.916951 18914 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:02:52.916988 18914 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.66
I20260812 08:02:52.920750 18914 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:39373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:41121
--never_fsync=true
--heap_profile_path=/tmp/kudu.18914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:52.921818 18914 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:52.922950 18914 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:52.929493 18920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:52.929466 18921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:52.929584 18914 server_base.cc:1061] running on GCE node
W20260812 08:02:52.929472 18923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:52.930058 18914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:52.930595 18914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:52.931746 18914 hybrid_clock.cc:648] HybridClock initialized: now 1786521772931733 us; error 31 us; skew 500 ppm
I20260812 08:02:52.933535 18914 webserver.cc:533] Webserver started at http://127.0.186.66:37705/ using document root <none> and password file <none>
I20260812 08:02:52.934103 18914 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:52.934157 18914 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:52.934330 18914 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:52.935899 18914 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "0f2354580c984e58afa894b599f5e9bc"
format_stamp: "Formatted at 2026-08-12 08:02:52 on dist-test-slave-49h3"
I20260812 08:02:52.936331 18914 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "0f2354580c984e58afa894b599f5e9bc"
format_stamp: "Formatted at 2026-08-12 08:02:52 on dist-test-slave-49h3"
I20260812 08:02:52.939530 18914 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.004s
I20260812 08:02:52.941555 18931 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:52.942497 18914 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:52.942593 18914 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "0f2354580c984e58afa894b599f5e9bc"
format_stamp: "Formatted at 2026-08-12 08:02:52 on dist-test-slave-49h3"
I20260812 08:02:52.942684 18914 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:52.961925 18914 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:52.962572 18914 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:52.962731 18914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:52.963295 18914 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:52.964332 18914 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:52.964380 18914 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:52.964434 18914 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:52.964460 18914 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:52.973299 18914 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.66:46511
I20260812 08:02:52.973332 19063 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.66:46511 every 8 connection(s)
I20260812 08:02:52.974339 18914 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:02:52.982592 19064 heartbeater.cc:344] Connected to a master server at 127.0.186.126:41121
I20260812 08:02:52.982828 19064 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:52.983294 19064 heartbeater.cc:507] Master 127.0.186.126:41121 requested a full tablet report, sending...
I20260812 08:02:52.984273 18678 ts_manager.cc:194] Registered new tserver with Master: 0f2354580c984e58afa894b599f5e9bc (127.0.186.66:46511)
I20260812 08:02:52.984510   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 18914
I20260812 08:02:52.984612   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:02:52.984866 18678 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.66:41371
I20260812 08:02:52.986644   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.67:0
--local_ip_for_outbound_sockets=127.0.186.67
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:41121
--builtin_ntp_servers=127.0.186.84:39373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20260812 08:02:53.090858 19068 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:02:53.091085 19068 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:53.091121 19068 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:53.091178 19068 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:53.094398 19068 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:02:53.094452 19068 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:02:53.094489 19068 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:53.094575 19068 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:02:53.094605 19068 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.67
I20260812 08:02:53.098405 19068 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:39373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:41121
--never_fsync=true
--heap_profile_path=/tmp/kudu.19068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:53.099479 19068 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:53.100670 19068 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:53.107066 19076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:53.107038 19075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:53.107167 19068 server_base.cc:1061] running on GCE node
W20260812 08:02:53.107048 19079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:53.107671 19068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:53.108208 19068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:53.109365 19068 hybrid_clock.cc:648] HybridClock initialized: now 1786521773109348 us; error 28 us; skew 500 ppm
I20260812 08:02:53.111068 19068 webserver.cc:533] Webserver started at http://127.0.186.67:46661/ using document root <none> and password file <none>
I20260812 08:02:53.111614 19068 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:53.111673 19068 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:53.111904 19068 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:53.113435 19068 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "72a34a2e9f8c43ee885e2e4785a6571f"
format_stamp: "Formatted at 2026-08-12 08:02:53 on dist-test-slave-49h3"
I20260812 08:02:53.113960 19068 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "72a34a2e9f8c43ee885e2e4785a6571f"
format_stamp: "Formatted at 2026-08-12 08:02:53 on dist-test-slave-49h3"
I20260812 08:02:53.117163 19068 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.001s
I20260812 08:02:53.119277 19085 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:53.120188 19068 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:53.120294 19068 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "72a34a2e9f8c43ee885e2e4785a6571f"
format_stamp: "Formatted at 2026-08-12 08:02:53 on dist-test-slave-49h3"
I20260812 08:02:53.120378 19068 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:53.132026 19068 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:53.132757 19068 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:53.132925 19068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:53.133620 19068 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:53.134878 19068 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:53.134933 19068 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:53.134992 19068 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:53.135020 19068 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:53.144572 19068 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.67:40581
I20260812 08:02:53.144590 19220 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.67:40581 every 8 connection(s)
I20260812 08:02:53.145627 19068 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:02:53.152607   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 19068
I20260812 08:02:53.152707   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:02:53.154263 19222 heartbeater.cc:344] Connected to a master server at 127.0.186.126:41121
I20260812 08:02:53.154485 19222 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:53.155023 19222 heartbeater.cc:507] Master 127.0.186.126:41121 requested a full tablet report, sending...
I20260812 08:02:53.155973 18678 ts_manager.cc:194] Registered new tserver with Master: 72a34a2e9f8c43ee885e2e4785a6571f (127.0.186.67:40581)
I20260812 08:02:53.156508 18678 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.67:52997
I20260812 08:02:53.166539   745 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:02:53.174554 18678 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53352:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
W20260812 08:02:53.177153 18678 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 08:02:53.192231 18825 tablet_service.cc:1511] Processing CreateTablet for tablet ce1eb828ad5045f09e23e60754424369 (DEFAULT_TABLE table=TestTable [id=3e8203b53f5d4808859a7ce09204b161]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:53.193295 18825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce1eb828ad5045f09e23e60754424369. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:53.192973 18988 tablet_service.cc:1511] Processing CreateTablet for tablet ce1eb828ad5045f09e23e60754424369 (DEFAULT_TABLE table=TestTable [id=3e8203b53f5d4808859a7ce09204b161]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:53.193099 19143 tablet_service.cc:1511] Processing CreateTablet for tablet ce1eb828ad5045f09e23e60754424369 (DEFAULT_TABLE table=TestTable [id=3e8203b53f5d4808859a7ce09204b161]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:53.193879 19143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce1eb828ad5045f09e23e60754424369. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:53.193874 18988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce1eb828ad5045f09e23e60754424369. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:53.198295 19243 tablet_bootstrap.cc:492] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f: Bootstrap starting.
I20260812 08:02:53.199384 19244 tablet_bootstrap.cc:492] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f: Bootstrap starting.
I20260812 08:02:53.200358 19243 tablet_bootstrap.cc:654] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:53.200636 19246 tablet_bootstrap.cc:492] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: Bootstrap starting.
I20260812 08:02:53.201100 19243 log.cc:826] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:53.202405 19246 tablet_bootstrap.cc:654] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:53.202514 19243 tablet_bootstrap.cc:492] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f: No bootstrap required, opened a new log
I20260812 08:02:53.202687 19243 ts_tablet_manager.cc:1403] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.000s
I20260812 08:02:53.203078 19246 log.cc:826] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:53.204119 19244 tablet_bootstrap.cc:654] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:53.204632 19246 tablet_bootstrap.cc:492] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: No bootstrap required, opened a new log
I20260812 08:02:53.204814 19246 ts_tablet_manager.cc:1403] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: Time spent bootstrapping tablet: real 0.004s	user 0.003s	sys 0.000s
I20260812 08:02:53.204885 19244 log.cc:826] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:53.205175 19243 raft_consensus.cc:359] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46511 } }
I20260812 08:02:53.205395 19243 raft_consensus.cc:385] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:53.205515 19243 raft_consensus.cc:740] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72a34a2e9f8c43ee885e2e4785a6571f, State: Initialized, Role: FOLLOWER
I20260812 08:02:53.205938 19243 consensus_queue.cc:260] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46511 } }
I20260812 08:02:53.206472 19244 tablet_bootstrap.cc:492] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f: No bootstrap required, opened a new log
I20260812 08:02:53.206549 19222 heartbeater.cc:499] Master 127.0.186.126:41121 was elected leader, sending a full tablet report...
I20260812 08:02:53.206754 19244 ts_tablet_manager.cc:1403] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f: Time spent bootstrapping tablet: real 0.008s	user 0.004s	sys 0.000s
I20260812 08:02:53.207809 19243 ts_tablet_manager.cc:1434] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 08:02:53.207767 19246 raft_consensus.cc:359] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46511 } }
I20260812 08:02:53.207990 19246 raft_consensus.cc:385] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:53.208071 19246 raft_consensus.cc:740] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f2354580c984e58afa894b599f5e9bc, State: Initialized, Role: FOLLOWER
I20260812 08:02:53.208487 19246 consensus_queue.cc:260] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46511 } }
I20260812 08:02:53.209112 19064 heartbeater.cc:499] Master 127.0.186.126:41121 was elected leader, sending a full tablet report...
I20260812 08:02:53.209292 19246 ts_tablet_manager.cc:1434] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:02:53.209798 19244 raft_consensus.cc:359] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46511 } }
I20260812 08:02:53.209936 19244 raft_consensus.cc:385] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:53.210021 19244 raft_consensus.cc:740] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2eada00db5434f33816ab72fa82e096f, State: Initialized, Role: FOLLOWER
I20260812 08:02:53.210405 19244 consensus_queue.cc:260] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46511 } }
I20260812 08:02:53.211236 19244 ts_tablet_manager.cc:1434] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260812 08:02:53.211596 18911 heartbeater.cc:499] Master 127.0.186.126:41121 was elected leader, sending a full tablet report...
W20260812 08:02:53.246532 19223 tablet.cc:2406] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:02:53.275615 19065 tablet.cc:2406] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:02:53.278767 19251 raft_consensus.cc:493] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:02:53.278939 19251 raft_consensus.cc:515] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46511 } }
I20260812 08:02:53.280728 19251 leader_election.cc:290] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2eada00db5434f33816ab72fa82e096f (127.0.186.65:44541), 72a34a2e9f8c43ee885e2e4785a6571f (127.0.186.67:40581)
I20260812 08:02:53.284124 18853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce1eb828ad5045f09e23e60754424369" candidate_uuid: "0f2354580c984e58afa894b599f5e9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2eada00db5434f33816ab72fa82e096f" is_pre_election: true
I20260812 08:02:53.284401 18853 raft_consensus.cc:2468] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f2354580c984e58afa894b599f5e9bc in term 0.
I20260812 08:02:53.284372 19169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce1eb828ad5045f09e23e60754424369" candidate_uuid: "0f2354580c984e58afa894b599f5e9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" is_pre_election: true
I20260812 08:02:53.284610 19169 raft_consensus.cc:2468] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f2354580c984e58afa894b599f5e9bc in term 0.
I20260812 08:02:53.284888 18932 leader_election.cc:304] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f2354580c984e58afa894b599f5e9bc, 2eada00db5434f33816ab72fa82e096f; no voters: 
I20260812 08:02:53.285130 19251 raft_consensus.cc:2804] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:02:53.285200 19251 raft_consensus.cc:493] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:02:53.285248 19251 raft_consensus.cc:3060] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:53.286110 19251 raft_consensus.cc:515] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46511 } }
I20260812 08:02:53.286442 19251 leader_election.cc:290] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [CANDIDATE]: Term 1 election: Requested vote from peers 2eada00db5434f33816ab72fa82e096f (127.0.186.65:44541), 72a34a2e9f8c43ee885e2e4785a6571f (127.0.186.67:40581)
I20260812 08:02:53.286765 18853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce1eb828ad5045f09e23e60754424369" candidate_uuid: "0f2354580c984e58afa894b599f5e9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2eada00db5434f33816ab72fa82e096f"
I20260812 08:02:53.286794 19169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce1eb828ad5045f09e23e60754424369" candidate_uuid: "0f2354580c984e58afa894b599f5e9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72a34a2e9f8c43ee885e2e4785a6571f"
I20260812 08:02:53.286871 18853 raft_consensus.cc:3060] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:53.286886 19169 raft_consensus.cc:3060] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:53.287667 19169 raft_consensus.cc:2468] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f2354580c984e58afa894b599f5e9bc in term 1.
I20260812 08:02:53.287961 18935 leader_election.cc:304] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f2354580c984e58afa894b599f5e9bc, 72a34a2e9f8c43ee885e2e4785a6571f; no voters: 
I20260812 08:02:53.288120 19251 raft_consensus.cc:2804] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:53.288136 18853 raft_consensus.cc:2468] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f2354580c984e58afa894b599f5e9bc in term 1.
I20260812 08:02:53.288318 19251 raft_consensus.cc:697] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [term 1 LEADER]: Becoming Leader. State: Replica: 0f2354580c984e58afa894b599f5e9bc, State: Running, Role: LEADER
I20260812 08:02:53.288667 19251 consensus_queue.cc:237] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46511 } }
I20260812 08:02:53.291688 18678 catalog_manager.cc:5719] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f2354580c984e58afa894b599f5e9bc (127.0.186.66). New cstate: current_term: 1 leader_uuid: "0f2354580c984e58afa894b599f5e9bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46511 } health_report { overall_health: HEALTHY } } }
W20260812 08:02:53.304319 18912 tablet.cc:2406] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:02:53.308209   745 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:02:53.309782   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 2eada00db5434f33816ab72fa82e096f to finish bootstrapping
I20260812 08:02:53.315868   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 0f2354580c984e58afa894b599f5e9bc to finish bootstrapping
I20260812 08:02:53.320921   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 72a34a2e9f8c43ee885e2e4785a6571f to finish bootstrapping
I20260812 08:02:53.335688   745 cluster_itest_util.cc:258] Not converged past 1 yet: 1.1 0.0 0.0
I20260812 08:02:53.438635   745 cluster_itest_util.cc:258] Not converged past 1 yet: 1.1 0.0 0.0
I20260812 08:02:53.641439   745 cluster_itest_util.cc:258] Not converged past 1 yet: 1.1 0.0 0.0
I20260812 08:02:53.774437 19251 consensus_queue.cc:1048] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:02:53.780141 19255 consensus_queue.cc:1048] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:02:53.944923   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 18914
I20260812 08:02:55.280427 19270 raft_consensus.cc:493] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0f2354580c984e58afa894b599f5e9bc)
I20260812 08:02:55.280588 19270 raft_consensus.cc:515] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46511 } }
I20260812 08:02:55.281791 19270 leader_election.cc:290] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 72a34a2e9f8c43ee885e2e4785a6571f (127.0.186.67:40581), 0f2354580c984e58afa894b599f5e9bc (127.0.186.66:46511)
I20260812 08:02:55.281960 19271 raft_consensus.cc:493] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0f2354580c984e58afa894b599f5e9bc)
I20260812 08:02:55.282397 19271 raft_consensus.cc:515] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46511 } }
W20260812 08:02:55.282601 18773 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.66:46511: connect: Connection refused (error 111)
W20260812 08:02:55.283497 18773 leader_election.cc:336] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0f2354580c984e58afa894b599f5e9bc (127.0.186.66:46511): Network error: Client connection negotiation failed: client connection to 127.0.186.66:46511: connect: Connection refused (error 111)
I20260812 08:02:55.283967 19271 leader_election.cc:290] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2eada00db5434f33816ab72fa82e096f (127.0.186.65:44541), 0f2354580c984e58afa894b599f5e9bc (127.0.186.66:46511)
W20260812 08:02:55.284353 19090 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.66:46511: connect: Connection refused (error 111)
W20260812 08:02:55.285212 19090 leader_election.cc:336] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0f2354580c984e58afa894b599f5e9bc (127.0.186.66:46511): Network error: Client connection negotiation failed: client connection to 127.0.186.66:46511: connect: Connection refused (error 111)
I20260812 08:02:55.286167 19169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce1eb828ad5045f09e23e60754424369" candidate_uuid: "2eada00db5434f33816ab72fa82e096f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" is_pre_election: true
I20260812 08:02:55.286278 19169 raft_consensus.cc:2468] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2eada00db5434f33816ab72fa82e096f in term 1.
I20260812 08:02:55.286612 18772 leader_election.cc:304] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2eada00db5434f33816ab72fa82e096f, 72a34a2e9f8c43ee885e2e4785a6571f; no voters: 0f2354580c984e58afa894b599f5e9bc
I20260812 08:02:55.286819 19270 raft_consensus.cc:2804] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 08:02:55.286878 19270 raft_consensus.cc:493] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0f2354580c984e58afa894b599f5e9bc)
I20260812 08:02:55.286916 19270 raft_consensus.cc:3060] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:02:55.287210 18853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce1eb828ad5045f09e23e60754424369" candidate_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2eada00db5434f33816ab72fa82e096f" is_pre_election: true
I20260812 08:02:55.287810 19270 raft_consensus.cc:515] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46511 } }
I20260812 08:02:55.288030 18853 raft_consensus.cc:2393] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 72a34a2e9f8c43ee885e2e4785a6571f in current term 2: Already voted for candidate 2eada00db5434f33816ab72fa82e096f in this term.
I20260812 08:02:55.288170 19270 leader_election.cc:290] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [CANDIDATE]: Term 2 election: Requested vote from peers 72a34a2e9f8c43ee885e2e4785a6571f (127.0.186.67:40581), 0f2354580c984e58afa894b599f5e9bc (127.0.186.66:46511)
I20260812 08:02:55.288442 19087 leader_election.cc:304] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72a34a2e9f8c43ee885e2e4785a6571f; no voters: 0f2354580c984e58afa894b599f5e9bc, 2eada00db5434f33816ab72fa82e096f
I20260812 08:02:55.288486 19169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce1eb828ad5045f09e23e60754424369" candidate_uuid: "2eada00db5434f33816ab72fa82e096f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72a34a2e9f8c43ee885e2e4785a6571f"
I20260812 08:02:55.288621 19169 raft_consensus.cc:3060] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f [term 1 FOLLOWER]: Advancing to term 2
W20260812 08:02:55.289095 18773 leader_election.cc:336] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0f2354580c984e58afa894b599f5e9bc (127.0.186.66:46511): Network error: Client connection negotiation failed: client connection to 127.0.186.66:46511: connect: Connection refused (error 111)
I20260812 08:02:55.289438 19169 raft_consensus.cc:2468] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2eada00db5434f33816ab72fa82e096f in term 2.
I20260812 08:02:55.289548 19271 raft_consensus.cc:2749] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 08:02:55.289783 18772 leader_election.cc:304] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2eada00db5434f33816ab72fa82e096f, 72a34a2e9f8c43ee885e2e4785a6571f; no voters: 0f2354580c984e58afa894b599f5e9bc
I20260812 08:02:55.289958 19270 raft_consensus.cc:2804] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:02:55.290163 19270 raft_consensus.cc:697] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [term 2 LEADER]: Becoming Leader. State: Replica: 2eada00db5434f33816ab72fa82e096f, State: Running, Role: LEADER
I20260812 08:02:55.290453 19270 consensus_queue.cc:237] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46511 } }
I20260812 08:02:55.292819 18678 catalog_manager.cc:5719] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f reported cstate change: term changed from 1 to 2, leader changed from 0f2354580c984e58afa894b599f5e9bc (127.0.186.66) to 2eada00db5434f33816ab72fa82e096f (127.0.186.65). New cstate: current_term: 2 leader_uuid: "2eada00db5434f33816ab72fa82e096f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46511 } health_report { overall_health: UNKNOWN } } }
I20260812 08:02:55.491643   745 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
I20260812 08:02:55.491991   745 raft_consensus-itest-base.cc:268] Waiting until we've written at least 4MB...
I20260812 08:02:55.506268 19169 raft_consensus.cc:1275] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f [term 2 FOLLOWER]: Refusing update from remote peer 2eada00db5434f33816ab72fa82e096f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260812 08:02:55.506846 19282 consensus_queue.cc:1048] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [LEADER]: Connected to new peer: Peer: permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260812 08:02:55.507040 18773 consensus_peers.cc:597] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f -> Peer 0f2354580c984e58afa894b599f5e9bc (127.0.186.66:46511): Couldn't send request to peer 0f2354580c984e58afa894b599f5e9bc. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.66:46511: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 08:02:55.514946 19302 mvcc.cc:204] Tried to move back new op lower bound from 7317593192475176960 to 7317593191591940096. Current Snapshot: MvccSnapshot[applied={T|T < 7317593192467005440}]
I20260812 08:02:55.515801 19303 mvcc.cc:204] Tried to move back new op lower bound from 7317593192475176960 to 7317593191591940096. Current Snapshot: MvccSnapshot[applied={T|T < 7317593192467005440}]
I20260812 08:02:55.554919 19223 maintenance_manager.cc:419] P 72a34a2e9f8c43ee885e2e4785a6571f: Scheduling FlushMRSOp(ce1eb828ad5045f09e23e60754424369): 2629807 bytes log retention, and flush 4038921 bytes memory
I20260812 08:02:55.601016   745 raft_consensus-itest-base.cc:274] Waiting for log GC on 2eada00db5434f33816ab72fa82e096f
I20260812 08:02:55.607447 19093 maintenance_manager.cc:643] P 72a34a2e9f8c43ee885e2e4785a6571f: FlushMRSOp(ce1eb828ad5045f09e23e60754424369) complete. Timing: real 0.052s	user 0.040s	sys 0.004s Metrics: {"bytes_written":135869,"cfile_init":1,"compiler_manager_pool.queue_time_us":233,"dirs.queue_time_us":3972,"dirs.run_cpu_time_us":193,"dirs.run_wall_time_us":833,"drs_written":1,"lbm_read_time_us":73,"lbm_reads_lt_1ms":4,"lbm_write_time_us":688,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":22,"spinlock_wait_cycles":133504,"thread_start_us":259,"threads_started":2}
I20260812 08:02:55.608342 19223 maintenance_manager.cc:419] P 72a34a2e9f8c43ee885e2e4785a6571f: Scheduling LogGCOp(ce1eb828ad5045f09e23e60754424369): free 2629807 bytes of WAL
I20260812 08:02:55.608605 19093 log_reader.cc:385] T ce1eb828ad5045f09e23e60754424369: removed 3 log segments from log reader
I20260812 08:02:55.608690 19093 log.cc:1079] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/ce1eb828ad5045f09e23e60754424369/wal-000000001 (ops 1-9)
I20260812 08:02:55.608748 19093 log.cc:1079] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/ce1eb828ad5045f09e23e60754424369/wal-000000002 (ops 10-16)
I20260812 08:02:55.608830 19093 log.cc:1079] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/ce1eb828ad5045f09e23e60754424369/wal-000000003 (ops 17-22)
I20260812 08:02:55.609807 19093 maintenance_manager.cc:643] P 72a34a2e9f8c43ee885e2e4785a6571f: LogGCOp(ce1eb828ad5045f09e23e60754424369) complete. Timing: real 0.001s	user 0.000s	sys 0.001s Metrics: {}
I20260812 08:02:55.610080 19223 maintenance_manager.cc:419] P 72a34a2e9f8c43ee885e2e4785a6571f: Scheduling FlushMRSOp(ce1eb828ad5045f09e23e60754424369): 1709307 bytes log retention, and flush 2998537 bytes memory
I20260812 08:02:55.612673 18912 maintenance_manager.cc:419] P 2eada00db5434f33816ab72fa82e096f: Scheduling FlushMRSOp(ce1eb828ad5045f09e23e60754424369): 4602350 bytes log retention, and flush 6119689 bytes memory
I20260812 08:02:55.632915 19093 maintenance_manager.cc:643] P 72a34a2e9f8c43ee885e2e4785a6571f: FlushMRSOp(ce1eb828ad5045f09e23e60754424369) complete. Timing: real 0.023s	user 0.018s	sys 0.002s Metrics: {"bytes_written":135803,"cfile_init":1,"dirs.queue_time_us":54,"dirs.run_cpu_time_us":181,"dirs.run_wall_time_us":1303,"drs_written":1,"lbm_read_time_us":49,"lbm_reads_lt_1ms":4,"lbm_write_time_us":572,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":15}
I20260812 08:02:55.634016 19223 maintenance_manager.cc:419] P 72a34a2e9f8c43ee885e2e4785a6571f: Scheduling UndoDeltaBlockGCOp(ce1eb828ad5045f09e23e60754424369): 689 bytes on disk
I20260812 08:02:55.634786 19093 maintenance_manager.cc:643] P 72a34a2e9f8c43ee885e2e4785a6571f: UndoDeltaBlockGCOp(ce1eb828ad5045f09e23e60754424369) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":102,"lbm_reads_lt_1ms":8}
I20260812 08:02:55.635245 19223 maintenance_manager.cc:419] P 72a34a2e9f8c43ee885e2e4785a6571f: Scheduling CompactRowSetsOp(ce1eb828ad5045f09e23e60754424369): perf score=0.884207
I20260812 08:02:55.671509 18774 maintenance_manager.cc:643] P 2eada00db5434f33816ab72fa82e096f: FlushMRSOp(ce1eb828ad5045f09e23e60754424369) complete. Timing: real 0.058s	user 0.056s	sys 0.000s Metrics: {"bytes_written":136001,"cfile_init":1,"compiler_manager_pool.queue_time_us":3849,"drs_written":1,"lbm_read_time_us":66,"lbm_reads_lt_1ms":4,"lbm_write_time_us":596,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":37,"thread_start_us":273,"threads_started":2}
I20260812 08:02:55.672626 18912 maintenance_manager.cc:419] P 2eada00db5434f33816ab72fa82e096f: Scheduling LogGCOp(ce1eb828ad5045f09e23e60754424369): free 2761400 bytes of WAL
I20260812 08:02:55.672935 18774 log_reader.cc:385] T ce1eb828ad5045f09e23e60754424369: removed 3 log segments from log reader
I20260812 08:02:55.673035 18774 log.cc:1079] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/ce1eb828ad5045f09e23e60754424369/wal-000000001 (ops 1-9)
I20260812 08:02:55.673134 18774 log.cc:1079] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/ce1eb828ad5045f09e23e60754424369/wal-000000002 (ops 10-16)
I20260812 08:02:55.673213 18774 log.cc:1079] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/ce1eb828ad5045f09e23e60754424369/wal-000000003 (ops 17-23)
I20260812 08:02:55.674223 18774 maintenance_manager.cc:643] P 2eada00db5434f33816ab72fa82e096f: LogGCOp(ce1eb828ad5045f09e23e60754424369) complete. Timing: real 0.001s	user 0.000s	sys 0.001s Metrics: {}
I20260812 08:02:55.674685 18912 maintenance_manager.cc:419] P 2eada00db5434f33816ab72fa82e096f: Scheduling UndoDeltaBlockGCOp(ce1eb828ad5045f09e23e60754424369): 553 bytes on disk
I20260812 08:02:55.675292 18774 maintenance_manager.cc:643] P 2eada00db5434f33816ab72fa82e096f: UndoDeltaBlockGCOp(ce1eb828ad5045f09e23e60754424369) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":66,"lbm_reads_lt_1ms":4}
I20260812 08:02:55.682878   745 raft_consensus-itest-base.cc:279] Log GC complete on 2eada00db5434f33816ab72fa82e096f
I20260812 08:02:55.684278 19093 maintenance_manager.cc:643] P 72a34a2e9f8c43ee885e2e4785a6571f: CompactRowSetsOp(ce1eb828ad5045f09e23e60754424369) complete. Timing: real 0.049s	user 0.048s	sys 0.000s Metrics: {"bytes_written":136001,"cfile_cache_hit":10,"cfile_cache_hit_bytes":262392,"cfile_cache_miss":26,"cfile_cache_miss_bytes":264318,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":770,"dirs.run_cpu_time_us":168,"dirs.run_wall_time_us":1375,"drs_written":1,"lbm_read_time_us":596,"lbm_reads_lt_1ms":46,"lbm_write_time_us":467,"lbm_writes_lt_1ms":23,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":37,"thread_start_us":371,"threads_started":5}
I20260812 08:02:55.684805 19223 maintenance_manager.cc:419] P 72a34a2e9f8c43ee885e2e4785a6571f: Scheduling UndoDeltaBlockGCOp(ce1eb828ad5045f09e23e60754424369): 553 bytes on disk
I20260812 08:02:55.685209 19093 maintenance_manager.cc:643] P 72a34a2e9f8c43ee885e2e4785a6571f: UndoDeltaBlockGCOp(ce1eb828ad5045f09e23e60754424369) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":50,"lbm_reads_lt_1ms":4}
I20260812 08:02:56.064404 19314 consensus_queue.cc:753] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [LEADER]: The logs necessary to catch up peer 0f2354580c984e58afa894b599f5e9bc have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..32: Segment 1 which contained index 2 has been GCed)
I20260812 08:02:56.064525 19314 consensus_queue.cc:579] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [LEADER]: The logs necessary to catch up peer 0f2354580c984e58afa894b599f5e9bc have been garbage collected. The replica will never be able to catch up
I20260812 08:02:56.067708 18853 consensus_queue.cc:237] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 39, Committed index: 39, Last appended: 2.39, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } }
I20260812 08:02:56.068980 19169 raft_consensus.cc:1275] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f [term 2 FOLLOWER]: Refusing update from remote peer 2eada00db5434f33816ab72fa82e096f: Log matching property violated. Preceding OpId in replica: term: 2 index: 39. Preceding OpId from leader: term: 2 index: 40. (index mismatch)
I20260812 08:02:56.069416 19314 consensus_queue.cc:1048] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [LEADER]: Connected to new peer: Peer: permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 39, Time since last communication: 0.000s
I20260812 08:02:56.071156 19318 raft_consensus.cc:2955] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [term 2 LEADER]: Committing config change with OpId 2.40: config changed from index -1 to 40, VOTER 0f2354580c984e58afa894b599f5e9bc (127.0.186.66) evicted. New config: { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } } }
I20260812 08:02:56.071656 19169 raft_consensus.cc:2955] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f [term 2 FOLLOWER]: Committing config change with OpId 2.40: config changed from index -1 to 40, VOTER 0f2354580c984e58afa894b599f5e9bc (127.0.186.66) evicted. New config: { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } } }
I20260812 08:02:56.072966 18656 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet ce1eb828ad5045f09e23e60754424369 with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:02:56.074069 18676 catalog_manager.cc:5719] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f reported cstate change: config changed from index -1 to 40, VOTER 0f2354580c984e58afa894b599f5e9bc (127.0.186.66) evicted. New cstate: current_term: 2 leader_uuid: "2eada00db5434f33816ab72fa82e096f" committed_config { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } health_report { overall_health: HEALTHY } } }
I20260812 08:02:56.077297 18912 maintenance_manager.cc:419] P 2eada00db5434f33816ab72fa82e096f: Scheduling LogGCOp(ce1eb828ad5045f09e23e60754424369): free 1840950 bytes of WAL
W20260812 08:02:56.077350 18659 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.66:46511: connect: Connection refused (error 111)
I20260812 08:02:56.077538 18774 log_reader.cc:385] T ce1eb828ad5045f09e23e60754424369: removed 2 log segments from log reader
I20260812 08:02:56.077584 18774 log.cc:1079] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/ce1eb828ad5045f09e23e60754424369/wal-000000004 (ops 24-30)
I20260812 08:02:56.077620 18774 log.cc:1079] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/ce1eb828ad5045f09e23e60754424369/wal-000000005 (ops 31-37)
I20260812 08:02:56.078127 18774 maintenance_manager.cc:643] P 2eada00db5434f33816ab72fa82e096f: LogGCOp(ce1eb828ad5045f09e23e60754424369) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
W20260812 08:02:56.078361 18659 catalog_manager.cc:4777] TS 0f2354580c984e58afa894b599f5e9bc (127.0.186.66:46511): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet ce1eb828ad5045f09e23e60754424369: Network error: Client connection negotiation failed: client connection to 127.0.186.66:46511: connect: Connection refused (error 111)
I20260812 08:02:56.087173 19223 maintenance_manager.cc:419] P 72a34a2e9f8c43ee885e2e4785a6571f: Scheduling LogGCOp(ce1eb828ad5045f09e23e60754424369): free 1709307 bytes of WAL
I20260812 08:02:56.087445 19093 log_reader.cc:385] T ce1eb828ad5045f09e23e60754424369: removed 2 log segments from log reader
I20260812 08:02:56.087494 19093 log.cc:1079] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/ce1eb828ad5045f09e23e60754424369/wal-000000004 (ops 23-28)
I20260812 08:02:56.087524 19093 log.cc:1079] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/ce1eb828ad5045f09e23e60754424369/wal-000000005 (ops 29-35)
I20260812 08:02:56.087983 19093 maintenance_manager.cc:643] P 72a34a2e9f8c43ee885e2e4785a6571f: LogGCOp(ce1eb828ad5045f09e23e60754424369) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
W20260812 08:02:57.289918 18659 catalog_manager.cc:4777] TS 0f2354580c984e58afa894b599f5e9bc (127.0.186.66:46511): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet ce1eb828ad5045f09e23e60754424369: Network error: Client connection negotiation failed: client connection to 127.0.186.66:46511: connect: Connection refused (error 111)
I20260812 08:02:57.684264   745 raft_consensus-itest-base.cc:295] Servers converged with original term 2
I20260812 08:02:57.684333   745 raft_consensus-itest-base.cc:303] Restarting 0f2354580c984e58afa894b599f5e9bc
I20260812 08:02:57.684672   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.66:46511
--local_ip_for_outbound_sockets=127.0.186.66
--tserver_master_addrs=127.0.186.126:41121
--webserver_port=37705
--webserver_interface=127.0.186.66
--builtin_ntp_servers=127.0.186.84:39373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20260812 08:02:57.790979 19342 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:02:57.791232 19342 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:57.791256 19342 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:57.791316 19342 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:57.794591 19342 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:02:57.794653 19342 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:02:57.794696 19342 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:57.794773 19342 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:02:57.794808 19342 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.66
I20260812 08:02:57.798648 19342 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:39373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.66:46511
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.186.66
--webserver_port=37705
--tserver_master_addrs=127.0.186.126:41121
--never_fsync=true
--heap_profile_path=/tmp/kudu.19342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:57.799758 19342 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:57.801003 19342 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:57.808370 19348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:57.808485 19342 server_base.cc:1061] running on GCE node
W20260812 08:02:57.808528 19349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:57.808399 19352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:57.809005 19342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:57.809597 19342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:57.810711 19342 hybrid_clock.cc:648] HybridClock initialized: now 1786521777810687 us; error 31 us; skew 500 ppm
I20260812 08:02:57.812302 19342 webserver.cc:533] Webserver started at http://127.0.186.66:37705/ using document root <none> and password file <none>
I20260812 08:02:57.812866 19342 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:57.812939 19342 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:57.816181 19342 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:57.818002 19361 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:57.818953 19342 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:57.819058 19342 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "0f2354580c984e58afa894b599f5e9bc"
format_stamp: "Formatted at 2026-08-12 08:02:52 on dist-test-slave-49h3"
I20260812 08:02:57.819444 19342 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:57.835214 19342 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:57.835870 19342 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:57.836042 19342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:57.836591 19342 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:57.837872 19368 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:02:57.841140 19342 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:02:57.841197 19342 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:02:57.841248 19342 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:02:57.842700 19342 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:02:57.842754 19342 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:02:57.843094 19368 tablet_bootstrap.cc:492] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: Bootstrap starting.
I20260812 08:02:57.852583 19368 log.cc:826] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:57.854022 19342 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.66:46511
I20260812 08:02:57.854080 19510 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.66:46511 every 8 connection(s)
I20260812 08:02:57.855115 19342 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:02:57.855643 19368 tablet_bootstrap.cc:492] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:02:57.856226 19368 tablet_bootstrap.cc:492] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: Bootstrap complete.
I20260812 08:02:57.856592 19368 ts_tablet_manager.cc:1403] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: Time spent bootstrapping tablet: real 0.014s	user 0.003s	sys 0.009s
I20260812 08:02:57.860049 19368 raft_consensus.cc:359] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46511 } }
I20260812 08:02:57.860505 19368 raft_consensus.cc:740] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f2354580c984e58afa894b599f5e9bc, State: Initialized, Role: FOLLOWER
I20260812 08:02:57.860948 19368 consensus_queue.cc:260] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46511 } }
I20260812 08:02:57.861615   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 19342
I20260812 08:02:57.861882 19368 ts_tablet_manager.cc:1434] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.001s
I20260812 08:02:57.866420   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 18635
W20260812 08:02:57.877375 19511 heartbeater.cc:646] Failed to heartbeat to 127.0.186.126:41121 (0 consecutive failures): Network error: Failed to ping master at 127.0.186.126:41121: recv got EOF from 127.0.186.126:41121 (error 108)
I20260812 08:02:57.877727   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:41121
--webserver_interface=127.0.186.126
--webserver_port=36313
--builtin_ntp_servers=127.0.186.84:39373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:41121
--raft_prepare_replacement_before_eviction=true
--master_add_server_when_underreplicated=false
--master_add_server_when_underreplicated=true with env {}
W20260812 08:02:57.878774 19511 heartbeater.cc:412] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 08:02:57.988363 19516 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:57.988595 19516 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:57.988639 19516 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:57.991926 19516 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:02:57.991984 19516 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:57.992004 19516 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:02:57.992020 19516 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:02:57.996037 19516 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:39373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:41121
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:41121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=36313
--never_fsync=true
--heap_profile_path=/tmp/kudu.19516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:57.997088 19516 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:57.998227 19516 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:58.003504 19522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:58.003605 19528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:58.003504 19525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:58.003809 19516 server_base.cc:1061] running on GCE node
I20260812 08:02:58.004277 19516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:58.005174 19516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:58.006316 19516 hybrid_clock.cc:648] HybridClock initialized: now 1786521778006303 us; error 28 us; skew 500 ppm
I20260812 08:02:58.007987 19516 webserver.cc:533] Webserver started at http://127.0.186.126:36313/ using document root <none> and password file <none>
I20260812 08:02:58.008538 19516 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:58.008605 19516 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:58.011919 19516 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:58.013896 19534 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:58.014866 19516 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:58.014973 19516 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "faba2ef34b794bfcb5ef62a91dadc899"
format_stamp: "Formatted at 2026-08-12 08:02:52 on dist-test-slave-49h3"
I20260812 08:02:58.015354 19516 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:58.025180 19516 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:58.025816 19516 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:58.025981 19516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:58.032768 19516 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:41121
I20260812 08:02:58.032784 19602 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:41121 every 8 connection(s)
I20260812 08:02:58.033905 19516 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:02:58.037827 19603 sys_catalog.cc:263] Verifying existing consensus state
I20260812 08:02:58.039274 19603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899: Bootstrap starting.
I20260812 08:02:58.043946   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 19516
I20260812 08:02:58.054901 19603 log.cc:826] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:58.063410 19603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:02:58.063822 19603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899: Bootstrap complete.
I20260812 08:02:58.067308 19603 raft_consensus.cc:359] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faba2ef34b794bfcb5ef62a91dadc899" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41121 } }
I20260812 08:02:58.067628 19603 raft_consensus.cc:740] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: faba2ef34b794bfcb5ef62a91dadc899, State: Initialized, Role: FOLLOWER
I20260812 08:02:58.068020 19603 consensus_queue.cc:260] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faba2ef34b794bfcb5ef62a91dadc899" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41121 } }
I20260812 08:02:58.068137 19603 raft_consensus.cc:399] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:58.068184 19603 raft_consensus.cc:493] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:58.068262 19603 raft_consensus.cc:3060] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:02:58.069942 19603 raft_consensus.cc:515] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faba2ef34b794bfcb5ef62a91dadc899" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41121 } }
I20260812 08:02:58.070255 19603 leader_election.cc:304] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: faba2ef34b794bfcb5ef62a91dadc899; no voters: 
I20260812 08:02:58.070551 19603 leader_election.cc:290] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [CANDIDATE]: Term 2 election: Requested vote from peers 
I20260812 08:02:58.070680 19610 raft_consensus.cc:2804] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:02:58.070891 19610 raft_consensus.cc:697] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [term 2 LEADER]: Becoming Leader. State: Replica: faba2ef34b794bfcb5ef62a91dadc899, State: Running, Role: LEADER
I20260812 08:02:58.071202 19610 consensus_queue.cc:237] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faba2ef34b794bfcb5ef62a91dadc899" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41121 } }
I20260812 08:02:58.071620 19603 sys_catalog.cc:565] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:02:58.072671 19612 sys_catalog.cc:455] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [sys.catalog]: SysCatalogTable state changed. Reason: New leader faba2ef34b794bfcb5ef62a91dadc899. Latest consensus state: current_term: 2 leader_uuid: "faba2ef34b794bfcb5ef62a91dadc899" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faba2ef34b794bfcb5ef62a91dadc899" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41121 } } }
I20260812 08:02:58.072818 19612 sys_catalog.cc:458] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:58.073050 19611 sys_catalog.cc:455] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "faba2ef34b794bfcb5ef62a91dadc899" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faba2ef34b794bfcb5ef62a91dadc899" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 41121 } } }
I20260812 08:02:58.073137 19611 sys_catalog.cc:458] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899 [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:58.073284 19622 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:02:58.076239 19622 catalog_manager.cc:679] Loaded metadata for table TestTable [id=3e8203b53f5d4808859a7ce09204b161]
I20260812 08:02:58.077486 19622 tablet_loader.cc:96] loaded metadata for tablet ce1eb828ad5045f09e23e60754424369 (table TestTable [id=3e8203b53f5d4808859a7ce09204b161])
I20260812 08:02:58.077632 19622 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:02:58.078277 19622 catalog_manager.cc:1295] Loaded cluster ID: 38b8a2fceb2d434ba7df4750d631641e
I20260812 08:02:58.078326 19622 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:02:58.080264 19622 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:02:58.080936 19622 catalog_manager.cc:6103] T 00000000000000000000000000000000 P faba2ef34b794bfcb5ef62a91dadc899: Loaded TSK: 0
I20260812 08:02:58.081809 19622 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:02:58.086917 19558 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" instance_seqno: 1786521773141806) as {username='slave'} at 127.0.186.67:54995; Asking this server to re-register.
I20260812 08:02:58.087651 19222 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:58.087888 19222 heartbeater.cc:507] Master 127.0.186.126:41121 requested a full tablet report, sending...
I20260812 08:02:58.088323 19558 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "2eada00db5434f33816ab72fa82e096f" instance_seqno: 1786521772799283) as {username='slave'} at 127.0.186.65:38711; Asking this server to re-register.
I20260812 08:02:58.088922 18911 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:58.089102 18911 heartbeater.cc:507] Master 127.0.186.126:41121 requested a full tablet report, sending...
I20260812 08:02:58.089286 19558 ts_manager.cc:194] Registered new tserver with Master: 72a34a2e9f8c43ee885e2e4785a6571f (127.0.186.67:40581)
I20260812 08:02:58.090243 19557 ts_manager.cc:194] Registered new tserver with Master: 2eada00db5434f33816ab72fa82e096f (127.0.186.65:44541)
W20260812 08:02:58.092851 19557 catalog_manager.cc:5397] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet ce1eb828ad5045f09e23e60754424369 with cas_config_opid_index 40: no extra replica candidate found for tablet ce1eb828ad5045f09e23e60754424369 (table TestTable [id=3e8203b53f5d4808859a7ce09204b161]): Not found: could not select location for extra replica: not enough tablet servers to satisfy replica placement policy: the total number of registered tablet servers (2) does not allow for adding an extra replica; consider bringing up more to have at least 4 tablet servers up and running
I20260812 08:02:58.883399 19511 heartbeater.cc:344] Connected to a master server at 127.0.186.126:41121
I20260812 08:02:58.883611 19511 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:58.884207 19511 heartbeater.cc:507] Master 127.0.186.126:41121 requested a full tablet report, sending...
I20260812 08:02:58.886142 19557 ts_manager.cc:194] Registered new tserver with Master: 0f2354580c984e58afa894b599f5e9bc (127.0.186.66:46511)
I20260812 08:02:58.887982 19557 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.66:54011
I20260812 08:02:58.893056 19421 tablet_service.cc:1558] Processing DeleteTablet for tablet ce1eb828ad5045f09e23e60754424369 with delete_type TABLET_DATA_TOMBSTONED (Replica has no consensus available (current committed config index is 40)) from {username='slave'} at 127.0.0.1:35432
I20260812 08:02:58.893591 19640 tablet_replica.cc:333] stopping tablet replica
I20260812 08:02:58.893850 19640 raft_consensus.cc:2243] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 08:02:58.894057 19640 raft_consensus.cc:2272] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 08:02:58.894779 19640 ts_tablet_manager.cc:1916] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 08:02:58.898149 19640 ts_tablet_manager.cc:1929] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.1
I20260812 08:02:58.898242 19640 log.cc:1199] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: Deleting WAL directory at /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/wals/ce1eb828ad5045f09e23e60754424369
I20260812 08:02:58.899335 19538 catalog_manager.cc:5050] TS 0f2354580c984e58afa894b599f5e9bc (127.0.186.66:46511): tablet ce1eb828ad5045f09e23e60754424369 (table TestTable [id=3e8203b53f5d4808859a7ce09204b161]) successfully deleted
I20260812 08:02:59.317821 18853 consensus_queue.cc:237] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 2.40, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 46511 } attrs { promote: true } }
I20260812 08:02:59.319486 19169 raft_consensus.cc:1275] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f [term 2 FOLLOWER]: Refusing update from remote peer 2eada00db5434f33816ab72fa82e096f: Log matching property violated. Preceding OpId in replica: term: 2 index: 40. Preceding OpId from leader: term: 2 index: 41. (index mismatch)
I20260812 08:02:59.320008 19644 consensus_queue.cc:1048] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [LEADER]: Connected to new peer: Peer: permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 41, Last known committed idx: 40, Time since last communication: 0.000s
I20260812 08:02:59.322088 19641 raft_consensus.cc:2955] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [term 2 LEADER]: Committing config change with OpId 2.41: config changed from index 40 to 41, NON_VOTER 0f2354580c984e58afa894b599f5e9bc (127.0.186.66) added. New config: { opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 46511 } attrs { promote: true } } }
I20260812 08:02:59.322629 19169 raft_consensus.cc:2955] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f [term 2 FOLLOWER]: Committing config change with OpId 2.41: config changed from index 40 to 41, NON_VOTER 0f2354580c984e58afa894b599f5e9bc (127.0.186.66) added. New config: { opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 46511 } attrs { promote: true } } }
W20260812 08:02:59.324450 18773 consensus_peers.cc:597] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f -> Peer 0f2354580c984e58afa894b599f5e9bc (127.0.186.66:46511): Couldn't send request to peer 0f2354580c984e58afa894b599f5e9bc. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:02:59.324685 19535 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet ce1eb828ad5045f09e23e60754424369 with cas_config_opid_index 40: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 7)
I20260812 08:02:59.324885 19557 catalog_manager.cc:5719] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f reported cstate change: config changed from index 40 to 41, NON_VOTER 0f2354580c984e58afa894b599f5e9bc (127.0.186.66) added. New cstate: current_term: 2 leader_uuid: "2eada00db5434f33816ab72fa82e096f" committed_config { opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 46511 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260812 08:02:59.823987 19655 ts_tablet_manager.cc:933] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: Initiating tablet copy from peer 2eada00db5434f33816ab72fa82e096f (127.0.186.65:44541)
I20260812 08:02:59.824589 19655 tablet_copy_client.cc:323] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: tablet copy: Beginning tablet copy session from remote peer at address 127.0.186.65:44541
I20260812 08:02:59.828353 18876 tablet_copy_service.cc:140] P 2eada00db5434f33816ab72fa82e096f: Received BeginTabletCopySession request for tablet ce1eb828ad5045f09e23e60754424369 from peer 0f2354580c984e58afa894b599f5e9bc ({username='slave'} at 127.0.186.66:36937)
I20260812 08:02:59.828502 18876 tablet_copy_service.cc:161] P 2eada00db5434f33816ab72fa82e096f: Beginning new tablet copy session on tablet ce1eb828ad5045f09e23e60754424369 from peer 0f2354580c984e58afa894b599f5e9bc at {username='slave'} at 127.0.186.66:36937: session id = 0f2354580c984e58afa894b599f5e9bc-ce1eb828ad5045f09e23e60754424369
I20260812 08:02:59.830001 18876 tablet_copy_source_session.cc:215] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f: Tablet Copy: opened 5 blocks and 1 log segments
I20260812 08:02:59.831871 19655 ts_tablet_manager.cc:1916] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260812 08:02:59.833777 19655 ts_tablet_manager.cc:1929] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.1
I20260812 08:02:59.834134 19655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce1eb828ad5045f09e23e60754424369. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:59.835364 19655 tablet_copy_client.cc:806] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: tablet copy: Starting download of 5 data blocks...
I20260812 08:02:59.842468 19655 tablet_copy_client.cc:670] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: tablet copy: Starting download of 1 WAL segments...
I20260812 08:02:59.845170 19655 tablet_copy_client.cc:538] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 08:02:59.846915 19655 tablet_bootstrap.cc:492] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: Bootstrap starting.
I20260812 08:02:59.856169 19655 tablet_bootstrap.cc:492] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=2} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=1. Pending: 0 replicates
I20260812 08:02:59.856403 19655 tablet_bootstrap.cc:492] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: Bootstrap complete.
I20260812 08:02:59.856611 19655 ts_tablet_manager.cc:1403] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: Time spent bootstrapping tablet: real 0.010s	user 0.008s	sys 0.000s
I20260812 08:02:59.857133 19655 raft_consensus.cc:359] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 46511 } attrs { promote: true } }
I20260812 08:02:59.857235 19655 raft_consensus.cc:740] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 0f2354580c984e58afa894b599f5e9bc, State: Initialized, Role: LEARNER
I20260812 08:02:59.857343 19655 consensus_queue.cc:260] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 41, Last appended: 2.41, Last appended by leader: 41, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 46511 } attrs { promote: true } }
I20260812 08:02:59.857781 19655 ts_tablet_manager.cc:1434] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc: Time spent starting tablet: real 0.001s	user 0.004s	sys 0.000s
I20260812 08:02:59.857865 19511 heartbeater.cc:499] Master 127.0.186.126:41121 was elected leader, sending a full tablet report...
I20260812 08:02:59.858542 18876 tablet_copy_service.cc:342] P 2eada00db5434f33816ab72fa82e096f: Request end of tablet copy session 0f2354580c984e58afa894b599f5e9bc-ce1eb828ad5045f09e23e60754424369 received from {username='slave'} at 127.0.186.66:36937
I20260812 08:02:59.858659 18876 tablet_copy_service.cc:434] P 2eada00db5434f33816ab72fa82e096f: ending tablet copy session 0f2354580c984e58afa894b599f5e9bc-ce1eb828ad5045f09e23e60754424369 on tablet ce1eb828ad5045f09e23e60754424369 with peer 0f2354580c984e58afa894b599f5e9bc
I20260812 08:02:59.861827 19512 maintenance_manager.cc:419] P 0f2354580c984e58afa894b599f5e9bc: Scheduling UndoDeltaBlockGCOp(ce1eb828ad5045f09e23e60754424369): 553 bytes on disk
I20260812 08:02:59.862655 19366 maintenance_manager.cc:643] P 0f2354580c984e58afa894b599f5e9bc: UndoDeltaBlockGCOp(ce1eb828ad5045f09e23e60754424369) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":49,"lbm_reads_lt_1ms":4}
I20260812 08:03:00.273981 19453 raft_consensus.cc:1217] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [term 2 LEARNER]: Deduplicated request from leader. Original: 2.40->[2.41-2.41]   Dedup: 2.41->[]
I20260812 08:03:00.709684 19662 raft_consensus.cc:1064] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f: attempting to promote NON_VOTER 0f2354580c984e58afa894b599f5e9bc to VOTER
I20260812 08:03:00.710166 19662 consensus_queue.cc:237] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 41, Committed index: 41, Last appended: 2.41, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46511 } attrs { promote: false } }
I20260812 08:03:00.711519 19169 raft_consensus.cc:1275] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f [term 2 FOLLOWER]: Refusing update from remote peer 2eada00db5434f33816ab72fa82e096f: Log matching property violated. Preceding OpId in replica: term: 2 index: 41. Preceding OpId from leader: term: 2 index: 42. (index mismatch)
I20260812 08:03:00.711591 19453 raft_consensus.cc:1275] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [term 2 LEARNER]: Refusing update from remote peer 2eada00db5434f33816ab72fa82e096f: Log matching property violated. Preceding OpId in replica: term: 2 index: 41. Preceding OpId from leader: term: 2 index: 42. (index mismatch)
I20260812 08:03:00.711972 19663 consensus_queue.cc:1048] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [LEADER]: Connected to new peer: Peer: permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 42, Last known committed idx: 41, Time since last communication: 0.000s
I20260812 08:03:00.712177 19662 consensus_queue.cc:1048] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46511 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 42, Last known committed idx: 41, Time since last communication: 0.000s
I20260812 08:03:00.713994 19663 raft_consensus.cc:2955] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f [term 2 LEADER]: Committing config change with OpId 2.42: config changed from index 41 to 42, 0f2354580c984e58afa894b599f5e9bc (127.0.186.66) changed from NON_VOTER to VOTER. New config: { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46511 } attrs { promote: false } } }
I20260812 08:03:00.714497 19169 raft_consensus.cc:2955] T ce1eb828ad5045f09e23e60754424369 P 72a34a2e9f8c43ee885e2e4785a6571f [term 2 FOLLOWER]: Committing config change with OpId 2.42: config changed from index 41 to 42, 0f2354580c984e58afa894b599f5e9bc (127.0.186.66) changed from NON_VOTER to VOTER. New config: { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46511 } attrs { promote: false } } }
I20260812 08:03:00.715375 19453 raft_consensus.cc:2955] T ce1eb828ad5045f09e23e60754424369 P 0f2354580c984e58afa894b599f5e9bc [term 2 FOLLOWER]: Committing config change with OpId 2.42: config changed from index 41 to 42, 0f2354580c984e58afa894b599f5e9bc (127.0.186.66) changed from NON_VOTER to VOTER. New config: { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46511 } attrs { promote: false } } }
I20260812 08:03:00.716637 19558 catalog_manager.cc:5719] T ce1eb828ad5045f09e23e60754424369 P 2eada00db5434f33816ab72fa82e096f reported cstate change: config changed from index 41 to 42, 0f2354580c984e58afa894b599f5e9bc (127.0.186.66) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "2eada00db5434f33816ab72fa82e096f" committed_config { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "2eada00db5434f33816ab72fa82e096f" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72a34a2e9f8c43ee885e2e4785a6571f" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 40581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f2354580c984e58afa894b599f5e9bc" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 46511 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 08:03:00.792217 18825 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260812 08:03:00.792342 19421 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:03:00.792966 19143 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |       Address       | Status
----------------------------------+---------------------+---------
 faba2ef34b794bfcb5ef62a91dadc899 | 127.0.186.126:41121 | HEALTHY

Unusual flags for Master:
                   Flag                   |                                                                                               Value                                                                                                |         Tags          |         Master
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
 ipki_ca_key_size                         | 768                                                                                                                                                                                                | experimental          | all 1 server(s) checked
 ipki_server_key_size                     | 768                                                                                                                                                                                                | experimental          | all 1 server(s) checked
 master_add_server_when_underreplicated   | true                                                                                                                                                                                               | hidden                | all 1 server(s) checked
 never_fsync                              | true                                                                                                                                                                                               | unsafe,advanced       | all 1 server(s) checked
 openssl_security_level_override          | 0                                                                                                                                                                                                  | unsafe,hidden         | all 1 server(s) checked
 raft_prepare_replacement_before_eviction | true                                                                                                                                                                                               | experimental,advanced | all 1 server(s) checked
 rpc_reuseport                            | true                                                                                                                                                                                               | experimental          | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports         | true                                                                                                                                                                                               | unsafe                | all 1 server(s) checked
 server_dump_info_format                  | pb                                                                                                                                                                                                 | hidden                | all 1 server(s) checked
 server_dump_info_path                    | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden                | all 1 server(s) checked
 tsk_num_rsa_bits                         | 512                                                                                                                                                                                                | experimental          | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |       Value        |         Master
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.0.186.84:39373 | all 1 server(s) checked
 time_source         | builtin            | all 1 server(s) checked

Tablet Server Summary
               UUID               |      Address       | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
 0f2354580c984e58afa894b599f5e9bc | 127.0.186.66:46511 | HEALTHY | <none>   |       0        |       0
 2eada00db5434f33816ab72fa82e096f | 127.0.186.65:44541 | HEALTHY | <none>   |       1        |       0
 72a34a2e9f8c43ee885e2e4785a6571f | 127.0.186.67:40581 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
                   Flag                   |                                                                                             Value                                                                                              |             Tags              |      Tablet Server
------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------+-------------------------
 ipki_server_key_size                     | 768                                                                                                                                                                                            | experimental                  | all 3 server(s) checked
 local_ip_for_outbound_sockets            | 127.0.186.65                                                                                                                                                                                   | experimental                  | 127.0.186.65:44541
 local_ip_for_outbound_sockets            | 127.0.186.66                                                                                                                                                                                   | experimental                  | 127.0.186.66:46511
 local_ip_for_outbound_sockets            | 127.0.186.67                                                                                                                                                                                   | experimental                  | 127.0.186.67:40581
 log_compression_codec                    | no_compression                                                                                                                                                                                 | experimental                  | all 3 server(s) checked
 log_max_segments_to_retain               | 3                                                                                                                                                                                              | experimental,runtime,advanced | all 3 server(s) checked
 log_target_replay_size_mb                | 1                                                                                                                                                                                              | experimental                  | all 3 server(s) checked
 maintenance_manager_polling_interval_ms  | 100                                                                                                                                                                                            | hidden                        | all 3 server(s) checked
 max_cell_size_bytes                      | 1000000                                                                                                                                                                                        | unsafe                        | all 3 server(s) checked
 never_fsync                              | true                                                                                                                                                                                           | unsafe,advanced               | all 3 server(s) checked
 openssl_security_level_override          | 0                                                                                                                                                                                              | unsafe,hidden                 | all 3 server(s) checked
 raft_prepare_replacement_before_eviction | true                                                                                                                                                                                           | experimental,advanced         | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports         | true                                                                                                                                                                                           | unsafe                        | all 3 server(s) checked
 server_dump_info_format                  | pb                                                                                                                                                                                             | hidden                        | all 3 server(s) checked
 server_dump_info_path                    | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden                        | 127.0.186.65:44541
 server_dump_info_path                    | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden                        | 127.0.186.66:46511
 server_dump_info_path                    | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_2.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden                        | 127.0.186.67:40581

Flags of checked categories for Tablet Server:
        Flag         |       Value        |      Tablet Server
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.0.186.84:39373 | all 3 server(s) checked
 time_source         | builtin            | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
   Name    | RF | Status  | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-----------+----+---------+---------------+---------+------------+------------------+-------------
 TestTable | 3  | HEALTHY | 1             | 1       | 0          | 0                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

OK
I20260812 08:03:00.821143   745 log_verifier.cc:126] Checking tablet ce1eb828ad5045f09e23e60754424369
I20260812 08:03:00.826639   745 log_verifier.cc:177] Verified matching terms for 9 ops in tablet ce1eb828ad5045f09e23e60754424369
I20260812 08:03:00.827301   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 18753
I20260812 08:03:00.848448   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 19342
I20260812 08:03:00.857147   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 19068
I20260812 08:03:00.866403   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 19516
2026-08-12T08:03:00Z chronyd exiting
[       OK ] ReplicaBehindWalGcThresholdITest.ReplicaReplacement/2 (8496 ms)
[ RUN      ] ReplicaBehindWalGcThresholdITest.ReplicaReplacement/3
I20260812 08:03:00.897197   745 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:03:00.897258   745 ts_itest-base.cc:116] --------------
I20260812 08:03:00.897269   745 ts_itest-base.cc:117] 3 tablet servers
I20260812 08:03:00.897279   745 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:03:00.897289   745 ts_itest-base.cc:119] --------------
2026-08-12T08:03:00Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:03:00Z Disabled control of system clock
I20260812 08:03:00.907205   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:35865
--webserver_interface=127.0.186.126
--webserver_port=0
--builtin_ntp_servers=127.0.186.84:35847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:35865
--raft_prepare_replacement_before_eviction=true
--master_add_server_when_underreplicated=false with env {}
W20260812 08:03:01.010551 19715 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:01.010776 19715 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:01.010833 19715 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:01.014120 19715 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:03:01.014175 19715 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:01.014204 19715 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:03:01.014235 19715 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:03:01.018003 19715 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:35847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--master_add_server_when_underreplicated=false
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:35865
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:35865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.19715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:01.019071 19715 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:01.020182 19715 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:01.025588 19729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:01.025625 19715 server_base.cc:1061] running on GCE node
W20260812 08:03:01.025655 19725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:01.025588 19724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:01.026314 19715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:01.027177 19715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:01.028335 19715 hybrid_clock.cc:648] HybridClock initialized: now 1786521781028320 us; error 31 us; skew 500 ppm
I20260812 08:03:01.030086 19715 webserver.cc:533] Webserver started at http://127.0.186.126:45259/ using document root <none> and password file <none>
I20260812 08:03:01.030653 19715 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:01.030712 19715 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:01.030901 19715 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:01.032449 19715 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "5dd9251f7ef54785a91610de46ec0292"
format_stamp: "Formatted at 2026-08-12 08:03:01 on dist-test-slave-49h3"
I20260812 08:03:01.032922 19715 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "5dd9251f7ef54785a91610de46ec0292"
format_stamp: "Formatted at 2026-08-12 08:03:01 on dist-test-slave-49h3"
I20260812 08:03:01.036350 19715 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:03:01.038568 19736 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:01.039558 19715 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:03:01.039673 19715 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "5dd9251f7ef54785a91610de46ec0292"
format_stamp: "Formatted at 2026-08-12 08:03:01 on dist-test-slave-49h3"
I20260812 08:03:01.039764 19715 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:01.071833 19715 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:01.072607 19715 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:01.072794 19715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:01.080144 19801 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:35865 every 8 connection(s)
I20260812 08:03:01.080161 19715 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:35865
I20260812 08:03:01.081167 19715 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:03:01.082415   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 19715
I20260812 08:03:01.082542   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:03:01.084563 19802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:01.089840 19802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292: Bootstrap starting.
I20260812 08:03:01.092099 19802 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:01.092736 19802 log.cc:826] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:01.094321 19802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292: No bootstrap required, opened a new log
I20260812 08:03:01.096683 19802 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dd9251f7ef54785a91610de46ec0292" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 35865 } }
I20260812 08:03:01.096872 19802 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:01.096912 19802 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dd9251f7ef54785a91610de46ec0292, State: Initialized, Role: FOLLOWER
I20260812 08:03:01.097244 19802 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dd9251f7ef54785a91610de46ec0292" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 35865 } }
I20260812 08:03:01.097344 19802 raft_consensus.cc:399] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:03:01.097460 19802 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:03:01.097560 19802 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:01.098263 19802 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dd9251f7ef54785a91610de46ec0292" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 35865 } }
I20260812 08:03:01.098552 19802 leader_election.cc:304] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5dd9251f7ef54785a91610de46ec0292; no voters: 
I20260812 08:03:01.098788 19802 leader_election.cc:290] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:03:01.098901 19807 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:01.099117 19807 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [term 1 LEADER]: Becoming Leader. State: Replica: 5dd9251f7ef54785a91610de46ec0292, State: Running, Role: LEADER
I20260812 08:03:01.099431 19807 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dd9251f7ef54785a91610de46ec0292" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 35865 } }
I20260812 08:03:01.099927 19802 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:01.100515 19810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5dd9251f7ef54785a91610de46ec0292. Latest consensus state: current_term: 1 leader_uuid: "5dd9251f7ef54785a91610de46ec0292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dd9251f7ef54785a91610de46ec0292" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 35865 } } }
I20260812 08:03:01.100646 19810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:01.100948 19808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5dd9251f7ef54785a91610de46ec0292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dd9251f7ef54785a91610de46ec0292" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 35865 } } }
I20260812 08:03:01.101054 19808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:01.101572 19815 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:03:01.104769 19815 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:03:01.109709 19815 catalog_manager.cc:1383] Generated new cluster ID: fa5013677d9c4b299c4f2ffe6809c261
I20260812 08:03:01.109784 19815 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:03:01.123457 19815 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:03:01.124287 19815 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:03:01.134384 19815 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292: Generated new TSK 0
I20260812 08:03:01.134955 19815 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:03:01.140477   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:0
--local_ip_for_outbound_sockets=127.0.186.65
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:35865
--builtin_ntp_servers=127.0.186.84:35847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20260812 08:03:01.247144 19832 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:03:01.247414 19832 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:01.247458 19832 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:01.247520 19832 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:01.250872 19832 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:03:01.250936 19832 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:03:01.250983 19832 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:01.251072 19832 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:03:01.251106 19832 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:03:01.255100 19832 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:35847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:35865
--never_fsync=true
--heap_profile_path=/tmp/kudu.19832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:01.256294 19832 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:01.257510 19832 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:01.263871 19838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:01.263875 19837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:01.263914 19840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:01.264109 19832 server_base.cc:1061] running on GCE node
I20260812 08:03:01.264511 19832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:01.265039 19832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:01.266181 19832 hybrid_clock.cc:648] HybridClock initialized: now 1786521781266164 us; error 29 us; skew 500 ppm
I20260812 08:03:01.267804 19832 webserver.cc:533] Webserver started at http://127.0.186.65:46515/ using document root <none> and password file <none>
I20260812 08:03:01.268361 19832 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:01.268417 19832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:01.268610 19832 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:01.270201 19832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "4285c138ac1d46fcb5810461505f8639"
format_stamp: "Formatted at 2026-08-12 08:03:01 on dist-test-slave-49h3"
I20260812 08:03:01.270666 19832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "4285c138ac1d46fcb5810461505f8639"
format_stamp: "Formatted at 2026-08-12 08:03:01 on dist-test-slave-49h3"
I20260812 08:03:01.273878 19832 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.002s
I20260812 08:03:01.275900 19847 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:01.276835 19832 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:01.276942 19832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "4285c138ac1d46fcb5810461505f8639"
format_stamp: "Formatted at 2026-08-12 08:03:01 on dist-test-slave-49h3"
I20260812 08:03:01.277029 19832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:01.292841 19832 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:01.293501 19832 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:01.293664 19832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:01.294229 19832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:01.295252 19832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:01.295303 19832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:01.295356 19832 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:01.295382 19832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:01.304230 19832 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:41261
I20260812 08:03:01.304255 19982 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:41261 every 8 connection(s)
I20260812 08:03:01.305145 19832 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:03:01.307504   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 19832
I20260812 08:03:01.307631   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:03:01.310130   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.66:0
--local_ip_for_outbound_sockets=127.0.186.66
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:35865
--builtin_ntp_servers=127.0.186.84:35847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20260812 08:03:01.314383 19983 heartbeater.cc:344] Connected to a master server at 127.0.186.126:35865
I20260812 08:03:01.314627 19983 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:01.315171 19983 heartbeater.cc:507] Master 127.0.186.126:35865 requested a full tablet report, sending...
I20260812 08:03:01.316865 19759 ts_manager.cc:194] Registered new tserver with Master: 4285c138ac1d46fcb5810461505f8639 (127.0.186.65:41261)
I20260812 08:03:01.318540 19759 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.65:36881
W20260812 08:03:01.416972 19987 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:03:01.417186 19987 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:01.417212 19987 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:01.417250 19987 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:01.420399 19987 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:03:01.420447 19987 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:03:01.420478 19987 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:01.420540 19987 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:03:01.420565 19987 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.66
I20260812 08:03:01.424256 19987 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:35847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:35865
--never_fsync=true
--heap_profile_path=/tmp/kudu.19987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:01.425227 19987 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:01.426314 19987 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:01.432392 19993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:01.432410 19992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:01.432505 19995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:01.432595 19987 server_base.cc:1061] running on GCE node
I20260812 08:03:01.433012 19987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:01.433576 19987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:01.434713 19987 hybrid_clock.cc:648] HybridClock initialized: now 1786521781434706 us; error 30 us; skew 500 ppm
I20260812 08:03:01.436261 19987 webserver.cc:533] Webserver started at http://127.0.186.66:34375/ using document root <none> and password file <none>
I20260812 08:03:01.436791 19987 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:01.436846 19987 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:01.437032 19987 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:01.438583 19987 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "073614a77a994b8fbed8f8cd6b7927de"
format_stamp: "Formatted at 2026-08-12 08:03:01 on dist-test-slave-49h3"
I20260812 08:03:01.439033 19987 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "073614a77a994b8fbed8f8cd6b7927de"
format_stamp: "Formatted at 2026-08-12 08:03:01 on dist-test-slave-49h3"
I20260812 08:03:01.442183 19987 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:03:01.444252 20002 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:01.445184 19987 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:01.445295 19987 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "073614a77a994b8fbed8f8cd6b7927de"
format_stamp: "Formatted at 2026-08-12 08:03:01 on dist-test-slave-49h3"
I20260812 08:03:01.445381 19987 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:01.457770 19987 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:01.458360 19987 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:01.458518 19987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:01.459055 19987 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:01.460031 19987 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:01.460080 19987 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:01.460132 19987 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:01.460161 19987 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:01.468916 19987 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.66:38247
I20260812 08:03:01.468951 20140 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.66:38247 every 8 connection(s)
I20260812 08:03:01.469875 19987 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:03:01.477260   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 19987
I20260812 08:03:01.477368   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:03:01.478152 20141 heartbeater.cc:344] Connected to a master server at 127.0.186.126:35865
I20260812 08:03:01.478375 20141 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:01.478868 20141 heartbeater.cc:507] Master 127.0.186.126:35865 requested a full tablet report, sending...
I20260812 08:03:01.479533   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.67:0
--local_ip_for_outbound_sockets=127.0.186.67
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:35865
--builtin_ntp_servers=127.0.186.84:35847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20260812 08:03:01.480729 19759 ts_manager.cc:194] Registered new tserver with Master: 073614a77a994b8fbed8f8cd6b7927de (127.0.186.66:38247)
I20260812 08:03:01.481280 19759 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.66:56869
W20260812 08:03:01.585196 20144 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:03:01.585433 20144 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:01.585492 20144 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:01.585554 20144 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:01.588783 20144 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:03:01.588835 20144 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:03:01.588868 20144 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:01.588941 20144 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:03:01.588976 20144 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.67
I20260812 08:03:01.592789 20144 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:35847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:35865
--never_fsync=true
--heap_profile_path=/tmp/kudu.20144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:01.593951 20144 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:01.595052 20144 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:01.601306 20153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:01.601325 20151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:01.601331 20150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:01.601608 20144 server_base.cc:1061] running on GCE node
I20260812 08:03:01.601943 20144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:01.602437 20144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:01.603574 20144 hybrid_clock.cc:648] HybridClock initialized: now 1786521781603560 us; error 27 us; skew 500 ppm
I20260812 08:03:01.605130 20144 webserver.cc:533] Webserver started at http://127.0.186.67:42807/ using document root <none> and password file <none>
I20260812 08:03:01.605753 20144 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:01.605809 20144 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:01.605974 20144 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:01.607467 20144 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "7dabac9590204b268f758f8f24ed87be"
format_stamp: "Formatted at 2026-08-12 08:03:01 on dist-test-slave-49h3"
I20260812 08:03:01.607899 20144 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "7dabac9590204b268f758f8f24ed87be"
format_stamp: "Formatted at 2026-08-12 08:03:01 on dist-test-slave-49h3"
I20260812 08:03:01.611114 20144 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.003s
I20260812 08:03:01.613174 20159 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:01.614161 20144 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 08:03:01.614274 20144 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "7dabac9590204b268f758f8f24ed87be"
format_stamp: "Formatted at 2026-08-12 08:03:01 on dist-test-slave-49h3"
I20260812 08:03:01.614362 20144 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:01.650334 20144 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:01.651079 20144 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:01.651268 20144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:01.651875 20144 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:01.652945 20144 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:01.652995 20144 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:01.653051 20144 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:01.653075 20144 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:01.661626 20144 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.67:39869
I20260812 08:03:01.661679 20293 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.67:39869 every 8 connection(s)
I20260812 08:03:01.662565 20144 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:03:01.669566   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 20144
I20260812 08:03:01.669708   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:03:01.670722 20294 heartbeater.cc:344] Connected to a master server at 127.0.186.126:35865
I20260812 08:03:01.671017 20294 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:01.671593 20294 heartbeater.cc:507] Master 127.0.186.126:35865 requested a full tablet report, sending...
I20260812 08:03:01.672636 19759 ts_manager.cc:194] Registered new tserver with Master: 7dabac9590204b268f758f8f24ed87be (127.0.186.67:39869)
I20260812 08:03:01.673169 19759 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.67:33165
I20260812 08:03:01.684033   745 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:03:01.692864 19759 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:32944:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
W20260812 08:03:01.695370 19759 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 08:03:01.712019 19906 tablet_service.cc:1511] Processing CreateTablet for tablet 39f77f897dab4a528691252eb9a48c03 (DEFAULT_TABLE table=TestTable [id=9bf92388afda40eeb62d1598dbc607d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:01.712054 20218 tablet_service.cc:1511] Processing CreateTablet for tablet 39f77f897dab4a528691252eb9a48c03 (DEFAULT_TABLE table=TestTable [id=9bf92388afda40eeb62d1598dbc607d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:01.712177 20062 tablet_service.cc:1511] Processing CreateTablet for tablet 39f77f897dab4a528691252eb9a48c03 (DEFAULT_TABLE table=TestTable [id=9bf92388afda40eeb62d1598dbc607d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:01.712980 19906 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39f77f897dab4a528691252eb9a48c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:01.712997 20062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39f77f897dab4a528691252eb9a48c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:01.712997 20218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39f77f897dab4a528691252eb9a48c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:01.718425 20312 tablet_bootstrap.cc:492] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639: Bootstrap starting.
I20260812 08:03:01.718962 20313 tablet_bootstrap.cc:492] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: Bootstrap starting.
I20260812 08:03:01.719345 20314 tablet_bootstrap.cc:492] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be: Bootstrap starting.
I20260812 08:03:01.720515 20312 tablet_bootstrap.cc:654] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:01.721105 20313 tablet_bootstrap.cc:654] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:01.721207 20314 tablet_bootstrap.cc:654] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:01.721232 20312 log.cc:826] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:01.721956 20313 log.cc:826] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:01.721956 20314 log.cc:826] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:01.722958 20312 tablet_bootstrap.cc:492] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639: No bootstrap required, opened a new log
I20260812 08:03:01.723166 20312 ts_tablet_manager.cc:1403] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.000s
I20260812 08:03:01.723711 20314 tablet_bootstrap.cc:492] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be: No bootstrap required, opened a new log
I20260812 08:03:01.723872 20314 ts_tablet_manager.cc:1403] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be: Time spent bootstrapping tablet: real 0.005s	user 0.000s	sys 0.003s
I20260812 08:03:01.723905 20313 tablet_bootstrap.cc:492] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: No bootstrap required, opened a new log
I20260812 08:03:01.724077 20313 ts_tablet_manager.cc:1403] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: Time spent bootstrapping tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 08:03:01.726152 20314 raft_consensus.cc:359] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } } peers { permanent_uuid: "073614a77a994b8fbed8f8cd6b7927de" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 38247 } }
I20260812 08:03:01.726328 20314 raft_consensus.cc:385] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:01.726411 20314 raft_consensus.cc:740] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dabac9590204b268f758f8f24ed87be, State: Initialized, Role: FOLLOWER
I20260812 08:03:01.726307 20312 raft_consensus.cc:359] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } } peers { permanent_uuid: "073614a77a994b8fbed8f8cd6b7927de" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 38247 } }
I20260812 08:03:01.726505 20312 raft_consensus.cc:385] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:01.726565 20312 raft_consensus.cc:740] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4285c138ac1d46fcb5810461505f8639, State: Initialized, Role: FOLLOWER
I20260812 08:03:01.726891 20314 consensus_queue.cc:260] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } } peers { permanent_uuid: "073614a77a994b8fbed8f8cd6b7927de" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 38247 } }
I20260812 08:03:01.726979 20312 consensus_queue.cc:260] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } } peers { permanent_uuid: "073614a77a994b8fbed8f8cd6b7927de" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 38247 } }
I20260812 08:03:01.726997 20313 raft_consensus.cc:359] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } } peers { permanent_uuid: "073614a77a994b8fbed8f8cd6b7927de" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 38247 } }
I20260812 08:03:01.727188 20313 raft_consensus.cc:385] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:01.727293 20313 raft_consensus.cc:740] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073614a77a994b8fbed8f8cd6b7927de, State: Initialized, Role: FOLLOWER
I20260812 08:03:01.727566 20294 heartbeater.cc:499] Master 127.0.186.126:35865 was elected leader, sending a full tablet report...
I20260812 08:03:01.727753 20314 ts_tablet_manager.cc:1434] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.001s
I20260812 08:03:01.727826 20312 ts_tablet_manager.cc:1434] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 08:03:01.728173 19983 heartbeater.cc:499] Master 127.0.186.126:35865 was elected leader, sending a full tablet report...
I20260812 08:03:01.728433 20313 consensus_queue.cc:260] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } } peers { permanent_uuid: "073614a77a994b8fbed8f8cd6b7927de" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 38247 } }
I20260812 08:03:01.729244 20313 ts_tablet_manager.cc:1434] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:03:01.729570 20141 heartbeater.cc:499] Master 127.0.186.126:35865 was elected leader, sending a full tablet report...
W20260812 08:03:01.763713 20295 tablet.cc:2406] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:03:01.771126 20142 tablet.cc:2406] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:03:01.805374 20321 raft_consensus.cc:493] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:01.805605 20321 raft_consensus.cc:515] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } } peers { permanent_uuid: "073614a77a994b8fbed8f8cd6b7927de" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 38247 } }
I20260812 08:03:01.806731 20321 leader_election.cc:290] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dabac9590204b268f758f8f24ed87be (127.0.186.67:39869), 073614a77a994b8fbed8f8cd6b7927de (127.0.186.66:38247)
W20260812 08:03:01.806766 19984 tablet.cc:2406] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:03:01.810781 20243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39f77f897dab4a528691252eb9a48c03" candidate_uuid: "4285c138ac1d46fcb5810461505f8639" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dabac9590204b268f758f8f24ed87be" is_pre_election: true
I20260812 08:03:01.811100 20243 raft_consensus.cc:2468] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4285c138ac1d46fcb5810461505f8639 in term 0.
I20260812 08:03:01.811192 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39f77f897dab4a528691252eb9a48c03" candidate_uuid: "4285c138ac1d46fcb5810461505f8639" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073614a77a994b8fbed8f8cd6b7927de" is_pre_election: true
I20260812 08:03:01.811473 20088 raft_consensus.cc:2468] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4285c138ac1d46fcb5810461505f8639 in term 0.
I20260812 08:03:01.811616 19851 leader_election.cc:304] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4285c138ac1d46fcb5810461505f8639, 7dabac9590204b268f758f8f24ed87be; no voters: 
I20260812 08:03:01.811897 20321 raft_consensus.cc:2804] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:03:01.811968 20321 raft_consensus.cc:493] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:03:01.812000 20321 raft_consensus.cc:3060] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:01.812902 20321 raft_consensus.cc:515] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } } peers { permanent_uuid: "073614a77a994b8fbed8f8cd6b7927de" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 38247 } }
I20260812 08:03:01.813293 20321 leader_election.cc:290] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [CANDIDATE]: Term 1 election: Requested vote from peers 7dabac9590204b268f758f8f24ed87be (127.0.186.67:39869), 073614a77a994b8fbed8f8cd6b7927de (127.0.186.66:38247)
I20260812 08:03:01.813720 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39f77f897dab4a528691252eb9a48c03" candidate_uuid: "4285c138ac1d46fcb5810461505f8639" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073614a77a994b8fbed8f8cd6b7927de"
I20260812 08:03:01.813720 20243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39f77f897dab4a528691252eb9a48c03" candidate_uuid: "4285c138ac1d46fcb5810461505f8639" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dabac9590204b268f758f8f24ed87be"
I20260812 08:03:01.813856 20243 raft_consensus.cc:3060] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:01.813856 20088 raft_consensus.cc:3060] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:01.814961 20243 raft_consensus.cc:2468] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4285c138ac1d46fcb5810461505f8639 in term 1.
I20260812 08:03:01.814961 20088 raft_consensus.cc:2468] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4285c138ac1d46fcb5810461505f8639 in term 1.
I20260812 08:03:01.815316 19852 leader_election.cc:304] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073614a77a994b8fbed8f8cd6b7927de, 4285c138ac1d46fcb5810461505f8639; no voters: 
I20260812 08:03:01.815512 20321 raft_consensus.cc:2804] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:01.815747 20321 raft_consensus.cc:697] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [term 1 LEADER]: Becoming Leader. State: Replica: 4285c138ac1d46fcb5810461505f8639, State: Running, Role: LEADER
I20260812 08:03:01.816108 20321 consensus_queue.cc:237] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } } peers { permanent_uuid: "073614a77a994b8fbed8f8cd6b7927de" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 38247 } }
I20260812 08:03:01.819446 19758 catalog_manager.cc:5719] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4285c138ac1d46fcb5810461505f8639 (127.0.186.65). New cstate: current_term: 1 leader_uuid: "4285c138ac1d46fcb5810461505f8639" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "073614a77a994b8fbed8f8cd6b7927de" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 38247 } health_report { overall_health: UNKNOWN } } }
I20260812 08:03:01.830400   745 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:03:01.832026   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 4285c138ac1d46fcb5810461505f8639 to finish bootstrapping
I20260812 08:03:01.837186   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 073614a77a994b8fbed8f8cd6b7927de to finish bootstrapping
I20260812 08:03:01.842031   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 7dabac9590204b268f758f8f24ed87be to finish bootstrapping
I20260812 08:03:01.857648   745 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 0.0 1.1
I20260812 08:03:01.960632   745 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 0.0 1.1
I20260812 08:03:02.163565   745 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 0.0 1.1
I20260812 08:03:02.384497 20330 consensus_queue.cc:1048] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:02.390116 20330 consensus_queue.cc:1048] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [LEADER]: Connected to new peer: Peer: permanent_uuid: "073614a77a994b8fbed8f8cd6b7927de" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 38247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:02.467566   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 19987
W20260812 08:03:02.815167 19852 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.66:38247: connect: Connection refused (error 111)
W20260812 08:03:02.816217 19852 consensus_peers.cc:597] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 -> Peer 073614a77a994b8fbed8f8cd6b7927de (127.0.186.66:38247): Couldn't send request to peer 073614a77a994b8fbed8f8cd6b7927de. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.66:38247: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260812 08:03:05.380872 19852 consensus_peers.cc:597] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 -> Peer 073614a77a994b8fbed8f8cd6b7927de (127.0.186.66:38247): Couldn't send request to peer 073614a77a994b8fbed8f8cd6b7927de. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.66:38247: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260812 08:03:07.501873 20342 consensus_queue.cc:579] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [LEADER]: Leader has been unable to successfully communicate with peer 073614a77a994b8fbed8f8cd6b7927de for more than 5 seconds (5.109s)
W20260812 08:03:08.112605 19852 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.66:38247: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260812 08:03:08.113529 19852 consensus_peers.cc:597] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 -> Peer 073614a77a994b8fbed8f8cd6b7927de (127.0.186.66:38247): Couldn't send request to peer 073614a77a994b8fbed8f8cd6b7927de. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.66:38247: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260812 08:03:10.810420 19852 consensus_peers.cc:597] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 -> Peer 073614a77a994b8fbed8f8cd6b7927de (127.0.186.66:38247): Couldn't send request to peer 073614a77a994b8fbed8f8cd6b7927de. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.66:38247: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260812 08:03:13.419019 19852 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.66:38247: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260812 08:03:13.419672 19852 consensus_peers.cc:597] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 -> Peer 073614a77a994b8fbed8f8cd6b7927de (127.0.186.66:38247): Couldn't send request to peer 073614a77a994b8fbed8f8cd6b7927de. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.66:38247: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260812 08:03:16.214071 19852 consensus_peers.cc:597] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 -> Peer 073614a77a994b8fbed8f8cd6b7927de (127.0.186.66:38247): Couldn't send request to peer 073614a77a994b8fbed8f8cd6b7927de. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.66:38247: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260812 08:03:17.495443   745 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
I20260812 08:03:17.495772   745 raft_consensus-itest-base.cc:268] Waiting until we've written at least 4MB...
I20260812 08:03:17.565253 19984 maintenance_manager.cc:419] P 4285c138ac1d46fcb5810461505f8639: Scheduling FlushMRSOp(39f77f897dab4a528691252eb9a48c03): 3681746 bytes log retention, and flush 4038921 bytes memory
I20260812 08:03:17.606452 20385 consensus_queue.cc:799] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [LEADER]: Peer 073614a77a994b8fbed8f8cd6b7927de is lagging by at least 29 ops behind the committed index 
I20260812 08:03:17.619263   745 raft_consensus-itest-base.cc:274] Waiting for log GC on 4285c138ac1d46fcb5810461505f8639
I20260812 08:03:17.620044 20295 maintenance_manager.cc:419] P 7dabac9590204b268f758f8f24ed87be: Scheduling FlushMRSOp(39f77f897dab4a528691252eb9a48c03): 4865177 bytes log retention, and flush 6119689 bytes memory
I20260812 08:03:17.623188 19853 maintenance_manager.cc:643] P 4285c138ac1d46fcb5810461505f8639: FlushMRSOp(39f77f897dab4a528691252eb9a48c03) complete. Timing: real 0.057s	user 0.040s	sys 0.005s Metrics: {"bytes_written":135937,"cfile_init":1,"compiler_manager_pool.queue_time_us":270,"dirs.queue_time_us":7496,"dirs.run_cpu_time_us":211,"dirs.run_wall_time_us":1217,"drs_written":1,"lbm_read_time_us":58,"lbm_reads_lt_1ms":4,"lbm_write_time_us":656,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":25,"thread_start_us":282,"threads_started":2}
I20260812 08:03:17.624221 19984 maintenance_manager.cc:419] P 4285c138ac1d46fcb5810461505f8639: Scheduling LogGCOp(39f77f897dab4a528691252eb9a48c03): free 2761328 bytes of WAL
I20260812 08:03:17.624481 19853 log_reader.cc:385] T 39f77f897dab4a528691252eb9a48c03: removed 3 log segments from log reader
I20260812 08:03:17.624583 19853 log.cc:1079] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/39f77f897dab4a528691252eb9a48c03/wal-000000001 (ops 1-8)
I20260812 08:03:17.624672 19853 log.cc:1079] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/39f77f897dab4a528691252eb9a48c03/wal-000000002 (ops 9-15)
I20260812 08:03:17.624729 19853 log.cc:1079] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/39f77f897dab4a528691252eb9a48c03/wal-000000003 (ops 16-22)
I20260812 08:03:17.625705 19853 maintenance_manager.cc:643] P 4285c138ac1d46fcb5810461505f8639: LogGCOp(39f77f897dab4a528691252eb9a48c03) complete. Timing: real 0.001s	user 0.000s	sys 0.001s Metrics: {}
I20260812 08:03:17.626071 19984 maintenance_manager.cc:419] P 4285c138ac1d46fcb5810461505f8639: Scheduling FlushMRSOp(39f77f897dab4a528691252eb9a48c03): 1840874 bytes log retention, and flush 2998537 bytes memory
I20260812 08:03:17.629702   745 raft_consensus-itest-base.cc:279] Log GC complete on 4285c138ac1d46fcb5810461505f8639
I20260812 08:03:17.650223 19853 maintenance_manager.cc:643] P 4285c138ac1d46fcb5810461505f8639: FlushMRSOp(39f77f897dab4a528691252eb9a48c03) complete. Timing: real 0.024s	user 0.020s	sys 0.003s Metrics: {"bytes_written":135803,"cfile_init":1,"dirs.queue_time_us":69,"dirs.run_cpu_time_us":304,"dirs.run_wall_time_us":1385,"drs_written":1,"lbm_read_time_us":100,"lbm_reads_lt_1ms":4,"lbm_write_time_us":712,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":15}
I20260812 08:03:17.651619 19984 maintenance_manager.cc:419] P 4285c138ac1d46fcb5810461505f8639: Scheduling CompactRowSetsOp(39f77f897dab4a528691252eb9a48c03): perf score=0.911917
I20260812 08:03:17.686686 20165 maintenance_manager.cc:643] P 7dabac9590204b268f758f8f24ed87be: FlushMRSOp(39f77f897dab4a528691252eb9a48c03) complete. Timing: real 0.066s	user 0.065s	sys 0.000s Metrics: {"bytes_written":136001,"cfile_init":1,"compiler_manager_pool.queue_time_us":247,"drs_written":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4,"lbm_write_time_us":741,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":40,"thread_start_us":273,"threads_started":2}
I20260812 08:03:17.687841 20295 maintenance_manager.cc:419] P 7dabac9590204b268f758f8f24ed87be: Scheduling LogGCOp(39f77f897dab4a528691252eb9a48c03): free 3287287 bytes of WAL
I20260812 08:03:17.688179 20165 log_reader.cc:385] T 39f77f897dab4a528691252eb9a48c03: removed 4 log segments from log reader
I20260812 08:03:17.688287 20165 log.cc:1079] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/39f77f897dab4a528691252eb9a48c03/wal-000000001 (ops 1-8)
I20260812 08:03:17.688397 20165 log.cc:1079] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/39f77f897dab4a528691252eb9a48c03/wal-000000002 (ops 9-14)
I20260812 08:03:17.688473 20165 log.cc:1079] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/39f77f897dab4a528691252eb9a48c03/wal-000000003 (ops 15-21)
I20260812 08:03:17.688544 20165 log.cc:1079] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/39f77f897dab4a528691252eb9a48c03/wal-000000004 (ops 22-26)
I20260812 08:03:17.689741 20165 maintenance_manager.cc:643] P 7dabac9590204b268f758f8f24ed87be: LogGCOp(39f77f897dab4a528691252eb9a48c03) complete. Timing: real 0.002s	user 0.000s	sys 0.001s Metrics: {}
I20260812 08:03:17.690272 20295 maintenance_manager.cc:419] P 7dabac9590204b268f758f8f24ed87be: Scheduling UndoDeltaBlockGCOp(39f77f897dab4a528691252eb9a48c03): 591 bytes on disk
I20260812 08:03:17.690898 20165 maintenance_manager.cc:643] P 7dabac9590204b268f758f8f24ed87be: UndoDeltaBlockGCOp(39f77f897dab4a528691252eb9a48c03) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":65,"lbm_reads_lt_1ms":4}
I20260812 08:03:17.711439 19853 maintenance_manager.cc:643] P 4285c138ac1d46fcb5810461505f8639: CompactRowSetsOp(39f77f897dab4a528691252eb9a48c03) complete. Timing: real 0.060s	user 0.056s	sys 0.003s Metrics: {"bytes_written":136001,"cfile_cache_hit":10,"cfile_cache_hit_bytes":262394,"cfile_cache_miss":26,"cfile_cache_miss_bytes":264530,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":881,"dirs.run_cpu_time_us":216,"dirs.run_wall_time_us":1157,"drs_written":1,"lbm_read_time_us":913,"lbm_reads_lt_1ms":54,"lbm_write_time_us":730,"lbm_writes_lt_1ms":23,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":40,"thread_start_us":384,"threads_started":5}
I20260812 08:03:17.712294 19984 maintenance_manager.cc:419] P 4285c138ac1d46fcb5810461505f8639: Scheduling UndoDeltaBlockGCOp(39f77f897dab4a528691252eb9a48c03): 591 bytes on disk
I20260812 08:03:17.712909 19853 maintenance_manager.cc:643] P 4285c138ac1d46fcb5810461505f8639: UndoDeltaBlockGCOp(39f77f897dab4a528691252eb9a48c03) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":62,"lbm_reads_lt_1ms":4}
I20260812 08:03:18.167848 20433 consensus_queue.cc:753] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [LEADER]: The logs necessary to catch up peer 073614a77a994b8fbed8f8cd6b7927de have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..34: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:18.167948 20433 consensus_queue.cc:579] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [LEADER]: The logs necessary to catch up peer 073614a77a994b8fbed8f8cd6b7927de have been garbage collected. The replica will never be able to catch up
I20260812 08:03:18.171396 19929 consensus_queue.cc:237] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 41, Committed index: 41, Last appended: 1.41, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } }
I20260812 08:03:18.172770 20243 raft_consensus.cc:1275] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be [term 1 FOLLOWER]: Refusing update from remote peer 4285c138ac1d46fcb5810461505f8639: Log matching property violated. Preceding OpId in replica: term: 1 index: 41. Preceding OpId from leader: term: 1 index: 42. (index mismatch)
I20260812 08:03:18.173216 20434 consensus_queue.cc:1048] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 42, Last known committed idx: 41, Time since last communication: 0.000s
I20260812 08:03:18.175158 20433 raft_consensus.cc:2955] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [term 1 LEADER]: Committing config change with OpId 1.42: config changed from index -1 to 42, VOTER 073614a77a994b8fbed8f8cd6b7927de (127.0.186.66) evicted. New config: { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } } }
I20260812 08:03:18.175637 20243 raft_consensus.cc:2955] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be [term 1 FOLLOWER]: Committing config change with OpId 1.42: config changed from index -1 to 42, VOTER 073614a77a994b8fbed8f8cd6b7927de (127.0.186.66) evicted. New config: { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } } }
I20260812 08:03:18.177162 19739 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 39f77f897dab4a528691252eb9a48c03 with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:18.177997 19757 catalog_manager.cc:5719] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 reported cstate change: config changed from index -1 to 42, VOTER 073614a77a994b8fbed8f8cd6b7927de (127.0.186.66) evicted. New cstate: current_term: 1 leader_uuid: "4285c138ac1d46fcb5810461505f8639" committed_config { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } health_report { overall_health: HEALTHY } } }
W20260812 08:03:18.181203 19741 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.66:38247: connect: Connection refused (error 111)
W20260812 08:03:18.182174 19741 catalog_manager.cc:4777] TS 073614a77a994b8fbed8f8cd6b7927de (127.0.186.66:38247): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 39f77f897dab4a528691252eb9a48c03: Network error: Client connection negotiation failed: client connection to 127.0.186.66:38247: connect: Connection refused (error 111)
I20260812 08:03:18.193532 20295 maintenance_manager.cc:419] P 7dabac9590204b268f758f8f24ed87be: Scheduling LogGCOp(39f77f897dab4a528691252eb9a48c03): free 1577890 bytes of WAL
I20260812 08:03:18.193822 20165 log_reader.cc:385] T 39f77f897dab4a528691252eb9a48c03: removed 2 log segments from log reader
I20260812 08:03:18.193873 20165 log.cc:1079] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/39f77f897dab4a528691252eb9a48c03/wal-000000005 (ops 27-33)
I20260812 08:03:18.193905 20165 log.cc:1079] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/39f77f897dab4a528691252eb9a48c03/wal-000000006 (ops 34-38)
I20260812 08:03:18.194375 20165 maintenance_manager.cc:643] P 7dabac9590204b268f758f8f24ed87be: LogGCOp(39f77f897dab4a528691252eb9a48c03) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 08:03:18.215612 19984 maintenance_manager.cc:419] P 4285c138ac1d46fcb5810461505f8639: Scheduling LogGCOp(39f77f897dab4a528691252eb9a48c03): free 1840874 bytes of WAL
I20260812 08:03:18.215939 19853 log_reader.cc:385] T 39f77f897dab4a528691252eb9a48c03: removed 2 log segments from log reader
I20260812 08:03:18.216004 19853 log.cc:1079] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/39f77f897dab4a528691252eb9a48c03/wal-000000004 (ops 23-29)
I20260812 08:03:18.216078 19853 log.cc:1079] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/39f77f897dab4a528691252eb9a48c03/wal-000000005 (ops 30-36)
I20260812 08:03:18.216624 19853 maintenance_manager.cc:643] P 4285c138ac1d46fcb5810461505f8639: LogGCOp(39f77f897dab4a528691252eb9a48c03) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
W20260812 08:03:19.393371 19741 catalog_manager.cc:4777] TS 073614a77a994b8fbed8f8cd6b7927de (127.0.186.66:38247): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 39f77f897dab4a528691252eb9a48c03: Network error: Client connection negotiation failed: client connection to 127.0.186.66:38247: connect: Connection refused (error 111)
I20260812 08:03:19.631130   745 raft_consensus-itest-base.cc:295] Servers converged with original term 1
I20260812 08:03:19.631203   745 raft_consensus-itest-base.cc:303] Restarting 073614a77a994b8fbed8f8cd6b7927de
I20260812 08:03:19.631534   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.66:38247
--local_ip_for_outbound_sockets=127.0.186.66
--tserver_master_addrs=127.0.186.126:35865
--webserver_port=34375
--webserver_interface=127.0.186.66
--builtin_ntp_servers=127.0.186.84:35847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20260812 08:03:19.742429 20448 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:03:19.742664 20448 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:19.742686 20448 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:19.742748 20448 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:19.746197 20448 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:03:19.746286 20448 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:03:19.746320 20448 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:19.746393 20448 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:03:19.746434 20448 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.66
I20260812 08:03:19.750545 20448 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:35847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.66:38247
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.186.66
--webserver_port=34375
--tserver_master_addrs=127.0.186.126:35865
--never_fsync=true
--heap_profile_path=/tmp/kudu.20448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:19.751703 20448 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:19.752910 20448 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:19.759519 20457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:19.759629 20448 server_base.cc:1061] running on GCE node
W20260812 08:03:19.759509 20454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:19.759509 20453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:19.760141 20448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:19.760680 20448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:19.761860 20448 hybrid_clock.cc:648] HybridClock initialized: now 1786521799761840 us; error 32 us; skew 500 ppm
I20260812 08:03:19.763713 20448 webserver.cc:533] Webserver started at http://127.0.186.66:34375/ using document root <none> and password file <none>
I20260812 08:03:19.764303 20448 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:19.764384 20448 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:19.768002 20448 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:19.770123 20463 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:19.771201 20448 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 08:03:19.771338 20448 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "073614a77a994b8fbed8f8cd6b7927de"
format_stamp: "Formatted at 2026-08-12 08:03:01 on dist-test-slave-49h3"
I20260812 08:03:19.771770 20448 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:19.799309 20448 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:19.800017 20448 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:19.800205 20448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:19.800936 20448 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:19.802397 20474 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:03:19.805950 20448 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:19.806022 20448 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.001s	sys 0.000s
I20260812 08:03:19.806136 20448 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:19.807683 20448 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:19.807730 20448 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:03:19.808046 20474 tablet_bootstrap.cc:492] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: Bootstrap starting.
I20260812 08:03:19.815498 20474 log.cc:826] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:19.818110 20474 tablet_bootstrap.cc:492] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:03:19.818390 20448 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.66:38247
I20260812 08:03:19.818549 20604 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.66:38247 every 8 connection(s)
I20260812 08:03:19.818753 20474 tablet_bootstrap.cc:492] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: Bootstrap complete.
I20260812 08:03:19.819176 20474 ts_tablet_manager.cc:1403] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: Time spent bootstrapping tablet: real 0.011s	user 0.007s	sys 0.004s
I20260812 08:03:19.819536 20448 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:03:19.822503 20474 raft_consensus.cc:359] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } } peers { permanent_uuid: "073614a77a994b8fbed8f8cd6b7927de" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 38247 } }
I20260812 08:03:19.823004 20474 raft_consensus.cc:740] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073614a77a994b8fbed8f8cd6b7927de, State: Initialized, Role: FOLLOWER
I20260812 08:03:19.823473 20474 consensus_queue.cc:260] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } } peers { permanent_uuid: "073614a77a994b8fbed8f8cd6b7927de" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 38247 } }
I20260812 08:03:19.824245 20474 ts_tablet_manager.cc:1434] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.004s
I20260812 08:03:19.827491 20606 heartbeater.cc:344] Connected to a master server at 127.0.186.126:35865
I20260812 08:03:19.827739 20606 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:19.828214 20606 heartbeater.cc:507] Master 127.0.186.126:35865 requested a full tablet report, sending...
I20260812 08:03:19.829056   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 20448
I20260812 08:03:19.830165 19759 ts_manager.cc:194] Re-registered known tserver with Master: 073614a77a994b8fbed8f8cd6b7927de (127.0.186.66:38247)
I20260812 08:03:19.831094 19759 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.66:53489
I20260812 08:03:19.832811 20606 heartbeater.cc:499] Master 127.0.186.126:35865 was elected leader, sending a full tablet report...
I20260812 08:03:19.833585   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 19715
W20260812 08:03:19.845687 20606 heartbeater.cc:646] Failed to heartbeat to 127.0.186.126:35865 (0 consecutive failures): Network error: Failed to send heartbeat to master: recv got EOF from 127.0.186.126:35865 (error 108)
I20260812 08:03:19.846074   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:35865
--webserver_interface=127.0.186.126
--webserver_port=45259
--builtin_ntp_servers=127.0.186.84:35847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:35865
--raft_prepare_replacement_before_eviction=true
--master_add_server_when_underreplicated=false
--master_add_server_when_underreplicated=true with env {}
W20260812 08:03:19.955790 20614 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:19.956043 20614 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:19.956082 20614 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:19.959547 20614 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:03:19.959604 20614 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:19.959625 20614 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:03:19.959641 20614 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:03:19.963449 20614 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:35847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:35865
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:35865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=45259
--never_fsync=true
--heap_profile_path=/tmp/kudu.20614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:19.964555 20614 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:19.965727 20614 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:19.971606 20619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:19.971728 20622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:19.971851 20620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:19.971886 20614 server_base.cc:1061] running on GCE node
I20260812 08:03:19.972476 20614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:19.973609 20614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:19.974825 20614 hybrid_clock.cc:648] HybridClock initialized: now 1786521799974805 us; error 35 us; skew 500 ppm
I20260812 08:03:19.977842 20614 webserver.cc:533] Webserver started at http://127.0.186.126:45259/ using document root <none> and password file <none>
I20260812 08:03:19.978681 20614 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:19.978794 20614 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:19.983099 20614 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:03:19.985375 20628 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:19.986604 20614 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:03:19.986814 20614 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "5dd9251f7ef54785a91610de46ec0292"
format_stamp: "Formatted at 2026-08-12 08:03:01 on dist-test-slave-49h3"
I20260812 08:03:19.987265 20614 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:20.018222 20614 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:20.019109 20614 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:20.019310 20614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:20.027964 20692 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:35865 every 8 connection(s)
I20260812 08:03:20.027992 20614 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:35865
I20260812 08:03:20.029300 20614 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:03:20.032884   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 20614
I20260812 08:03:20.033936 20693 sys_catalog.cc:263] Verifying existing consensus state
I20260812 08:03:20.036172 20693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292: Bootstrap starting.
I20260812 08:03:20.050189 20693 log.cc:826] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:20.058571 20693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=3 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:03:20.058965 20693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292: Bootstrap complete.
I20260812 08:03:20.062712 20693 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dd9251f7ef54785a91610de46ec0292" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 35865 } }
I20260812 08:03:20.063130 20693 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dd9251f7ef54785a91610de46ec0292, State: Initialized, Role: FOLLOWER
I20260812 08:03:20.063637 20693 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dd9251f7ef54785a91610de46ec0292" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 35865 } }
I20260812 08:03:20.063774 20693 raft_consensus.cc:399] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:03:20.063846 20693 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:03:20.063968 20693 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:03:20.065747 20693 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dd9251f7ef54785a91610de46ec0292" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 35865 } }
I20260812 08:03:20.066140 20693 leader_election.cc:304] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5dd9251f7ef54785a91610de46ec0292; no voters: 
I20260812 08:03:20.066495 20693 leader_election.cc:290] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [CANDIDATE]: Term 2 election: Requested vote from peers 
I20260812 08:03:20.066696 20698 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:03:20.066941 20698 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [term 2 LEADER]: Becoming Leader. State: Replica: 5dd9251f7ef54785a91610de46ec0292, State: Running, Role: LEADER
I20260812 08:03:20.067266 20698 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dd9251f7ef54785a91610de46ec0292" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 35865 } }
I20260812 08:03:20.067754 20693 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:20.068802 20701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5dd9251f7ef54785a91610de46ec0292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dd9251f7ef54785a91610de46ec0292" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 35865 } } }
I20260812 08:03:20.068936 20701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:20.069061 20702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5dd9251f7ef54785a91610de46ec0292. Latest consensus state: current_term: 2 leader_uuid: "5dd9251f7ef54785a91610de46ec0292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dd9251f7ef54785a91610de46ec0292" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 35865 } } }
I20260812 08:03:20.069141 20702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:20.069614 20711 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:03:20.073271 20711 catalog_manager.cc:679] Loaded metadata for table TestTable [id=9bf92388afda40eeb62d1598dbc607d7]
I20260812 08:03:20.074687 20711 tablet_loader.cc:96] loaded metadata for tablet 39f77f897dab4a528691252eb9a48c03 (table TestTable [id=9bf92388afda40eeb62d1598dbc607d7])
I20260812 08:03:20.074848 20711 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:03:20.075703 20711 catalog_manager.cc:1295] Loaded cluster ID: fa5013677d9c4b299c4f2ffe6809c261
I20260812 08:03:20.075769 20711 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:03:20.078166 20711 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:03:20.078966 20711 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 5dd9251f7ef54785a91610de46ec0292: Loaded TSK: 0
I20260812 08:03:20.080096 20711 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:03:20.189634 20647 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "7dabac9590204b268f758f8f24ed87be" instance_seqno: 1786521781659328) as {username='slave'} at 127.0.186.67:33599; Asking this server to re-register.
I20260812 08:03:20.189889 20643 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "4285c138ac1d46fcb5810461505f8639" instance_seqno: 1786521781301894) as {username='slave'} at 127.0.186.65:41325; Asking this server to re-register.
I20260812 08:03:20.190321 20294 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:20.190553 20294 heartbeater.cc:507] Master 127.0.186.126:35865 requested a full tablet report, sending...
I20260812 08:03:20.191201 19983 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:20.191392 19983 heartbeater.cc:507] Master 127.0.186.126:35865 requested a full tablet report, sending...
I20260812 08:03:20.192669 20647 ts_manager.cc:194] Registered new tserver with Master: 4285c138ac1d46fcb5810461505f8639 (127.0.186.65:41261)
I20260812 08:03:20.192759 20643 ts_manager.cc:194] Registered new tserver with Master: 7dabac9590204b268f758f8f24ed87be (127.0.186.67:39869)
W20260812 08:03:20.195909 20647 catalog_manager.cc:5397] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 39f77f897dab4a528691252eb9a48c03 with cas_config_opid_index 42: no extra replica candidate found for tablet 39f77f897dab4a528691252eb9a48c03 (table TestTable [id=9bf92388afda40eeb62d1598dbc607d7]): Not found: could not select location for extra replica: not enough tablet servers to satisfy replica placement policy: the total number of registered tablet servers (2) does not allow for adding an extra replica; consider bringing up more to have at least 4 tablet servers up and running
I20260812 08:03:20.850545 20606 heartbeater.cc:344] Connected to a master server at 127.0.186.126:35865
I20260812 08:03:20.850709 20606 heartbeater.cc:499] Master 127.0.186.126:35865 was elected leader, sending a full tablet report...
I20260812 08:03:20.851527 20647 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "073614a77a994b8fbed8f8cd6b7927de" instance_seqno: 1786521799815579) as {username='slave'} at 127.0.186.66:32847; Asking this server to re-register.
I20260812 08:03:20.852020 20606 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:20.852186 20606 heartbeater.cc:507] Master 127.0.186.126:35865 requested a full tablet report, sending...
I20260812 08:03:20.852970 20647 ts_manager.cc:194] Registered new tserver with Master: 073614a77a994b8fbed8f8cd6b7927de (127.0.186.66:38247)
I20260812 08:03:20.858819 20523 tablet_service.cc:1558] Processing DeleteTablet for tablet 39f77f897dab4a528691252eb9a48c03 with delete_type TABLET_DATA_TOMBSTONED (Replica has no consensus available (current committed config index is 42)) from {username='slave'} at 127.0.0.1:36326
I20260812 08:03:20.859393 20732 tablet_replica.cc:333] stopping tablet replica
I20260812 08:03:20.859717 20732 raft_consensus.cc:2243] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 08:03:20.860020 20732 raft_consensus.cc:2272] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 08:03:20.861056 20732 ts_tablet_manager.cc:1916] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 08:03:20.864408 19929 consensus_queue.cc:237] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 42, Committed index: 42, Last appended: 1.42, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } } peers { permanent_uuid: "073614a77a994b8fbed8f8cd6b7927de" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 38247 } attrs { promote: true } }
I20260812 08:03:20.865052 20732 ts_tablet_manager.cc:1929] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.1
I20260812 08:03:20.865149 20732 log.cc:1199] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: Deleting WAL directory at /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/wals/39f77f897dab4a528691252eb9a48c03
I20260812 08:03:20.866433 20243 raft_consensus.cc:1275] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be [term 1 FOLLOWER]: Refusing update from remote peer 4285c138ac1d46fcb5810461505f8639: Log matching property violated. Preceding OpId in replica: term: 1 index: 42. Preceding OpId from leader: term: 1 index: 43. (index mismatch)
I20260812 08:03:20.866683 20632 catalog_manager.cc:5050] TS 073614a77a994b8fbed8f8cd6b7927de (127.0.186.66:38247): tablet 39f77f897dab4a528691252eb9a48c03 (table TestTable [id=9bf92388afda40eeb62d1598dbc607d7]) successfully deleted
I20260812 08:03:20.867015 20735 consensus_queue.cc:1048] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 43, Last known committed idx: 42, Time since last communication: 0.000s
I20260812 08:03:20.868997 20724 raft_consensus.cc:2955] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [term 1 LEADER]: Committing config change with OpId 1.43: config changed from index 42 to 43, NON_VOTER 073614a77a994b8fbed8f8cd6b7927de (127.0.186.66) added. New config: { opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } } peers { permanent_uuid: "073614a77a994b8fbed8f8cd6b7927de" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 38247 } attrs { promote: true } } }
W20260812 08:03:20.870429 19852 consensus_peers.cc:597] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 -> Peer 073614a77a994b8fbed8f8cd6b7927de (127.0.186.66:38247): Couldn't send request to peer 073614a77a994b8fbed8f8cd6b7927de. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:03:20.870801 20630 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 39f77f897dab4a528691252eb9a48c03 with cas_config_opid_index 42: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 6)
I20260812 08:03:20.871411 20243 raft_consensus.cc:2955] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be [term 1 FOLLOWER]: Committing config change with OpId 1.43: config changed from index 42 to 43, NON_VOTER 073614a77a994b8fbed8f8cd6b7927de (127.0.186.66) added. New config: { opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } } peers { permanent_uuid: "073614a77a994b8fbed8f8cd6b7927de" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 38247 } attrs { promote: true } } }
I20260812 08:03:20.871757 20647 catalog_manager.cc:5719] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 reported cstate change: config changed from index 42 to 43, NON_VOTER 073614a77a994b8fbed8f8cd6b7927de (127.0.186.66) added. New cstate: current_term: 1 leader_uuid: "4285c138ac1d46fcb5810461505f8639" committed_config { opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "073614a77a994b8fbed8f8cd6b7927de" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 38247 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260812 08:03:21.277304 20745 ts_tablet_manager.cc:933] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: Initiating tablet copy from peer 4285c138ac1d46fcb5810461505f8639 (127.0.186.65:41261)
I20260812 08:03:21.278012 20745 tablet_copy_client.cc:323] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: tablet copy: Beginning tablet copy session from remote peer at address 127.0.186.65:41261
I20260812 08:03:21.281949 19953 tablet_copy_service.cc:140] P 4285c138ac1d46fcb5810461505f8639: Received BeginTabletCopySession request for tablet 39f77f897dab4a528691252eb9a48c03 from peer 073614a77a994b8fbed8f8cd6b7927de ({username='slave'} at 127.0.186.66:48911)
I20260812 08:03:21.282121 19953 tablet_copy_service.cc:161] P 4285c138ac1d46fcb5810461505f8639: Beginning new tablet copy session on tablet 39f77f897dab4a528691252eb9a48c03 from peer 073614a77a994b8fbed8f8cd6b7927de at {username='slave'} at 127.0.186.66:48911: session id = 073614a77a994b8fbed8f8cd6b7927de-39f77f897dab4a528691252eb9a48c03
I20260812 08:03:21.283618 19953 tablet_copy_source_session.cc:215] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639: Tablet Copy: opened 5 blocks and 1 log segments
I20260812 08:03:21.285219 20745 ts_tablet_manager.cc:1916] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260812 08:03:21.287019 20745 ts_tablet_manager.cc:1929] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.1
I20260812 08:03:21.287361 20745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39f77f897dab4a528691252eb9a48c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:21.288475 20745 tablet_copy_client.cc:806] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: tablet copy: Starting download of 5 data blocks...
I20260812 08:03:21.295368 20745 tablet_copy_client.cc:670] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: tablet copy: Starting download of 1 WAL segments...
I20260812 08:03:21.299559 20745 tablet_copy_client.cc:538] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 08:03:21.301532 20745 tablet_bootstrap.cc:492] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: Bootstrap starting.
I20260812 08:03:21.313038 20745 tablet_bootstrap.cc:492] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=5} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=3. Pending: 0 replicates
I20260812 08:03:21.313334 20745 tablet_bootstrap.cc:492] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: Bootstrap complete.
I20260812 08:03:21.313715 20745 ts_tablet_manager.cc:1403] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: Time spent bootstrapping tablet: real 0.012s	user 0.015s	sys 0.000s
I20260812 08:03:21.314281 20745 raft_consensus.cc:359] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } } peers { permanent_uuid: "073614a77a994b8fbed8f8cd6b7927de" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 38247 } attrs { promote: true } }
I20260812 08:03:21.314393 20745 raft_consensus.cc:740] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 073614a77a994b8fbed8f8cd6b7927de, State: Initialized, Role: LEARNER
I20260812 08:03:21.314503 20745 consensus_queue.cc:260] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 43, Last appended: 1.43, Last appended by leader: 43, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } } peers { permanent_uuid: "073614a77a994b8fbed8f8cd6b7927de" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 38247 } attrs { promote: true } }
I20260812 08:03:21.314879 20745 ts_tablet_manager.cc:1434] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20260812 08:03:21.315675 19953 tablet_copy_service.cc:342] P 4285c138ac1d46fcb5810461505f8639: Request end of tablet copy session 073614a77a994b8fbed8f8cd6b7927de-39f77f897dab4a528691252eb9a48c03 received from {username='slave'} at 127.0.186.66:48911
I20260812 08:03:21.315809 19953 tablet_copy_service.cc:434] P 4285c138ac1d46fcb5810461505f8639: ending tablet copy session 073614a77a994b8fbed8f8cd6b7927de-39f77f897dab4a528691252eb9a48c03 on tablet 39f77f897dab4a528691252eb9a48c03 with peer 073614a77a994b8fbed8f8cd6b7927de
I20260812 08:03:21.326010 20607 maintenance_manager.cc:419] P 073614a77a994b8fbed8f8cd6b7927de: Scheduling UndoDeltaBlockGCOp(39f77f897dab4a528691252eb9a48c03): 591 bytes on disk
I20260812 08:03:21.327299 20471 maintenance_manager.cc:643] P 073614a77a994b8fbed8f8cd6b7927de: UndoDeltaBlockGCOp(39f77f897dab4a528691252eb9a48c03) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":74,"lbm_reads_lt_1ms":4}
I20260812 08:03:21.816573 20547 raft_consensus.cc:1217] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de [term 1 LEARNER]: Deduplicated request from leader. Original: 1.42->[1.43-1.43]   Dedup: 1.43->[]
I20260812 08:03:22.216709 20735 raft_consensus.cc:1064] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639: attempting to promote NON_VOTER 073614a77a994b8fbed8f8cd6b7927de to VOTER
I20260812 08:03:22.217257 20735 consensus_queue.cc:237] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 43, Committed index: 43, Last appended: 1.43, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } } peers { permanent_uuid: "073614a77a994b8fbed8f8cd6b7927de" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 38247 } attrs { promote: false } }
I20260812 08:03:22.218840 20243 raft_consensus.cc:1275] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be [term 1 FOLLOWER]: Refusing update from remote peer 4285c138ac1d46fcb5810461505f8639: Log matching property violated. Preceding OpId in replica: term: 1 index: 43. Preceding OpId from leader: term: 1 index: 44. (index mismatch)
I20260812 08:03:22.218916 20547 raft_consensus.cc:1275] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de [term 1 LEARNER]: Refusing update from remote peer 4285c138ac1d46fcb5810461505f8639: Log matching property violated. Preceding OpId in replica: term: 1 index: 43. Preceding OpId from leader: term: 1 index: 44. (index mismatch)
I20260812 08:03:22.219352 20752 consensus_queue.cc:1048] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 44, Last known committed idx: 43, Time since last communication: 0.000s
I20260812 08:03:22.219594 20751 consensus_queue.cc:1048] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [LEADER]: Connected to new peer: Peer: permanent_uuid: "073614a77a994b8fbed8f8cd6b7927de" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 38247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 44, Last known committed idx: 43, Time since last communication: 0.000s
I20260812 08:03:22.221553 20751 raft_consensus.cc:2955] T 39f77f897dab4a528691252eb9a48c03 P 4285c138ac1d46fcb5810461505f8639 [term 1 LEADER]: Committing config change with OpId 1.44: config changed from index 43 to 44, 073614a77a994b8fbed8f8cd6b7927de (127.0.186.66) changed from NON_VOTER to VOTER. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } } peers { permanent_uuid: "073614a77a994b8fbed8f8cd6b7927de" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 38247 } attrs { promote: false } } }
I20260812 08:03:22.222218 20243 raft_consensus.cc:2955] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be [term 1 FOLLOWER]: Committing config change with OpId 1.44: config changed from index 43 to 44, 073614a77a994b8fbed8f8cd6b7927de (127.0.186.66) changed from NON_VOTER to VOTER. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } } peers { permanent_uuid: "073614a77a994b8fbed8f8cd6b7927de" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 38247 } attrs { promote: false } } }
I20260812 08:03:22.222996 20547 raft_consensus.cc:2955] T 39f77f897dab4a528691252eb9a48c03 P 073614a77a994b8fbed8f8cd6b7927de [term 1 FOLLOWER]: Committing config change with OpId 1.44: config changed from index 43 to 44, 073614a77a994b8fbed8f8cd6b7927de (127.0.186.66) changed from NON_VOTER to VOTER. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } } peers { permanent_uuid: "073614a77a994b8fbed8f8cd6b7927de" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 38247 } attrs { promote: false } } }
I20260812 08:03:22.224844 20647 catalog_manager.cc:5719] T 39f77f897dab4a528691252eb9a48c03 P 7dabac9590204b268f758f8f24ed87be reported cstate change: config changed from index 43 to 44, 073614a77a994b8fbed8f8cd6b7927de (127.0.186.66) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4285c138ac1d46fcb5810461505f8639" committed_config { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "7dabac9590204b268f758f8f24ed87be" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 39869 } } peers { permanent_uuid: "4285c138ac1d46fcb5810461505f8639" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 41261 } } peers { permanent_uuid: "073614a77a994b8fbed8f8cd6b7927de" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 38247 } attrs { promote: false } } }
I20260812 08:03:22.279237 19906 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260812 08:03:22.279685 20218 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:03:22.280071 20523 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |       Address       | Status
----------------------------------+---------------------+---------
 5dd9251f7ef54785a91610de46ec0292 | 127.0.186.126:35865 | HEALTHY

Unusual flags for Master:
                   Flag                   |                                                                                               Value                                                                                                |         Tags          |         Master
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
 ipki_ca_key_size                         | 768                                                                                                                                                                                                | experimental          | all 1 server(s) checked
 ipki_server_key_size                     | 768                                                                                                                                                                                                | experimental          | all 1 server(s) checked
 master_add_server_when_underreplicated   | true                                                                                                                                                                                               | hidden                | all 1 server(s) checked
 never_fsync                              | true                                                                                                                                                                                               | unsafe,advanced       | all 1 server(s) checked
 openssl_security_level_override          | 0                                                                                                                                                                                                  | unsafe,hidden         | all 1 server(s) checked
 raft_prepare_replacement_before_eviction | true                                                                                                                                                                                               | experimental,advanced | all 1 server(s) checked
 rpc_reuseport                            | true                                                                                                                                                                                               | experimental          | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports         | true                                                                                                                                                                                               | unsafe                | all 1 server(s) checked
 server_dump_info_format                  | pb                                                                                                                                                                                                 | hidden                | all 1 server(s) checked
 server_dump_info_path                    | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden                | all 1 server(s) checked
 tsk_num_rsa_bits                         | 512                                                                                                                                                                                                | experimental          | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |       Value        |         Master
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.0.186.84:35847 | all 1 server(s) checked
 time_source         | builtin            | all 1 server(s) checked

Tablet Server Summary
               UUID               |      Address       | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
 073614a77a994b8fbed8f8cd6b7927de | 127.0.186.66:38247 | HEALTHY | <none>   |       0        |       0
 4285c138ac1d46fcb5810461505f8639 | 127.0.186.65:41261 | HEALTHY | <none>   |       1        |       0
 7dabac9590204b268f758f8f24ed87be | 127.0.186.67:39869 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
                   Flag                   |                                                                                             Value                                                                                              |             Tags              |      Tablet Server
------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------+-------------------------
 ipki_server_key_size                     | 768                                                                                                                                                                                            | experimental                  | all 3 server(s) checked
 local_ip_for_outbound_sockets            | 127.0.186.65                                                                                                                                                                                   | experimental                  | 127.0.186.65:41261
 local_ip_for_outbound_sockets            | 127.0.186.66                                                                                                                                                                                   | experimental                  | 127.0.186.66:38247
 local_ip_for_outbound_sockets            | 127.0.186.67                                                                                                                                                                                   | experimental                  | 127.0.186.67:39869
 log_compression_codec                    | no_compression                                                                                                                                                                                 | experimental                  | all 3 server(s) checked
 log_max_segments_to_retain               | 3                                                                                                                                                                                              | experimental,runtime,advanced | all 3 server(s) checked
 log_target_replay_size_mb                | 1                                                                                                                                                                                              | experimental                  | all 3 server(s) checked
 maintenance_manager_polling_interval_ms  | 100                                                                                                                                                                                            | hidden                        | all 3 server(s) checked
 max_cell_size_bytes                      | 1000000                                                                                                                                                                                        | unsafe                        | all 3 server(s) checked
 never_fsync                              | true                                                                                                                                                                                           | unsafe,advanced               | all 3 server(s) checked
 openssl_security_level_override          | 0                                                                                                                                                                                              | unsafe,hidden                 | all 3 server(s) checked
 raft_prepare_replacement_before_eviction | true                                                                                                                                                                                           | experimental,advanced         | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports         | true                                                                                                                                                                                           | unsafe                        | all 3 server(s) checked
 server_dump_info_format                  | pb                                                                                                                                                                                             | hidden                        | all 3 server(s) checked
 server_dump_info_path                    | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden                        | 127.0.186.65:41261
 server_dump_info_path                    | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden                        | 127.0.186.66:38247
 server_dump_info_path                    | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_3.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden                        | 127.0.186.67:39869

Flags of checked categories for Tablet Server:
        Flag         |       Value        |      Tablet Server
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.0.186.84:35847 | all 3 server(s) checked
 time_source         | builtin            | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
   Name    | RF | Status  | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-----------+----+---------+---------------+---------+------------+------------------+-------------
 TestTable | 3  | HEALTHY | 1             | 1       | 0          | 0                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

OK
I20260812 08:03:22.303428   745 log_verifier.cc:126] Checking tablet 39f77f897dab4a528691252eb9a48c03
I20260812 08:03:22.308835   745 log_verifier.cc:177] Verified matching terms for 11 ops in tablet 39f77f897dab4a528691252eb9a48c03
I20260812 08:03:22.309501   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 19832
I20260812 08:03:22.323360   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 20448
I20260812 08:03:22.334095   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 20144
I20260812 08:03:22.347138   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 20614
2026-08-12T08:03:22Z chronyd exiting
[       OK ] ReplicaBehindWalGcThresholdITest.ReplicaReplacement/3 (21480 ms)
[ RUN      ] ReplicaBehindWalGcThresholdITest.ReplicaReplacement/4
I20260812 08:03:22.377792   745 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:03:22.377844   745 ts_itest-base.cc:116] --------------
I20260812 08:03:22.377861   745 ts_itest-base.cc:117] 3 tablet servers
I20260812 08:03:22.377873   745 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:03:22.377887   745 ts_itest-base.cc:119] --------------
2026-08-12T08:03:22Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:03:22Z Disabled control of system clock
I20260812 08:03:22.388195   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:37909
--webserver_interface=127.0.186.126
--webserver_port=0
--builtin_ntp_servers=127.0.186.84:41405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:37909
--raft_prepare_replacement_before_eviction=true with env {}
W20260812 08:03:22.499393 20808 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:22.499653 20808 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:22.499709 20808 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:22.503489 20808 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:03:22.503579 20808 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:22.503607 20808 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:03:22.503636 20808 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:03:22.507709 20808 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:41405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:37909
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:37909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.20808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:22.508945 20808 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:22.510174 20808 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:22.516263 20813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:22.516338 20815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:22.516340 20808 server_base.cc:1061] running on GCE node
W20260812 08:03:22.516263 20817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:22.517014 20808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:22.518103 20808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:22.519273 20808 hybrid_clock.cc:648] HybridClock initialized: now 1786521802519251 us; error 32 us; skew 500 ppm
I20260812 08:03:22.521354 20808 webserver.cc:533] Webserver started at http://127.0.186.126:41849/ using document root <none> and password file <none>
I20260812 08:03:22.522202 20808 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:22.522286 20808 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:22.522531 20808 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:22.524665 20808 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "42480246dfbf41ba84d75bddeadd4fe3"
format_stamp: "Formatted at 2026-08-12 08:03:22 on dist-test-slave-49h3"
I20260812 08:03:22.525352 20808 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "42480246dfbf41ba84d75bddeadd4fe3"
format_stamp: "Formatted at 2026-08-12 08:03:22 on dist-test-slave-49h3"
I20260812 08:03:22.530341 20808 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.000s
I20260812 08:03:22.533234 20825 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:22.534557 20808 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:03:22.534718 20808 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "42480246dfbf41ba84d75bddeadd4fe3"
format_stamp: "Formatted at 2026-08-12 08:03:22 on dist-test-slave-49h3"
I20260812 08:03:22.534860 20808 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:22.567129 20808 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:22.568049 20808 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:22.568257 20808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:22.576563 20808 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:37909
I20260812 08:03:22.576563 20894 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:37909 every 8 connection(s)
I20260812 08:03:22.577860 20808 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:03:22.581591 20895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:22.584264   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 20808
I20260812 08:03:22.584384   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:03:22.587296 20895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42480246dfbf41ba84d75bddeadd4fe3: Bootstrap starting.
I20260812 08:03:22.589789 20895 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42480246dfbf41ba84d75bddeadd4fe3: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:22.590739 20895 log.cc:826] T 00000000000000000000000000000000 P 42480246dfbf41ba84d75bddeadd4fe3: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:22.592898 20895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42480246dfbf41ba84d75bddeadd4fe3: No bootstrap required, opened a new log
I20260812 08:03:22.596277 20895 raft_consensus.cc:359] T 00000000000000000000000000000000 P 42480246dfbf41ba84d75bddeadd4fe3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42480246dfbf41ba84d75bddeadd4fe3" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 37909 } }
I20260812 08:03:22.596591 20895 raft_consensus.cc:385] T 00000000000000000000000000000000 P 42480246dfbf41ba84d75bddeadd4fe3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:22.596668 20895 raft_consensus.cc:740] T 00000000000000000000000000000000 P 42480246dfbf41ba84d75bddeadd4fe3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42480246dfbf41ba84d75bddeadd4fe3, State: Initialized, Role: FOLLOWER
I20260812 08:03:22.597182 20895 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42480246dfbf41ba84d75bddeadd4fe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42480246dfbf41ba84d75bddeadd4fe3" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 37909 } }
I20260812 08:03:22.597363 20895 raft_consensus.cc:399] T 00000000000000000000000000000000 P 42480246dfbf41ba84d75bddeadd4fe3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:03:22.597441 20895 raft_consensus.cc:493] T 00000000000000000000000000000000 P 42480246dfbf41ba84d75bddeadd4fe3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:03:22.597625 20895 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 42480246dfbf41ba84d75bddeadd4fe3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:22.598528 20895 raft_consensus.cc:515] T 00000000000000000000000000000000 P 42480246dfbf41ba84d75bddeadd4fe3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42480246dfbf41ba84d75bddeadd4fe3" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 37909 } }
I20260812 08:03:22.598941 20895 leader_election.cc:304] T 00000000000000000000000000000000 P 42480246dfbf41ba84d75bddeadd4fe3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42480246dfbf41ba84d75bddeadd4fe3; no voters: 
I20260812 08:03:22.599274 20895 leader_election.cc:290] T 00000000000000000000000000000000 P 42480246dfbf41ba84d75bddeadd4fe3 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:03:22.599404 20900 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 42480246dfbf41ba84d75bddeadd4fe3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:22.599614 20900 raft_consensus.cc:697] T 00000000000000000000000000000000 P 42480246dfbf41ba84d75bddeadd4fe3 [term 1 LEADER]: Becoming Leader. State: Replica: 42480246dfbf41ba84d75bddeadd4fe3, State: Running, Role: LEADER
I20260812 08:03:22.599995 20900 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42480246dfbf41ba84d75bddeadd4fe3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42480246dfbf41ba84d75bddeadd4fe3" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 37909 } }
I20260812 08:03:22.600536 20895 sys_catalog.cc:565] T 00000000000000000000000000000000 P 42480246dfbf41ba84d75bddeadd4fe3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:22.601850 20902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42480246dfbf41ba84d75bddeadd4fe3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42480246dfbf41ba84d75bddeadd4fe3. Latest consensus state: current_term: 1 leader_uuid: "42480246dfbf41ba84d75bddeadd4fe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42480246dfbf41ba84d75bddeadd4fe3" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 37909 } } }
I20260812 08:03:22.602010 20902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42480246dfbf41ba84d75bddeadd4fe3 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:22.602308 20901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42480246dfbf41ba84d75bddeadd4fe3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42480246dfbf41ba84d75bddeadd4fe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42480246dfbf41ba84d75bddeadd4fe3" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 37909 } } }
I20260812 08:03:22.602411 20901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42480246dfbf41ba84d75bddeadd4fe3 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:22.602689 20910 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:03:22.606084 20910 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:03:22.611442 20910 catalog_manager.cc:1383] Generated new cluster ID: 7cccb9f639d448dd8551d26798f1ed44
I20260812 08:03:22.611514 20910 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:03:22.653381 20910 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:03:22.654505 20910 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:03:22.667080 20910 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 42480246dfbf41ba84d75bddeadd4fe3: Generated new TSK 0
I20260812 08:03:22.667874 20910 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:03:22.673171   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:0
--local_ip_for_outbound_sockets=127.0.186.65
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:37909
--builtin_ntp_servers=127.0.186.84:41405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20260812 08:03:22.786791 20920 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:03:22.787070 20920 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:22.787111 20920 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:22.787175 20920 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:22.790535 20920 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:03:22.790596 20920 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:03:22.790640 20920 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:22.790725 20920 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:03:22.790758 20920 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:03:22.794692 20920 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:41405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:37909
--never_fsync=true
--heap_profile_path=/tmp/kudu.20920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:22.795897 20920 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:22.797099 20920 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 08:03:22.803918 20920 server_base.cc:1061] running on GCE node
W20260812 08:03:22.803854 20927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:22.803936 20926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:22.803854 20929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:22.804481 20920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:22.805066 20920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:22.806216 20920 hybrid_clock.cc:648] HybridClock initialized: now 1786521802806201 us; error 32 us; skew 500 ppm
I20260812 08:03:22.808183 20920 webserver.cc:533] Webserver started at http://127.0.186.65:43937/ using document root <none> and password file <none>
I20260812 08:03:22.808898 20920 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:22.808983 20920 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:22.809237 20920 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:22.811383 20920 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "f00ae2646b20468d80628fbe90a33c79"
format_stamp: "Formatted at 2026-08-12 08:03:22 on dist-test-slave-49h3"
I20260812 08:03:22.811942 20920 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "f00ae2646b20468d80628fbe90a33c79"
format_stamp: "Formatted at 2026-08-12 08:03:22 on dist-test-slave-49h3"
I20260812 08:03:22.815837 20920 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:03:22.818374 20935 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:22.819517 20920 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:03:22.819656 20920 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "f00ae2646b20468d80628fbe90a33c79"
format_stamp: "Formatted at 2026-08-12 08:03:22 on dist-test-slave-49h3"
I20260812 08:03:22.819762 20920 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:22.845162 20920 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:22.846130 20920 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:22.846367 20920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:22.847069 20920 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:22.848306 20920 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:22.848362 20920 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:22.848423 20920 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:22.848450 20920 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:22.857846 20920 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:40623
I20260812 08:03:22.857867 21068 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:40623 every 8 connection(s)
I20260812 08:03:22.858971 20920 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:03:22.861343   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 20920
I20260812 08:03:22.861493   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:03:22.863971   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.66:0
--local_ip_for_outbound_sockets=127.0.186.66
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:37909
--builtin_ntp_servers=127.0.186.84:41405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20260812 08:03:22.868953 21070 heartbeater.cc:344] Connected to a master server at 127.0.186.126:37909
I20260812 08:03:22.869253 21070 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:22.869871 21070 heartbeater.cc:507] Master 127.0.186.126:37909 requested a full tablet report, sending...
I20260812 08:03:22.871471 20847 ts_manager.cc:194] Registered new tserver with Master: f00ae2646b20468d80628fbe90a33c79 (127.0.186.65:40623)
I20260812 08:03:22.872653 20847 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.65:40347
W20260812 08:03:22.974531 21075 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:03:22.974844 21075 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:22.974884 21075 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:22.974948 21075 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:22.978569 21075 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:03:22.978655 21075 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:03:22.978700 21075 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:22.978780 21075 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:03:22.978816 21075 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.66
I20260812 08:03:22.983172 21075 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:41405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:37909
--never_fsync=true
--heap_profile_path=/tmp/kudu.21075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:22.984417 21075 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:22.985646 21075 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:22.991920 21085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:22.991925 21081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:22.992084 21082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:22.992137 21075 server_base.cc:1061] running on GCE node
I20260812 08:03:22.992511 21075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:22.992995 21075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:22.994166 21075 hybrid_clock.cc:648] HybridClock initialized: now 1786521802994152 us; error 25 us; skew 500 ppm
I20260812 08:03:22.995748 21075 webserver.cc:533] Webserver started at http://127.0.186.66:46223/ using document root <none> and password file <none>
I20260812 08:03:22.996273 21075 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:22.996333 21075 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:22.996507 21075 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:22.998044 21075 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "416b6427e9244877a46ab232ff39f8cd"
format_stamp: "Formatted at 2026-08-12 08:03:22 on dist-test-slave-49h3"
I20260812 08:03:22.998489 21075 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "416b6427e9244877a46ab232ff39f8cd"
format_stamp: "Formatted at 2026-08-12 08:03:22 on dist-test-slave-49h3"
I20260812 08:03:23.001889 21075 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:03:23.004062 21091 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:23.004987 21075 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:23.005125 21075 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "416b6427e9244877a46ab232ff39f8cd"
format_stamp: "Formatted at 2026-08-12 08:03:22 on dist-test-slave-49h3"
I20260812 08:03:23.005209 21075 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:23.054484 21075 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:23.055205 21075 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:23.055387 21075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:23.056049 21075 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:23.057153 21075 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:23.057205 21075 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:23.057261 21075 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:23.057289 21075 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:23.066674 21075 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.66:36375
I20260812 08:03:23.066710 21239 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.66:36375 every 8 connection(s)
I20260812 08:03:23.067675 21075 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:03:23.071952   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 21075
I20260812 08:03:23.072082   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:03:23.074620   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.67:0
--local_ip_for_outbound_sockets=127.0.186.67
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:37909
--builtin_ntp_servers=127.0.186.84:41405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20260812 08:03:23.076876 21242 heartbeater.cc:344] Connected to a master server at 127.0.186.126:37909
I20260812 08:03:23.077184 21242 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:23.077962 21242 heartbeater.cc:507] Master 127.0.186.126:37909 requested a full tablet report, sending...
I20260812 08:03:23.079198 20847 ts_manager.cc:194] Registered new tserver with Master: 416b6427e9244877a46ab232ff39f8cd (127.0.186.66:36375)
I20260812 08:03:23.079780 20847 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.66:60615
W20260812 08:03:23.180733 21245 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:03:23.180954 21245 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:23.180991 21245 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:23.181051 21245 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:23.184286 21245 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:03:23.184340 21245 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:03:23.184378 21245 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:23.184463 21245 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:03:23.184494 21245 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.67
I20260812 08:03:23.188266 21245 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:41405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:37909
--never_fsync=true
--heap_profile_path=/tmp/kudu.21245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:23.189273 21245 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:23.190382 21245 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:23.196882 21257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:23.196874 21252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:23.196935 21245 server_base.cc:1061] running on GCE node
W20260812 08:03:23.196874 21255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:23.197388 21245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:23.197930 21245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:23.199076 21245 hybrid_clock.cc:648] HybridClock initialized: now 1786521803199058 us; error 28 us; skew 500 ppm
I20260812 08:03:23.200681 21245 webserver.cc:533] Webserver started at http://127.0.186.67:33877/ using document root <none> and password file <none>
I20260812 08:03:23.201248 21245 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:23.201306 21245 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:23.201545 21245 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:23.203119 21245 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "a451eb9951d54df2b69f0c01b98e1b8d"
format_stamp: "Formatted at 2026-08-12 08:03:23 on dist-test-slave-49h3"
I20260812 08:03:23.203565 21245 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "a451eb9951d54df2b69f0c01b98e1b8d"
format_stamp: "Formatted at 2026-08-12 08:03:23 on dist-test-slave-49h3"
I20260812 08:03:23.206668 21245 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:03:23.208680 21265 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:23.209648 21245 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:03:23.209744 21245 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "a451eb9951d54df2b69f0c01b98e1b8d"
format_stamp: "Formatted at 2026-08-12 08:03:23 on dist-test-slave-49h3"
I20260812 08:03:23.209854 21245 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:23.229599 21245 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:23.230204 21245 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:23.230370 21245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:23.230928 21245 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:23.231981 21245 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:23.232030 21245 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:23.232074 21245 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:23.232105 21245 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:23.240629 21245 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.67:36983
I20260812 08:03:23.240662 21403 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.67:36983 every 8 connection(s)
I20260812 08:03:23.241590 21245 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:03:23.250061 21404 heartbeater.cc:344] Connected to a master server at 127.0.186.126:37909
I20260812 08:03:23.250313 21404 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:23.250809 21404 heartbeater.cc:507] Master 127.0.186.126:37909 requested a full tablet report, sending...
I20260812 08:03:23.251741   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 21245
I20260812 08:03:23.251821 20847 ts_manager.cc:194] Registered new tserver with Master: a451eb9951d54df2b69f0c01b98e1b8d (127.0.186.67:36983)
I20260812 08:03:23.251861   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:03:23.252372 20847 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.67:38461
I20260812 08:03:23.254150   745 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:03:23.262012 20847 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41620:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
W20260812 08:03:23.264367 20847 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 08:03:23.283195 21324 tablet_service.cc:1511] Processing CreateTablet for tablet 926dba79b58c42b78ca97f6c5f1032c9 (DEFAULT_TABLE table=TestTable [id=2c2ede4592a049e5a67af8eb14d0cba5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:23.283195 20991 tablet_service.cc:1511] Processing CreateTablet for tablet 926dba79b58c42b78ca97f6c5f1032c9 (DEFAULT_TABLE table=TestTable [id=2c2ede4592a049e5a67af8eb14d0cba5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:23.284298 21324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 926dba79b58c42b78ca97f6c5f1032c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:23.284298 20991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 926dba79b58c42b78ca97f6c5f1032c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:23.288131 21155 tablet_service.cc:1511] Processing CreateTablet for tablet 926dba79b58c42b78ca97f6c5f1032c9 (DEFAULT_TABLE table=TestTable [id=2c2ede4592a049e5a67af8eb14d0cba5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:23.288964 21155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 926dba79b58c42b78ca97f6c5f1032c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:23.290277 21424 tablet_bootstrap.cc:492] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79: Bootstrap starting.
I20260812 08:03:23.290683 21425 tablet_bootstrap.cc:492] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d: Bootstrap starting.
I20260812 08:03:23.293089 21425 tablet_bootstrap.cc:654] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:23.293988 21425 log.cc:826] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:23.294399 21426 tablet_bootstrap.cc:492] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: Bootstrap starting.
I20260812 08:03:23.296273 21426 tablet_bootstrap.cc:654] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:23.296407 21424 tablet_bootstrap.cc:654] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:23.296916 21426 log.cc:826] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:23.297056 21424 log.cc:826] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:23.298427 21426 tablet_bootstrap.cc:492] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: No bootstrap required, opened a new log
I20260812 08:03:23.298518 21425 tablet_bootstrap.cc:492] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d: No bootstrap required, opened a new log
I20260812 08:03:23.298549 21424 tablet_bootstrap.cc:492] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79: No bootstrap required, opened a new log
I20260812 08:03:23.298617 21426 ts_tablet_manager.cc:1403] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: Time spent bootstrapping tablet: real 0.004s	user 0.003s	sys 0.000s
I20260812 08:03:23.298707 21424 ts_tablet_manager.cc:1403] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79: Time spent bootstrapping tablet: real 0.009s	user 0.003s	sys 0.000s
I20260812 08:03:23.298708 21425 ts_tablet_manager.cc:1403] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d: Time spent bootstrapping tablet: real 0.008s	user 0.004s	sys 0.000s
I20260812 08:03:23.301096 21426 raft_consensus.cc:359] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 36375 } }
I20260812 08:03:23.301319 21426 raft_consensus.cc:385] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:23.301394 21426 raft_consensus.cc:740] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 416b6427e9244877a46ab232ff39f8cd, State: Initialized, Role: FOLLOWER
I20260812 08:03:23.301677 21425 raft_consensus.cc:359] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 36375 } }
I20260812 08:03:23.301681 21424 raft_consensus.cc:359] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 36375 } }
I20260812 08:03:23.301875 21424 raft_consensus.cc:385] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:23.301875 21425 raft_consensus.cc:385] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:23.301935 21424 raft_consensus.cc:740] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00ae2646b20468d80628fbe90a33c79, State: Initialized, Role: FOLLOWER
I20260812 08:03:23.301934 21425 raft_consensus.cc:740] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a451eb9951d54df2b69f0c01b98e1b8d, State: Initialized, Role: FOLLOWER
I20260812 08:03:23.301949 21426 consensus_queue.cc:260] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 36375 } }
I20260812 08:03:23.302345 21424 consensus_queue.cc:260] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 36375 } }
I20260812 08:03:23.302345 21425 consensus_queue.cc:260] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 36375 } }
I20260812 08:03:23.302781 21426 ts_tablet_manager.cc:1434] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:03:23.303171 21424 ts_tablet_manager.cc:1434] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:03:23.303171 21425 ts_tablet_manager.cc:1434] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:03:23.303187 21242 heartbeater.cc:499] Master 127.0.186.126:37909 was elected leader, sending a full tablet report...
I20260812 08:03:23.303289 21070 heartbeater.cc:499] Master 127.0.186.126:37909 was elected leader, sending a full tablet report...
I20260812 08:03:23.304180 21404 heartbeater.cc:499] Master 127.0.186.126:37909 was elected leader, sending a full tablet report...
I20260812 08:03:23.313114 21430 raft_consensus.cc:493] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:23.313258 21430 raft_consensus.cc:515] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 36375 } }
I20260812 08:03:23.314257 21430 leader_election.cc:290] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f00ae2646b20468d80628fbe90a33c79 (127.0.186.65:40623), a451eb9951d54df2b69f0c01b98e1b8d (127.0.186.67:36983)
I20260812 08:03:23.318017 21017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926dba79b58c42b78ca97f6c5f1032c9" candidate_uuid: "416b6427e9244877a46ab232ff39f8cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00ae2646b20468d80628fbe90a33c79" is_pre_election: true
I20260812 08:03:23.318301 21017 raft_consensus.cc:2468] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 416b6427e9244877a46ab232ff39f8cd in term 0.
I20260812 08:03:23.318619 21350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926dba79b58c42b78ca97f6c5f1032c9" candidate_uuid: "416b6427e9244877a46ab232ff39f8cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" is_pre_election: true
I20260812 08:03:23.318776 21093 leader_election.cc:304] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 416b6427e9244877a46ab232ff39f8cd, f00ae2646b20468d80628fbe90a33c79; no voters: 
I20260812 08:03:23.318852 21350 raft_consensus.cc:2468] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 416b6427e9244877a46ab232ff39f8cd in term 0.
I20260812 08:03:23.319008 21430 raft_consensus.cc:2804] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:03:23.319094 21430 raft_consensus.cc:493] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:03:23.319149 21430 raft_consensus.cc:3060] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:23.319975 21430 raft_consensus.cc:515] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 36375 } }
I20260812 08:03:23.320318 21430 leader_election.cc:290] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [CANDIDATE]: Term 1 election: Requested vote from peers f00ae2646b20468d80628fbe90a33c79 (127.0.186.65:40623), a451eb9951d54df2b69f0c01b98e1b8d (127.0.186.67:36983)
I20260812 08:03:23.320652 21350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926dba79b58c42b78ca97f6c5f1032c9" candidate_uuid: "416b6427e9244877a46ab232ff39f8cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a451eb9951d54df2b69f0c01b98e1b8d"
I20260812 08:03:23.320638 21017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926dba79b58c42b78ca97f6c5f1032c9" candidate_uuid: "416b6427e9244877a46ab232ff39f8cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00ae2646b20468d80628fbe90a33c79"
I20260812 08:03:23.320758 21350 raft_consensus.cc:3060] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:23.320758 21017 raft_consensus.cc:3060] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:23.322221 21017 raft_consensus.cc:2468] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 416b6427e9244877a46ab232ff39f8cd in term 1.
I20260812 08:03:23.322221 21350 raft_consensus.cc:2468] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 416b6427e9244877a46ab232ff39f8cd in term 1.
I20260812 08:03:23.322535 21093 leader_election.cc:304] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 416b6427e9244877a46ab232ff39f8cd, a451eb9951d54df2b69f0c01b98e1b8d, f00ae2646b20468d80628fbe90a33c79; no voters: 
I20260812 08:03:23.322708 21430 raft_consensus.cc:2804] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:23.322916 21430 raft_consensus.cc:697] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 1 LEADER]: Becoming Leader. State: Replica: 416b6427e9244877a46ab232ff39f8cd, State: Running, Role: LEADER
I20260812 08:03:23.323235 21430 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 36375 } }
I20260812 08:03:23.326283 20847 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 416b6427e9244877a46ab232ff39f8cd (127.0.186.66). New cstate: current_term: 1 leader_uuid: "416b6427e9244877a46ab232ff39f8cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 36375 } health_report { overall_health: HEALTHY } } }
W20260812 08:03:23.342425 21405 tablet.cc:2406] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:03:23.342957   745 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:03:23.344336   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver f00ae2646b20468d80628fbe90a33c79 to finish bootstrapping
I20260812 08:03:23.349674   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 416b6427e9244877a46ab232ff39f8cd to finish bootstrapping
I20260812 08:03:23.354430   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver a451eb9951d54df2b69f0c01b98e1b8d to finish bootstrapping
W20260812 08:03:23.360754 21071 tablet.cc:2406] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:03:23.368939 21243 tablet.cc:2406] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:03:23.369228   745 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 1.1 0.0
I20260812 08:03:23.472121   745 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 1.1 0.0
I20260812 08:03:23.675200   745 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 1.1 0.0
I20260812 08:03:23.757144 21430 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:23.762724 21430 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:23.978226   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 21075
I20260812 08:03:25.262194 21449 raft_consensus.cc:493] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 416b6427e9244877a46ab232ff39f8cd)
I20260812 08:03:25.262354 21449 raft_consensus.cc:515] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 36375 } }
I20260812 08:03:25.263453 21449 leader_election.cc:290] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f00ae2646b20468d80628fbe90a33c79 (127.0.186.65:40623), 416b6427e9244877a46ab232ff39f8cd (127.0.186.66:36375)
W20260812 08:03:25.264344 21269 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.66:36375: connect: Connection refused (error 111)
I20260812 08:03:25.264853 21455 raft_consensus.cc:493] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 416b6427e9244877a46ab232ff39f8cd)
I20260812 08:03:25.265033 21455 raft_consensus.cc:515] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 36375 } }
W20260812 08:03:25.265436 21269 leader_election.cc:336] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 416b6427e9244877a46ab232ff39f8cd (127.0.186.66:36375): Network error: Client connection negotiation failed: client connection to 127.0.186.66:36375: connect: Connection refused (error 111)
I20260812 08:03:25.266196 21455 leader_election.cc:290] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a451eb9951d54df2b69f0c01b98e1b8d (127.0.186.67:36983), 416b6427e9244877a46ab232ff39f8cd (127.0.186.66:36375)
W20260812 08:03:25.267109 20941 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.66:36375: connect: Connection refused (error 111)
I20260812 08:03:25.267753 21017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926dba79b58c42b78ca97f6c5f1032c9" candidate_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f00ae2646b20468d80628fbe90a33c79" is_pre_election: true
I20260812 08:03:25.267896 21017 raft_consensus.cc:2468] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a451eb9951d54df2b69f0c01b98e1b8d in term 1.
W20260812 08:03:25.267976 20941 leader_election.cc:336] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 416b6427e9244877a46ab232ff39f8cd (127.0.186.66:36375): Network error: Client connection negotiation failed: client connection to 127.0.186.66:36375: connect: Connection refused (error 111)
I20260812 08:03:25.268453 21266 leader_election.cc:304] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a451eb9951d54df2b69f0c01b98e1b8d, f00ae2646b20468d80628fbe90a33c79; no voters: 416b6427e9244877a46ab232ff39f8cd
I20260812 08:03:25.268697 21449 raft_consensus.cc:2804] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 08:03:25.268755 21449 raft_consensus.cc:493] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 416b6427e9244877a46ab232ff39f8cd)
I20260812 08:03:25.268800 21449 raft_consensus.cc:3060] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:03:25.269826 21449 raft_consensus.cc:515] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 36375 } }
I20260812 08:03:25.270170 21449 leader_election.cc:290] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [CANDIDATE]: Term 2 election: Requested vote from peers f00ae2646b20468d80628fbe90a33c79 (127.0.186.65:40623), 416b6427e9244877a46ab232ff39f8cd (127.0.186.66:36375)
I20260812 08:03:25.270179 21350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926dba79b58c42b78ca97f6c5f1032c9" candidate_uuid: "f00ae2646b20468d80628fbe90a33c79" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" is_pre_election: true
I20260812 08:03:25.270375 21350 raft_consensus.cc:2393] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f00ae2646b20468d80628fbe90a33c79 in current term 2: Already voted for candidate a451eb9951d54df2b69f0c01b98e1b8d in this term.
I20260812 08:03:25.270462 21017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926dba79b58c42b78ca97f6c5f1032c9" candidate_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f00ae2646b20468d80628fbe90a33c79"
I20260812 08:03:25.270574 21017 raft_consensus.cc:3060] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 1 FOLLOWER]: Advancing to term 2
W20260812 08:03:25.271190 21269 leader_election.cc:336] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 416b6427e9244877a46ab232ff39f8cd (127.0.186.66:36375): Network error: Client connection negotiation failed: client connection to 127.0.186.66:36375: connect: Connection refused (error 111)
I20260812 08:03:25.271497 21017 raft_consensus.cc:2468] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a451eb9951d54df2b69f0c01b98e1b8d in term 2.
I20260812 08:03:25.271809 21266 leader_election.cc:304] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a451eb9951d54df2b69f0c01b98e1b8d, f00ae2646b20468d80628fbe90a33c79; no voters: 416b6427e9244877a46ab232ff39f8cd
I20260812 08:03:25.271871 20941 leader_election.cc:304] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f00ae2646b20468d80628fbe90a33c79; no voters: 416b6427e9244877a46ab232ff39f8cd, a451eb9951d54df2b69f0c01b98e1b8d
I20260812 08:03:25.272027 21449 raft_consensus.cc:2804] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:03:25.272092 21455 raft_consensus.cc:2749] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 08:03:25.272238 21449 raft_consensus.cc:697] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Becoming Leader. State: Replica: a451eb9951d54df2b69f0c01b98e1b8d, State: Running, Role: LEADER
I20260812 08:03:25.272526 21449 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 36375 } }
I20260812 08:03:25.275002 20845 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: term changed from 1 to 2, leader changed from 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) to a451eb9951d54df2b69f0c01b98e1b8d (127.0.186.67). New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 36375 } health_report { overall_health: UNKNOWN } } }
I20260812 08:03:25.527372   745 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
I20260812 08:03:25.527698   745 raft_consensus-itest-base.cc:268] Waiting until we've written at least 4MB...
I20260812 08:03:25.542128 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260812 08:03:25.542701 21461 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260812 08:03:25.542958 21269 consensus_peers.cc:597] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d -> Peer 416b6427e9244877a46ab232ff39f8cd (127.0.186.66:36375): Couldn't send request to peer 416b6427e9244877a46ab232ff39f8cd. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.66:36375: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 08:03:25.552349 21483 mvcc.cc:204] Tried to move back new op lower bound from 7317593315499327488 to 7317593314398576640. Current Snapshot: MvccSnapshot[applied={T|T < 7317593315493265408}]
I20260812 08:03:25.624384   745 raft_consensus-itest-base.cc:274] Waiting for log GC on a451eb9951d54df2b69f0c01b98e1b8d
I20260812 08:03:25.651697 21405 maintenance_manager.cc:419] P a451eb9951d54df2b69f0c01b98e1b8d: Scheduling FlushMRSOp(926dba79b58c42b78ca97f6c5f1032c9): 4602274 bytes log retention, and flush 6119689 bytes memory
I20260812 08:03:25.670064 21071 maintenance_manager.cc:419] P f00ae2646b20468d80628fbe90a33c79: Scheduling FlushMRSOp(926dba79b58c42b78ca97f6c5f1032c9): 3944841 bytes log retention, and flush 6119689 bytes memory
I20260812 08:03:25.709534 21270 maintenance_manager.cc:643] P a451eb9951d54df2b69f0c01b98e1b8d: FlushMRSOp(926dba79b58c42b78ca97f6c5f1032c9) complete. Timing: real 0.057s	user 0.052s	sys 0.004s Metrics: {"bytes_written":136001,"cfile_init":1,"compiler_manager_pool.queue_time_us":292,"drs_written":1,"lbm_read_time_us":67,"lbm_reads_lt_1ms":4,"lbm_write_time_us":649,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":36,"thread_start_us":265,"threads_started":2}
I20260812 08:03:25.710788 21405 maintenance_manager.cc:419] P a451eb9951d54df2b69f0c01b98e1b8d: Scheduling LogGCOp(926dba79b58c42b78ca97f6c5f1032c9): free 2761362 bytes of WAL
I20260812 08:03:25.711102 21270 log_reader.cc:385] T 926dba79b58c42b78ca97f6c5f1032c9: removed 3 log segments from log reader
I20260812 08:03:25.711206 21270 log.cc:1079] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/926dba79b58c42b78ca97f6c5f1032c9/wal-000000001 (ops 1-9)
I20260812 08:03:25.711310 21270 log.cc:1079] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/926dba79b58c42b78ca97f6c5f1032c9/wal-000000002 (ops 10-16)
I20260812 08:03:25.711387 21270 log.cc:1079] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/926dba79b58c42b78ca97f6c5f1032c9/wal-000000003 (ops 17-23)
I20260812 08:03:25.712355 21270 maintenance_manager.cc:643] P a451eb9951d54df2b69f0c01b98e1b8d: LogGCOp(926dba79b58c42b78ca97f6c5f1032c9) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 08:03:25.712836 21405 maintenance_manager.cc:419] P a451eb9951d54df2b69f0c01b98e1b8d: Scheduling UndoDeltaBlockGCOp(926dba79b58c42b78ca97f6c5f1032c9): 534 bytes on disk
I20260812 08:03:25.713464 21270 maintenance_manager.cc:643] P a451eb9951d54df2b69f0c01b98e1b8d: UndoDeltaBlockGCOp(926dba79b58c42b78ca97f6c5f1032c9) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":65,"lbm_reads_lt_1ms":4}
I20260812 08:03:25.716677   745 raft_consensus-itest-base.cc:279] Log GC complete on a451eb9951d54df2b69f0c01b98e1b8d
I20260812 08:03:25.728740 20942 maintenance_manager.cc:643] P f00ae2646b20468d80628fbe90a33c79: FlushMRSOp(926dba79b58c42b78ca97f6c5f1032c9) complete. Timing: real 0.058s	user 0.052s	sys 0.000s Metrics: {"bytes_written":136001,"cfile_init":1,"compiler_manager_pool.queue_time_us":1922,"dirs.queue_time_us":250,"dirs.run_cpu_time_us":194,"dirs.run_wall_time_us":642,"drs_written":1,"lbm_read_time_us":60,"lbm_reads_lt_1ms":4,"lbm_write_time_us":540,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":36,"thread_start_us":283,"threads_started":2}
I20260812 08:03:25.729730 21071 maintenance_manager.cc:419] P f00ae2646b20468d80628fbe90a33c79: Scheduling LogGCOp(926dba79b58c42b78ca97f6c5f1032c9): free 2366935 bytes of WAL
I20260812 08:03:25.729990 20942 log_reader.cc:385] T 926dba79b58c42b78ca97f6c5f1032c9: removed 3 log segments from log reader
I20260812 08:03:25.730047 20942 log.cc:1079] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/926dba79b58c42b78ca97f6c5f1032c9/wal-000000001 (ops 1-8)
I20260812 08:03:25.730109 20942 log.cc:1079] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/926dba79b58c42b78ca97f6c5f1032c9/wal-000000002 (ops 9-15)
I20260812 08:03:25.730146 20942 log.cc:1079] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/926dba79b58c42b78ca97f6c5f1032c9/wal-000000003 (ops 16-20)
I20260812 08:03:25.730808 20942 maintenance_manager.cc:643] P f00ae2646b20468d80628fbe90a33c79: LogGCOp(926dba79b58c42b78ca97f6c5f1032c9) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 08:03:25.731072 21071 maintenance_manager.cc:419] P f00ae2646b20468d80628fbe90a33c79: Scheduling UndoDeltaBlockGCOp(926dba79b58c42b78ca97f6c5f1032c9): 534 bytes on disk
I20260812 08:03:25.731549 20942 maintenance_manager.cc:643] P f00ae2646b20468d80628fbe90a33c79: UndoDeltaBlockGCOp(926dba79b58c42b78ca97f6c5f1032c9) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":50,"lbm_reads_lt_1ms":4}
I20260812 08:03:26.145259 21498 consensus_queue.cc:753] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..31: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:26.145382 21498 consensus_queue.cc:579] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The replica will never be able to catch up
I20260812 08:03:26.148654 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 38, Committed index: 38, Last appended: 2.38, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } }
I20260812 08:03:26.149888 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 38. Preceding OpId from leader: term: 2 index: 39. (index mismatch)
I20260812 08:03:26.150326 21498 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 39, Last known committed idx: 38, Time since last communication: 0.000s
I20260812 08:03:26.151906 21503 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.39: config changed from index -1 to 39, VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:26.152330 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.39: config changed from index -1 to 39, VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:26.153481 20830 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:26.154465 20847 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index -1 to 39, VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } }
W20260812 08:03:26.157799 20830 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.66:36375: connect: Connection refused (error 111)
I20260812 08:03:26.158222 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 39, Committed index: 39, Last appended: 2.39, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } }
W20260812 08:03:26.158811 20830 catalog_manager.cc:4777] TS 416b6427e9244877a46ab232ff39f8cd (127.0.186.66:36375): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 926dba79b58c42b78ca97f6c5f1032c9: Network error: Client connection negotiation failed: client connection to 127.0.186.66:36375: connect: Connection refused (error 111)
I20260812 08:03:26.159598 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 39. Preceding OpId from leader: term: 2 index: 40. (index mismatch)
I20260812 08:03:26.159994 21503 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 39, Time since last communication: 0.000s
W20260812 08:03:26.160198 21269 consensus_peers.cc:597] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d -> Peer 416b6427e9244877a46ab232ff39f8cd (127.0.186.66:36375): Couldn't send request to peer 416b6427e9244877a46ab232ff39f8cd. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.66:36375: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 08:03:26.161562 21498 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.40: config changed from index 39 to 40, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:26.161994 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.40: config changed from index 39 to 40, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:26.162917 20830 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 39: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:26.163892 20847 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 39 to 40, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260812 08:03:27.371141 20830 catalog_manager.cc:4777] TS 416b6427e9244877a46ab232ff39f8cd (127.0.186.66:36375): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 926dba79b58c42b78ca97f6c5f1032c9: Network error: Client connection negotiation failed: client connection to 127.0.186.66:36375: connect: Connection refused (error 111)
I20260812 08:03:27.718299   745 raft_consensus-itest-base.cc:295] Servers converged with original term 2
I20260812 08:03:27.723565   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.66:36375
--local_ip_for_outbound_sockets=127.0.186.66
--tserver_master_addrs=127.0.186.126:37909
--webserver_port=46223
--webserver_interface=127.0.186.66
--builtin_ntp_servers=127.0.186.84:41405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20260812 08:03:27.837591 21518 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:03:27.837850 21518 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:27.837874 21518 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:27.837939 21518 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:27.841320 21518 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:03:27.841374 21518 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:03:27.841406 21518 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:27.841518 21518 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:03:27.841552 21518 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.66
I20260812 08:03:27.845463 21518 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:41405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.66:36375
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.186.66
--webserver_port=46223
--tserver_master_addrs=127.0.186.126:37909
--never_fsync=true
--heap_profile_path=/tmp/kudu.21518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:27.846663 21518 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:27.847890 21518 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:27.854858 21528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:27.854859 21525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:27.854984 21518 server_base.cc:1061] running on GCE node
W20260812 08:03:27.854907 21526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:27.855501 21518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:27.856076 21518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:27.857232 21518 hybrid_clock.cc:648] HybridClock initialized: now 1786521807857213 us; error 34 us; skew 500 ppm
I20260812 08:03:27.859282 21518 webserver.cc:533] Webserver started at http://127.0.186.66:46223/ using document root <none> and password file <none>
I20260812 08:03:27.860147 21518 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:27.860280 21518 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:27.864475 21518 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.002s	sys 0.000s
I20260812 08:03:27.867067 21535 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:27.868391 21518 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260812 08:03:27.868530 21518 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "416b6427e9244877a46ab232ff39f8cd"
format_stamp: "Formatted at 2026-08-12 08:03:22 on dist-test-slave-49h3"
I20260812 08:03:27.869017 21518 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:27.892974 21518 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:27.893728 21518 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:27.893895 21518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:27.894528 21518 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:27.895819 21544 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:03:27.899595 21518 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:27.899688 21518 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:03:27.899758 21518 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:27.901580 21518 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:27.901635 21518 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:27.902060 21544 tablet_bootstrap.cc:492] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: Bootstrap starting.
I20260812 08:03:27.911129 21544 log.cc:826] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:27.912683 21518 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.66:36375
I20260812 08:03:27.912839 21674 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.66:36375 every 8 connection(s)
I20260812 08:03:27.913785 21518 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:03:27.914325 21544 tablet_bootstrap.cc:492] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:03:27.914947 21544 tablet_bootstrap.cc:492] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: Bootstrap complete.
I20260812 08:03:27.915438 21544 ts_tablet_manager.cc:1403] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: Time spent bootstrapping tablet: real 0.014s	user 0.012s	sys 0.000s
I20260812 08:03:27.918423 21544 raft_consensus.cc:359] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 36375 } }
I20260812 08:03:27.918767 21544 raft_consensus.cc:740] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 416b6427e9244877a46ab232ff39f8cd, State: Initialized, Role: FOLLOWER
I20260812 08:03:27.919219 21544 consensus_queue.cc:260] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 36375 } }
I20260812 08:03:27.920068 21544 ts_tablet_manager.cc:1434] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 08:03:27.921483   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 21518
I20260812 08:03:27.926108 21676 heartbeater.cc:344] Connected to a master server at 127.0.186.126:37909
I20260812 08:03:27.926327 21676 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:27.926808 21676 heartbeater.cc:507] Master 127.0.186.126:37909 requested a full tablet report, sending...
I20260812 08:03:27.928584 20847 ts_manager.cc:194] Re-registered known tserver with Master: 416b6427e9244877a46ab232ff39f8cd (127.0.186.66:36375)
I20260812 08:03:27.929332 20847 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.66:58013
I20260812 08:03:27.931059 21676 heartbeater.cc:499] Master 127.0.186.126:37909 was elected leader, sending a full tablet report...
I20260812 08:03:28.170360 21620 raft_consensus.cc:3060] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:03:28.172973 21620 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 39. (index mismatch)
I20260812 08:03:28.174193 21498 consensus_queue.cc:1050] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Got LMP mismatch error from peer: Peer: permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:28.174510 21498 consensus_queue.cc:753] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..39: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:28.174566 21498 consensus_queue.cc:579] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The replica will never be able to catch up
I20260812 08:03:28.177268 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 2.40, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } }
I20260812 08:03:28.178722 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 40. Preceding OpId from leader: term: 2 index: 41. (index mismatch)
I20260812 08:03:28.179183 21498 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 41, Last known committed idx: 40, Time since last communication: 0.000s
I20260812 08:03:28.181128 21685 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.41: config changed from index 40 to 41, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.181728 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.41: config changed from index 40 to 41, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.182667 20830 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 40: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:28.183769 20847 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 40 to 41, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:28.187872 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 41, Committed index: 41, Last appended: 2.41, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } }
I20260812 08:03:28.189424 21620 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 42. (index mismatch)
I20260812 08:03:28.189572 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 41. Preceding OpId from leader: term: 2 index: 42. (index mismatch)
I20260812 08:03:28.189944 21498 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 42, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:28.190203 21498 consensus_queue.cc:753] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..41: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:28.190258 21498 consensus_queue.cc:579] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The replica will never be able to catch up
I20260812 08:03:28.190443 21685 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 42, Last known committed idx: 41, Time since last communication: 0.000s
I20260812 08:03:28.192414 21498 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.42: config changed from index 41 to 42, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.192960 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.42: config changed from index 41 to 42, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.194031 20830 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 41: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:28.194864 20847 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 41 to 42, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } health_report { overall_health: FAILED_UNRECOVERABLE } } }
I20260812 08:03:28.197633 21350 logging.cc:436] LogThrottler TrackedPeer WAL GC: suppressed but not reported on 1 messages since previous log ~0 seconds ago
I20260812 08:03:28.197815 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 42, Committed index: 42, Last appended: 2.42, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } }
I20260812 08:03:28.198942 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 42. Preceding OpId from leader: term: 2 index: 43. (index mismatch)
I20260812 08:03:28.199396 21695 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 43, Last known committed idx: 42, Time since last communication: 0.000s
I20260812 08:03:28.200802 21498 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.43: config changed from index 42 to 43, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.201279 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.43: config changed from index 42 to 43, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.202234 20830 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 42: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:28.203295 20846 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 42 to 43, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:28.206275 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 43, Committed index: 43, Last appended: 2.43, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } }
I20260812 08:03:28.207578 21620 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 44. (index mismatch)
I20260812 08:03:28.207686 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 43. Preceding OpId from leader: term: 2 index: 44. (index mismatch)
I20260812 08:03:28.208002 21695 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 44, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:28.208206 21498 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 44, Last known committed idx: 43, Time since last communication: 0.000s
I20260812 08:03:28.208268 21695 consensus_queue.cc:753] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..43: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:28.208334 21695 consensus_queue.cc:579] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The replica will never be able to catch up
I20260812 08:03:28.209975 21695 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.44: config changed from index 43 to 44, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.210565 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.44: config changed from index 43 to 44, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.211926 20830 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 43: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:28.212726 20846 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 43 to 44, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } health_report { overall_health: FAILED_UNRECOVERABLE } } }
I20260812 08:03:28.215765 21350 logging.cc:436] LogThrottler TrackedPeer WAL GC: suppressed but not reported on 1 messages since previous log ~0 seconds ago
I20260812 08:03:28.215931 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 44, Committed index: 44, Last appended: 2.44, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } }
I20260812 08:03:28.216984 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 44. Preceding OpId from leader: term: 2 index: 45. (index mismatch)
I20260812 08:03:28.217551 21498 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 45, Last known committed idx: 44, Time since last communication: 0.000s
I20260812 08:03:28.219034 21695 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.45: config changed from index 44 to 45, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.219480 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.45: config changed from index 44 to 45, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.220268 20830 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 44: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:28.221310 20846 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 44 to 45, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:28.223739 21405 maintenance_manager.cc:419] P a451eb9951d54df2b69f0c01b98e1b8d: Scheduling LogGCOp(926dba79b58c42b78ca97f6c5f1032c9): free 1840912 bytes of WAL
I20260812 08:03:28.223954 21270 log_reader.cc:385] T 926dba79b58c42b78ca97f6c5f1032c9: removed 2 log segments from log reader
I20260812 08:03:28.223992 21270 log.cc:1079] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/926dba79b58c42b78ca97f6c5f1032c9/wal-000000004 (ops 24-30)
I20260812 08:03:28.224023 21270 log.cc:1079] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/926dba79b58c42b78ca97f6c5f1032c9/wal-000000005 (ops 31-37)
I20260812 08:03:28.224136 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 45, Committed index: 45, Last appended: 2.45, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 46 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } }
I20260812 08:03:28.224596 21270 maintenance_manager.cc:643] P a451eb9951d54df2b69f0c01b98e1b8d: LogGCOp(926dba79b58c42b78ca97f6c5f1032c9) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 08:03:28.225296 21620 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 46. (index mismatch)
I20260812 08:03:28.225615 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 45. Preceding OpId from leader: term: 2 index: 46. (index mismatch)
I20260812 08:03:28.225822 21498 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 46, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:28.225975 21695 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 46, Last known committed idx: 45, Time since last communication: 0.000s
I20260812 08:03:28.226259 21498 consensus_queue.cc:753] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..45: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:28.226313 21498 consensus_queue.cc:579] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The replica will never be able to catch up
I20260812 08:03:28.227509 21498 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.46: config changed from index 45 to 46, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 46 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.228747 20830 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 45: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:28.229545 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.46: config changed from index 45 to 46, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 46 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.229843 20846 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 45 to 46, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 46 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } health_report { overall_health: FAILED_UNRECOVERABLE } } }
I20260812 08:03:28.232807 21350 logging.cc:436] LogThrottler TrackedPeer WAL GC: suppressed but not reported on 1 messages since previous log ~0 seconds ago
I20260812 08:03:28.232997 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 46, Committed index: 46, Last appended: 2.46, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 47 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } }
I20260812 08:03:28.234122 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 46. Preceding OpId from leader: term: 2 index: 47. (index mismatch)
I20260812 08:03:28.234488 21695 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 47, Last known committed idx: 46, Time since last communication: 0.000s
I20260812 08:03:28.236111 21696 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.47: config changed from index 46 to 47, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 47 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.236613 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.47: config changed from index 46 to 47, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 47 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.237501 20830 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 46: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:28.238480 20847 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 46 to 47, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 47 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:28.241729 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 47, Committed index: 47, Last appended: 2.47, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 48 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } }
I20260812 08:03:28.242528 21071 maintenance_manager.cc:419] P f00ae2646b20468d80628fbe90a33c79: Scheduling LogGCOp(926dba79b58c42b78ca97f6c5f1032c9): free 1577906 bytes of WAL
I20260812 08:03:28.242883 20942 log_reader.cc:385] T 926dba79b58c42b78ca97f6c5f1032c9: removed 2 log segments from log reader
I20260812 08:03:28.242934 20942 log.cc:1079] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/926dba79b58c42b78ca97f6c5f1032c9/wal-000000004 (ops 21-26)
I20260812 08:03:28.242975 20942 log.cc:1079] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/926dba79b58c42b78ca97f6c5f1032c9/wal-000000005 (ops 27-32)
I20260812 08:03:28.242980 21620 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 48. (index mismatch)
I20260812 08:03:28.243425 21695 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 48, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:28.243609 20942 maintenance_manager.cc:643] P f00ae2646b20468d80628fbe90a33c79: LogGCOp(926dba79b58c42b78ca97f6c5f1032c9) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 08:03:28.243752 21695 consensus_queue.cc:753] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..47: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:28.243839 21695 consensus_queue.cc:579] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The replica will never be able to catch up
I20260812 08:03:28.244352 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 47. Preceding OpId from leader: term: 2 index: 48. (index mismatch)
I20260812 08:03:28.244695 21695 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 48, Last known committed idx: 47, Time since last communication: 0.000s
I20260812 08:03:28.246238 21696 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.48: config changed from index 47 to 48, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 48 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.246644 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.48: config changed from index 47 to 48, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 48 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.247710 20830 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 47: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:28.248963 20847 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 reported cstate change: config changed from index 47 to 48, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 48 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.251679 21350 logging.cc:436] LogThrottler TrackedPeer WAL GC: suppressed but not reported on 1 messages since previous log ~0 seconds ago
I20260812 08:03:28.251868 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 48, Committed index: 48, Last appended: 2.48, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } }
I20260812 08:03:28.252800 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 48. Preceding OpId from leader: term: 2 index: 49. (index mismatch)
I20260812 08:03:28.253159 21696 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 49, Last known committed idx: 48, Time since last communication: 0.000s
I20260812 08:03:28.254578 21695 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.49: config changed from index 48 to 49, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.255012 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.49: config changed from index 48 to 49, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.255765 20830 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 48: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:28.256632 20846 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 48 to 49, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:28.259253 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 49, Committed index: 49, Last appended: 2.49, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 50 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } }
I20260812 08:03:28.260301 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 49. Preceding OpId from leader: term: 2 index: 50. (index mismatch)
I20260812 08:03:28.260494 21620 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 50. (index mismatch)
I20260812 08:03:28.260687 21696 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 50, Last known committed idx: 49, Time since last communication: 0.000s
I20260812 08:03:28.260910 21695 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 50, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:28.261135 21695 consensus_queue.cc:753] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..49: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:28.261186 21695 consensus_queue.cc:579] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The replica will never be able to catch up
I20260812 08:03:28.262459 21696 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.50: config changed from index 49 to 50, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 50 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.262892 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.50: config changed from index 49 to 50, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 50 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.263701 20830 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 49: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:28.264470 20846 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 49 to 50, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 50 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } health_report { overall_health: FAILED_UNRECOVERABLE } } }
I20260812 08:03:28.267539 21350 logging.cc:436] LogThrottler TrackedPeer WAL GC: suppressed but not reported on 1 messages since previous log ~0 seconds ago
I20260812 08:03:28.267765 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 50, Committed index: 50, Last appended: 2.50, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 51 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } }
I20260812 08:03:28.268827 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 50. Preceding OpId from leader: term: 2 index: 51. (index mismatch)
I20260812 08:03:28.269191 21685 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 51, Last known committed idx: 50, Time since last communication: 0.000s
I20260812 08:03:28.270905 21695 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.51: config changed from index 50 to 51, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 51 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.271310 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.51: config changed from index 50 to 51, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 51 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.272609 20830 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 50: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:28.273281 20847 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 50 to 51, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 51 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:28.276113 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 51, Committed index: 51, Last appended: 2.51, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 52 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } }
I20260812 08:03:28.277364 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 51. Preceding OpId from leader: term: 2 index: 52. (index mismatch)
I20260812 08:03:28.277387 21620 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 52. (index mismatch)
I20260812 08:03:28.277853 21695 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 52, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:28.278081 21685 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 52, Last known committed idx: 51, Time since last communication: 0.000s
I20260812 08:03:28.278136 21695 consensus_queue.cc:753] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..51: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:28.278203 21695 consensus_queue.cc:579] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The replica will never be able to catch up
I20260812 08:03:28.279723 21685 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.52: config changed from index 51 to 52, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 52 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.280970 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.52: config changed from index 51 to 52, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 52 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.281669 20830 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 51: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:28.282855 20847 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 51 to 52, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 52 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } health_report { overall_health: FAILED_UNRECOVERABLE } } }
I20260812 08:03:28.286104 21350 logging.cc:436] LogThrottler TrackedPeer WAL GC: suppressed but not reported on 1 messages since previous log ~0 seconds ago
I20260812 08:03:28.286312 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 52, Committed index: 52, Last appended: 2.52, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } }
I20260812 08:03:28.287458 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 52. Preceding OpId from leader: term: 2 index: 53. (index mismatch)
I20260812 08:03:28.287843 21696 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 53, Last known committed idx: 52, Time since last communication: 0.000s
I20260812 08:03:28.289323 21685 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.53: config changed from index 52 to 53, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.290678 20830 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 52: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:28.291085 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.53: config changed from index 52 to 53, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.292783 20847 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 52 to 53, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:28.295490 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 53, Committed index: 53, Last appended: 2.53, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 54 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } }
I20260812 08:03:28.296710 21620 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 54. (index mismatch)
I20260812 08:03:28.296774 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 53. Preceding OpId from leader: term: 2 index: 54. (index mismatch)
I20260812 08:03:28.297139 21685 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 54, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:28.297363 21685 consensus_queue.cc:753] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..53: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:28.297410 21685 consensus_queue.cc:579] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The replica will never be able to catch up
I20260812 08:03:28.297510 21685 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 54, Last known committed idx: 53, Time since last communication: 0.000s
I20260812 08:03:28.298861 21685 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.54: config changed from index 53 to 54, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 54 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.299270 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.54: config changed from index 53 to 54, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 54 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.300112 20830 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 53: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:28.300840 20847 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 53 to 54, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 54 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } health_report { overall_health: FAILED_UNRECOVERABLE } } }
I20260812 08:03:28.303224 21350 logging.cc:436] LogThrottler TrackedPeer WAL GC: suppressed but not reported on 1 messages since previous log ~0 seconds ago
I20260812 08:03:28.303390 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 54, Committed index: 54, Last appended: 2.54, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 55 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } }
I20260812 08:03:28.304313 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 54. Preceding OpId from leader: term: 2 index: 55. (index mismatch)
I20260812 08:03:28.304670 21498 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 55, Last known committed idx: 54, Time since last communication: 0.000s
I20260812 08:03:28.306167 21685 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.55: config changed from index 54 to 55, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 55 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.306635 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.55: config changed from index 54 to 55, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 55 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.307497 20830 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 54: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:28.308656 20846 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 54 to 55, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 55 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:28.311674 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 55, Committed index: 55, Last appended: 2.55, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } }
I20260812 08:03:28.312958 21620 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 56. (index mismatch)
I20260812 08:03:28.313061 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 55. Preceding OpId from leader: term: 2 index: 56. (index mismatch)
I20260812 08:03:28.313498 21685 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 56, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:28.313764 21685 consensus_queue.cc:753] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..55: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:28.313830 21685 consensus_queue.cc:579] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The replica will never be able to catch up
I20260812 08:03:28.314231 21685 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 56, Last known committed idx: 55, Time since last communication: 0.000s
I20260812 08:03:28.315727 21498 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.56: config changed from index 55 to 56, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.316133 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.56: config changed from index 55 to 56, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.317497 20830 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 55: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:28.318171 20846 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 reported cstate change: config changed from index 55 to 56, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.321195 21350 logging.cc:436] LogThrottler TrackedPeer WAL GC: suppressed but not reported on 1 messages since previous log ~0 seconds ago
I20260812 08:03:28.321395 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 56, Committed index: 56, Last appended: 2.56, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 57 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } }
I20260812 08:03:28.322538 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 56. Preceding OpId from leader: term: 2 index: 57. (index mismatch)
I20260812 08:03:28.322994 21685 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 57, Last known committed idx: 56, Time since last communication: 0.000s
I20260812 08:03:28.324710 21695 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.57: config changed from index 56 to 57, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 57 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.325184 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.57: config changed from index 56 to 57, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 57 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.326301 20830 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 56: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:28.327281 20847 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 56 to 57, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 57 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:28.330144 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 57, Committed index: 57, Last appended: 2.57, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 58 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } }
I20260812 08:03:28.331341 21620 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 58. (index mismatch)
I20260812 08:03:28.331573 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 57. Preceding OpId from leader: term: 2 index: 58. (index mismatch)
I20260812 08:03:28.331769 21685 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 58, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:28.332020 21685 consensus_queue.cc:753] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..57: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:28.331979 21695 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 58, Last known committed idx: 57, Time since last communication: 0.000s
I20260812 08:03:28.332126 21685 consensus_queue.cc:579] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The replica will never be able to catch up
I20260812 08:03:28.333770 21685 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.58: config changed from index 57 to 58, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 58 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.335255 20830 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 57: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:28.335587 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.58: config changed from index 57 to 58, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 58 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.336073 20847 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 57 to 58, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 58 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } health_report { overall_health: FAILED_UNRECOVERABLE } } }
I20260812 08:03:28.338733 21350 logging.cc:436] LogThrottler TrackedPeer WAL GC: suppressed but not reported on 1 messages since previous log ~0 seconds ago
I20260812 08:03:28.338923 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 58, Committed index: 58, Last appended: 2.58, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } }
I20260812 08:03:28.340046 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 58. Preceding OpId from leader: term: 2 index: 59. (index mismatch)
I20260812 08:03:28.340499 21498 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 59, Last known committed idx: 58, Time since last communication: 0.000s
I20260812 08:03:28.342111 21685 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.59: config changed from index 58 to 59, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.342546 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.59: config changed from index 58 to 59, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.343881 20830 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 58: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:28.344787 20846 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 58 to 59, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:28.347608 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 59, Committed index: 59, Last appended: 2.59, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } }
I20260812 08:03:28.348840 21620 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 60. (index mismatch)
I20260812 08:03:28.348907 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 59. Preceding OpId from leader: term: 2 index: 60. (index mismatch)
I20260812 08:03:28.349368 21685 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 60, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:28.349624 21498 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 60, Last known committed idx: 59, Time since last communication: 0.000s
I20260812 08:03:28.349900 21685 consensus_queue.cc:753] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..59: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:28.350102 21685 consensus_queue.cc:579] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The replica will never be able to catch up
I20260812 08:03:28.351302 21498 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.60: config changed from index 59 to 60, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.351739 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.60: config changed from index 59 to 60, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.352911 20830 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 59: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:28.353840 20846 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 59 to 60, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } health_report { overall_health: FAILED_UNRECOVERABLE } } }
I20260812 08:03:28.356858 21350 logging.cc:436] LogThrottler TrackedPeer WAL GC: suppressed but not reported on 1 messages since previous log ~0 seconds ago
I20260812 08:03:28.357054 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 60, Committed index: 60, Last appended: 2.60, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } }
I20260812 08:03:28.358130 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 60. Preceding OpId from leader: term: 2 index: 61. (index mismatch)
I20260812 08:03:28.358539 21695 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 61, Last known committed idx: 60, Time since last communication: 0.000s
I20260812 08:03:28.360121 21685 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.61: config changed from index 60 to 61, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.360677 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.61: config changed from index 60 to 61, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.361543 20830 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 60: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:28.363070 20846 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 reported cstate change: config changed from index 60 to 61, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.366168 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 61, Committed index: 61, Last appended: 2.61, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } }
I20260812 08:03:28.367404 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 61. Preceding OpId from leader: term: 2 index: 62. (index mismatch)
I20260812 08:03:28.367424 21620 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 62. (index mismatch)
I20260812 08:03:28.367794 21685 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 62, Last known committed idx: 61, Time since last communication: 0.000s
I20260812 08:03:28.368029 21695 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 62, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:28.368284 21695 consensus_queue.cc:753] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..61: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:28.368338 21695 consensus_queue.cc:579] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The replica will never be able to catch up
I20260812 08:03:28.369733 21695 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.62: config changed from index 61 to 62, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.371080 20830 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 61: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:28.371492 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.62: config changed from index 61 to 62, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.371690 20847 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 61 to 62, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } health_report { overall_health: FAILED_UNRECOVERABLE } } }
I20260812 08:03:28.374321 21350 logging.cc:436] LogThrottler TrackedPeer WAL GC: suppressed but not reported on 1 messages since previous log ~0 seconds ago
I20260812 08:03:28.374477 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 62, Committed index: 62, Last appended: 2.62, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 63 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } }
I20260812 08:03:28.375602 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 62. Preceding OpId from leader: term: 2 index: 63. (index mismatch)
I20260812 08:03:28.376021 21498 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 63, Last known committed idx: 62, Time since last communication: 0.000s
I20260812 08:03:28.377681 21695 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.63: config changed from index 62 to 63, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 63 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.378227 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.63: config changed from index 62 to 63, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 63 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.379336 20830 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 62: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:28.380059 20846 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 62 to 63, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 63 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:28.382910 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 63, Committed index: 63, Last appended: 2.63, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } }
I20260812 08:03:28.384141 21620 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 64. (index mismatch)
I20260812 08:03:28.384156 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 63. Preceding OpId from leader: term: 2 index: 64. (index mismatch)
I20260812 08:03:28.384505 21685 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 63, Time since last communication: 0.000s
I20260812 08:03:28.384684 21498 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:28.384881 21498 consensus_queue.cc:753] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..63: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:28.384940 21498 consensus_queue.cc:579] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The replica will never be able to catch up
I20260812 08:03:28.386184 21498 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.64: config changed from index 63 to 64, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.387431 20830 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 63: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:28.387487 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.64: config changed from index 63 to 64, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.388193 20846 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 63 to 64, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } health_report { overall_health: FAILED_UNRECOVERABLE } } }
I20260812 08:03:28.390556 21350 logging.cc:436] LogThrottler TrackedPeer WAL GC: suppressed but not reported on 1 messages since previous log ~0 seconds ago
I20260812 08:03:28.390708 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 64, Committed index: 64, Last appended: 2.64, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 65 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } }
I20260812 08:03:28.391623 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 64. Preceding OpId from leader: term: 2 index: 65. (index mismatch)
I20260812 08:03:28.391959 21685 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 65, Last known committed idx: 64, Time since last communication: 0.000s
I20260812 08:03:28.393280 21498 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.65: config changed from index 64 to 65, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 65 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.393761 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.65: config changed from index 64 to 65, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 65 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.394519 20830 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 64: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:28.395546 20847 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 64 to 65, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 65 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:28.398430 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 65, Committed index: 65, Last appended: 2.65, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } }
I20260812 08:03:28.399735 21620 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 66. (index mismatch)
I20260812 08:03:28.399827 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 65. Preceding OpId from leader: term: 2 index: 66. (index mismatch)
I20260812 08:03:28.400213 21498 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:28.400426 21685 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 65, Time since last communication: 0.000s
I20260812 08:03:28.400494 21498 consensus_queue.cc:753] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..65: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:28.400563 21498 consensus_queue.cc:579] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The replica will never be able to catch up
I20260812 08:03:28.402316 21498 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.66: config changed from index 65 to 66, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.402794 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.66: config changed from index 65 to 66, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.404002 20830 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 65: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:28.405192 20844 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 65 to 66, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } health_report { overall_health: FAILED_UNRECOVERABLE } } }
I20260812 08:03:28.408191 21350 logging.cc:436] LogThrottler TrackedPeer WAL GC: suppressed but not reported on 1 messages since previous log ~0 seconds ago
I20260812 08:03:28.408402 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 66, Committed index: 66, Last appended: 2.66, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } }
I20260812 08:03:28.409556 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 66. Preceding OpId from leader: term: 2 index: 67. (index mismatch)
I20260812 08:03:28.409976 21498 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 67, Last known committed idx: 66, Time since last communication: 0.000s
I20260812 08:03:28.411765 21685 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.67: config changed from index 66 to 67, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.412264 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.67: config changed from index 66 to 67, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.413341 20830 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 66: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:28.414546 20847 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 66 to 67, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:28.417793 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 67, Committed index: 67, Last appended: 2.67, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 68 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } }
I20260812 08:03:28.419107 21620 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 68. (index mismatch)
I20260812 08:03:28.419277 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 67. Preceding OpId from leader: term: 2 index: 68. (index mismatch)
I20260812 08:03:28.419620 21685 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 68, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:28.419909 21695 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 68, Last known committed idx: 67, Time since last communication: 0.000s
I20260812 08:03:28.420076 21685 consensus_queue.cc:753] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..67: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:28.420149 21685 consensus_queue.cc:579] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The replica will never be able to catch up
I20260812 08:03:28.421653 21498 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.68: config changed from index 67 to 68, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 68 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.422140 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.68: config changed from index 67 to 68, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 68 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.423434 20830 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 67: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:28.424472 20844 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 reported cstate change: config changed from index 67 to 68, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 68 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.427681 21350 logging.cc:436] LogThrottler TrackedPeer WAL GC: suppressed but not reported on 1 messages since previous log ~0 seconds ago
I20260812 08:03:28.427855 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 68, Committed index: 68, Last appended: 2.68, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 69 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } }
I20260812 08:03:28.428988 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 68. Preceding OpId from leader: term: 2 index: 69. (index mismatch)
I20260812 08:03:28.429502 21498 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 69, Last known committed idx: 68, Time since last communication: 0.000s
I20260812 08:03:28.431306 21695 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.69: config changed from index 68 to 69, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 69 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.431840 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.69: config changed from index 68 to 69, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 69 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.433000 20830 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 68: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:28.433933 20847 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 68 to 69, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 69 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:28.437234 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 69, Committed index: 69, Last appended: 2.69, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 70 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } }
I20260812 08:03:28.438699 21620 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 70. (index mismatch)
I20260812 08:03:28.438911 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 69. Preceding OpId from leader: term: 2 index: 70. (index mismatch)
I20260812 08:03:28.439111 21498 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 70, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:28.439363 21498 consensus_queue.cc:753] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..69: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:28.439393 21695 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 70, Last known committed idx: 69, Time since last communication: 0.000s
I20260812 08:03:28.439591 21498 consensus_queue.cc:579] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: The logs necessary to catch up peer 416b6427e9244877a46ab232ff39f8cd have been garbage collected. The replica will never be able to catch up
I20260812 08:03:28.441648 21685 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.70: config changed from index 69 to 70, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 70 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.442034 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.70: config changed from index 69 to 70, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 70 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.443483 20830 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 69: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:28.444108 20844 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 reported cstate change: config changed from index 69 to 70, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 70 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.445062 21594 tablet_service.cc:1558] Processing DeleteTablet for tablet 926dba79b58c42b78ca97f6c5f1032c9 with delete_type TABLET_DATA_TOMBSTONED (TS 416b6427e9244877a46ab232ff39f8cd not found in new config with opid_index 39) from {username='slave'} at 127.0.0.1:57132
I20260812 08:03:28.446012 21698 tablet_replica.cc:333] stopping tablet replica
I20260812 08:03:28.446369 21698 raft_consensus.cc:2243] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 08:03:28.446628 21698 raft_consensus.cc:2272] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 08:03:28.447300 21350 logging.cc:436] LogThrottler TrackedPeer WAL GC: suppressed but not reported on 1 messages since previous log ~0 seconds ago
I20260812 08:03:28.447466 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 70, Committed index: 70, Last appended: 2.70, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 71 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } }
I20260812 08:03:28.447574 21698 ts_tablet_manager.cc:1916] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 08:03:28.448577 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 70. Preceding OpId from leader: term: 2 index: 71. (index mismatch)
I20260812 08:03:28.449038 21685 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 71, Last known committed idx: 70, Time since last communication: 0.000s
I20260812 08:03:28.450567 21698 ts_tablet_manager.cc:1929] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.1
I20260812 08:03:28.450659 21698 log.cc:1199] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: Deleting WAL directory at /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/wals/926dba79b58c42b78ca97f6c5f1032c9
I20260812 08:03:28.450716 21695 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.71: config changed from index 70 to 71, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 71 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.451187 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.71: config changed from index 70 to 71, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New config: { opid_index: 71 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } }
I20260812 08:03:28.451424 20830 catalog_manager.cc:5050] TS 416b6427e9244877a46ab232ff39f8cd (127.0.186.66:36375): tablet 926dba79b58c42b78ca97f6c5f1032c9 (table TestTable [id=2c2ede4592a049e5a67af8eb14d0cba5]) successfully deleted
I20260812 08:03:28.452416 20830 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 70: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:28.453009 20847 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 70 to 71, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) evicted. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 71 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:28.455379 21594 tablet_service.cc:1558] Processing DeleteTablet for tablet 926dba79b58c42b78ca97f6c5f1032c9 with delete_type TABLET_DATA_TOMBSTONED (TS 416b6427e9244877a46ab232ff39f8cd not found in new config with opid_index 71) from {username='slave'} at 127.0.0.1:57132
I20260812 08:03:28.455825 20830 catalog_manager.cc:5050] TS 416b6427e9244877a46ab232ff39f8cd (127.0.186.66:36375): tablet 926dba79b58c42b78ca97f6c5f1032c9 (table TestTable [id=2c2ede4592a049e5a67af8eb14d0cba5]) successfully deleted
I20260812 08:03:28.456027 21350 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 71, Committed index: 71, Last appended: 2.71, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } }
I20260812 08:03:28.457254 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 71. Preceding OpId from leader: term: 2 index: 72. (index mismatch)
I20260812 08:03:28.457612 21685 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 72, Last known committed idx: 71, Time since last communication: 0.000s
W20260812 08:03:28.457806 21269 consensus_peers.cc:597] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d -> Peer 416b6427e9244877a46ab232ff39f8cd (127.0.186.66:36375): Couldn't send request to peer 416b6427e9244877a46ab232ff39f8cd. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:03:28.459065 21695 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.72: config changed from index 71 to 72, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.459544 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.72: config changed from index 71 to 72, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New config: { opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } } }
I20260812 08:03:28.460438 20830 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 926dba79b58c42b78ca97f6c5f1032c9 with cas_config_opid_index 71: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:28.461419 20847 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 71 to 72, NON_VOTER 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) added. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260812 08:03:29.003242 21700 ts_tablet_manager.cc:933] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: Initiating tablet copy from peer a451eb9951d54df2b69f0c01b98e1b8d (127.0.186.67:36983)
I20260812 08:03:29.003775 21700 tablet_copy_client.cc:323] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: tablet copy: Beginning tablet copy session from remote peer at address 127.0.186.67:36983
I20260812 08:03:29.007695 21373 tablet_copy_service.cc:140] P a451eb9951d54df2b69f0c01b98e1b8d: Received BeginTabletCopySession request for tablet 926dba79b58c42b78ca97f6c5f1032c9 from peer 416b6427e9244877a46ab232ff39f8cd ({username='slave'} at 127.0.186.66:37337)
I20260812 08:03:29.007850 21373 tablet_copy_service.cc:161] P a451eb9951d54df2b69f0c01b98e1b8d: Beginning new tablet copy session on tablet 926dba79b58c42b78ca97f6c5f1032c9 from peer 416b6427e9244877a46ab232ff39f8cd at {username='slave'} at 127.0.186.66:37337: session id = 416b6427e9244877a46ab232ff39f8cd-926dba79b58c42b78ca97f6c5f1032c9
I20260812 08:03:29.009305 21373 tablet_copy_source_session.cc:215] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d: Tablet Copy: opened 5 blocks and 1 log segments
I20260812 08:03:29.010911 21700 ts_tablet_manager.cc:1916] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260812 08:03:29.012684 21700 ts_tablet_manager.cc:1929] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.1
I20260812 08:03:29.013005 21700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 926dba79b58c42b78ca97f6c5f1032c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:29.014130 21700 tablet_copy_client.cc:806] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: tablet copy: Starting download of 5 data blocks...
I20260812 08:03:29.021024 21700 tablet_copy_client.cc:670] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: tablet copy: Starting download of 1 WAL segments...
I20260812 08:03:29.023265 21700 tablet_copy_client.cc:538] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 08:03:29.025027 21700 tablet_bootstrap.cc:492] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: Bootstrap starting.
I20260812 08:03:29.041009 21700 tablet_bootstrap.cc:492] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=35 overwritten=0 applied=35 ignored=1} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=3. Pending: 0 replicates
I20260812 08:03:29.041291 21700 tablet_bootstrap.cc:492] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: Bootstrap complete.
I20260812 08:03:29.041518 21700 ts_tablet_manager.cc:1403] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.004s
I20260812 08:03:29.042098 21700 raft_consensus.cc:359] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } }
I20260812 08:03:29.042209 21700 raft_consensus.cc:740] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 416b6427e9244877a46ab232ff39f8cd, State: Initialized, Role: LEARNER
I20260812 08:03:29.042291 21700 consensus_queue.cc:260] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 72, Last appended: 2.72, Last appended by leader: 72, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: NON_VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: true } }
I20260812 08:03:29.042614 21700 ts_tablet_manager.cc:1434] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20260812 08:03:29.043488 21373 tablet_copy_service.cc:342] P a451eb9951d54df2b69f0c01b98e1b8d: Request end of tablet copy session 416b6427e9244877a46ab232ff39f8cd-926dba79b58c42b78ca97f6c5f1032c9 received from {username='slave'} at 127.0.186.66:37337
I20260812 08:03:29.043592 21373 tablet_copy_service.cc:434] P a451eb9951d54df2b69f0c01b98e1b8d: ending tablet copy session 416b6427e9244877a46ab232ff39f8cd-926dba79b58c42b78ca97f6c5f1032c9 on tablet 926dba79b58c42b78ca97f6c5f1032c9 with peer 416b6427e9244877a46ab232ff39f8cd
I20260812 08:03:29.118508 21677 maintenance_manager.cc:419] P 416b6427e9244877a46ab232ff39f8cd: Scheduling UndoDeltaBlockGCOp(926dba79b58c42b78ca97f6c5f1032c9): 534 bytes on disk
I20260812 08:03:29.119655 21542 maintenance_manager.cc:643] P 416b6427e9244877a46ab232ff39f8cd: UndoDeltaBlockGCOp(926dba79b58c42b78ca97f6c5f1032c9) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":66,"lbm_reads_lt_1ms":4}
I20260812 08:03:29.621493 21620 raft_consensus.cc:1217] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 2 LEARNER]: Deduplicated request from leader. Original: 2.71->[2.72-2.72]   Dedup: 2.72->[]
I20260812 08:03:30.166224 21713 raft_consensus.cc:1064] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d: attempting to promote NON_VOTER 416b6427e9244877a46ab232ff39f8cd to VOTER
I20260812 08:03:30.166843 21713 consensus_queue.cc:237] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 72, Committed index: 72, Last appended: 2.72, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 73 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: false } }
I20260812 08:03:30.168288 21017 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 72. Preceding OpId from leader: term: 2 index: 73. (index mismatch)
I20260812 08:03:30.168288 21620 raft_consensus.cc:1275] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 2 LEARNER]: Refusing update from remote peer a451eb9951d54df2b69f0c01b98e1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 72. Preceding OpId from leader: term: 2 index: 73. (index mismatch)
I20260812 08:03:30.168787 21715 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 73, Last known committed idx: 72, Time since last communication: 0.000s
I20260812 08:03:30.169003 21711 consensus_queue.cc:1048] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 73, Last known committed idx: 72, Time since last communication: 0.000s
I20260812 08:03:30.170888 21715 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d [term 2 LEADER]: Committing config change with OpId 2.73: config changed from index 72 to 73, 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) changed from NON_VOTER to VOTER. New config: { opid_index: 73 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: false } } }
I20260812 08:03:30.171427 21017 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P f00ae2646b20468d80628fbe90a33c79 [term 2 FOLLOWER]: Committing config change with OpId 2.73: config changed from index 72 to 73, 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) changed from NON_VOTER to VOTER. New config: { opid_index: 73 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: false } } }
I20260812 08:03:30.172322 21620 raft_consensus.cc:2955] T 926dba79b58c42b78ca97f6c5f1032c9 P 416b6427e9244877a46ab232ff39f8cd [term 2 FOLLOWER]: Committing config change with OpId 2.73: config changed from index 72 to 73, 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) changed from NON_VOTER to VOTER. New config: { opid_index: 73 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: false } } }
I20260812 08:03:30.173878 20844 catalog_manager.cc:5719] T 926dba79b58c42b78ca97f6c5f1032c9 P a451eb9951d54df2b69f0c01b98e1b8d reported cstate change: config changed from index 72 to 73, 416b6427e9244877a46ab232ff39f8cd (127.0.186.66) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" committed_config { opid_index: 73 OBSOLETE_local: false peers { permanent_uuid: "f00ae2646b20468d80628fbe90a33c79" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a451eb9951d54df2b69f0c01b98e1b8d" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 36983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "416b6427e9244877a46ab232ff39f8cd" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 36375 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 08:03:30.235603 21324 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260812 08:03:30.235831 20991 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:03:30.237386 21594 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |       Address       | Status
----------------------------------+---------------------+---------
 42480246dfbf41ba84d75bddeadd4fe3 | 127.0.186.126:37909 | HEALTHY

Unusual flags for Master:
                   Flag                   |                                                                                               Value                                                                                                |         Tags          |         Master
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
 ipki_ca_key_size                         | 768                                                                                                                                                                                                | experimental          | all 1 server(s) checked
 ipki_server_key_size                     | 768                                                                                                                                                                                                | experimental          | all 1 server(s) checked
 never_fsync                              | true                                                                                                                                                                                               | unsafe,advanced       | all 1 server(s) checked
 openssl_security_level_override          | 0                                                                                                                                                                                                  | unsafe,hidden         | all 1 server(s) checked
 raft_prepare_replacement_before_eviction | true                                                                                                                                                                                               | experimental,advanced | all 1 server(s) checked
 rpc_reuseport                            | true                                                                                                                                                                                               | experimental          | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports         | true                                                                                                                                                                                               | unsafe                | all 1 server(s) checked
 server_dump_info_format                  | pb                                                                                                                                                                                                 | hidden                | all 1 server(s) checked
 server_dump_info_path                    | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden                | all 1 server(s) checked
 tsk_num_rsa_bits                         | 512                                                                                                                                                                                                | experimental          | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |       Value        |         Master
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.0.186.84:41405 | all 1 server(s) checked
 time_source         | builtin            | all 1 server(s) checked

Tablet Server Summary
               UUID               |      Address       | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
 416b6427e9244877a46ab232ff39f8cd | 127.0.186.66:36375 | HEALTHY | <none>   |       0        |       0
 a451eb9951d54df2b69f0c01b98e1b8d | 127.0.186.67:36983 | HEALTHY | <none>   |       1        |       0
 f00ae2646b20468d80628fbe90a33c79 | 127.0.186.65:40623 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
                   Flag                   |                                                                                             Value                                                                                              |             Tags              |      Tablet Server
------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------+-------------------------
 ipki_server_key_size                     | 768                                                                                                                                                                                            | experimental                  | all 3 server(s) checked
 local_ip_for_outbound_sockets            | 127.0.186.65                                                                                                                                                                                   | experimental                  | 127.0.186.65:40623
 local_ip_for_outbound_sockets            | 127.0.186.66                                                                                                                                                                                   | experimental                  | 127.0.186.66:36375
 local_ip_for_outbound_sockets            | 127.0.186.67                                                                                                                                                                                   | experimental                  | 127.0.186.67:36983
 log_compression_codec                    | no_compression                                                                                                                                                                                 | experimental                  | all 3 server(s) checked
 log_max_segments_to_retain               | 3                                                                                                                                                                                              | experimental,runtime,advanced | all 3 server(s) checked
 log_target_replay_size_mb                | 1                                                                                                                                                                                              | experimental                  | all 3 server(s) checked
 maintenance_manager_polling_interval_ms  | 100                                                                                                                                                                                            | hidden                        | all 3 server(s) checked
 max_cell_size_bytes                      | 1000000                                                                                                                                                                                        | unsafe                        | all 3 server(s) checked
 never_fsync                              | true                                                                                                                                                                                           | unsafe,advanced               | all 3 server(s) checked
 openssl_security_level_override          | 0                                                                                                                                                                                              | unsafe,hidden                 | all 3 server(s) checked
 raft_prepare_replacement_before_eviction | true                                                                                                                                                                                           | experimental,advanced         | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports         | true                                                                                                                                                                                           | unsafe                        | all 3 server(s) checked
 server_dump_info_format                  | pb                                                                                                                                                                                             | hidden                        | all 3 server(s) checked
 server_dump_info_path                    | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden                        | 127.0.186.65:40623
 server_dump_info_path                    | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden                        | 127.0.186.66:36375
 server_dump_info_path                    | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_4.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden                        | 127.0.186.67:36983

Flags of checked categories for Tablet Server:
        Flag         |       Value        |      Tablet Server
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.0.186.84:41405 | all 3 server(s) checked
 time_source         | builtin            | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
   Name    | RF | Status  | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-----------+----+---------+---------------+---------+------------+------------------+-------------
 TestTable | 3  | HEALTHY | 1             | 1       | 0          | 0                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

OK
I20260812 08:03:30.260414   745 log_verifier.cc:126] Checking tablet 926dba79b58c42b78ca97f6c5f1032c9
I20260812 08:03:30.271934   745 log_verifier.cc:177] Verified matching terms for 41 ops in tablet 926dba79b58c42b78ca97f6c5f1032c9
I20260812 08:03:30.272450   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 20920
I20260812 08:03:30.285755   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 21518
I20260812 08:03:30.295581   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 21245
I20260812 08:03:30.305713   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 20808
2026-08-12T08:03:30Z chronyd exiting
[       OK ] ReplicaBehindWalGcThresholdITest.ReplicaReplacement/4 (7956 ms)
[ RUN      ] ReplicaBehindWalGcThresholdITest.ReplicaReplacement/5
I20260812 08:03:30.334148   745 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:03:30.334197   745 ts_itest-base.cc:116] --------------
I20260812 08:03:30.334214   745 ts_itest-base.cc:117] 3 tablet servers
I20260812 08:03:30.334226   745 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:03:30.334240   745 ts_itest-base.cc:119] --------------
2026-08-12T08:03:30Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:03:30Z Disabled control of system clock
I20260812 08:03:30.343855   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:32839
--webserver_interface=127.0.186.126
--webserver_port=0
--builtin_ntp_servers=127.0.186.84:42325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.126:32839
--raft_prepare_replacement_before_eviction=true with env {}
W20260812 08:03:30.448815 21760 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:30.449065 21760 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:30.449122 21760 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:30.452478 21760 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:03:30.452546 21760 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:30.452574 21760 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:03:30.452598 21760 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:03:30.456387 21760 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:42325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.126:32839
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.126:32839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.186.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.21760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:30.457546 21760 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:30.458688 21760 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:30.464020 21768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:30.464066 21771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:30.464020 21769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:30.464375 21760 server_base.cc:1061] running on GCE node
I20260812 08:03:30.464911 21760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:30.465837 21760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:30.466989 21760 hybrid_clock.cc:648] HybridClock initialized: now 1786521810466970 us; error 34 us; skew 500 ppm
I20260812 08:03:30.468641 21760 webserver.cc:533] Webserver started at http://127.0.186.126:36781/ using document root <none> and password file <none>
I20260812 08:03:30.469200 21760 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:30.469270 21760 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:30.469475 21760 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:30.470995 21760 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "408a5cecd2c242388a8cac956732949d"
format_stamp: "Formatted at 2026-08-12 08:03:30 on dist-test-slave-49h3"
I20260812 08:03:30.471468 21760 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "408a5cecd2c242388a8cac956732949d"
format_stamp: "Formatted at 2026-08-12 08:03:30 on dist-test-slave-49h3"
I20260812 08:03:30.474720 21760 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:03:30.476823 21779 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:30.477824 21760 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:03:30.477936 21760 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
uuid: "408a5cecd2c242388a8cac956732949d"
format_stamp: "Formatted at 2026-08-12 08:03:30 on dist-test-slave-49h3"
I20260812 08:03:30.478022 21760 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:30.494135 21760 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:30.494848 21760 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:30.495026 21760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:30.502130 21842 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.126:32839 every 8 connection(s)
I20260812 08:03:30.502147 21760 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.126:32839
I20260812 08:03:30.503297 21760 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:03:30.506331 21843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:30.509615   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 21760
I20260812 08:03:30.509775   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:03:30.512959 21843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 408a5cecd2c242388a8cac956732949d: Bootstrap starting.
I20260812 08:03:30.515435 21843 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 408a5cecd2c242388a8cac956732949d: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:30.516185 21843 log.cc:826] T 00000000000000000000000000000000 P 408a5cecd2c242388a8cac956732949d: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:30.518316 21843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 408a5cecd2c242388a8cac956732949d: No bootstrap required, opened a new log
I20260812 08:03:30.521121 21843 raft_consensus.cc:359] T 00000000000000000000000000000000 P 408a5cecd2c242388a8cac956732949d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408a5cecd2c242388a8cac956732949d" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 32839 } }
I20260812 08:03:30.521318 21843 raft_consensus.cc:385] T 00000000000000000000000000000000 P 408a5cecd2c242388a8cac956732949d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:30.521373 21843 raft_consensus.cc:740] T 00000000000000000000000000000000 P 408a5cecd2c242388a8cac956732949d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408a5cecd2c242388a8cac956732949d, State: Initialized, Role: FOLLOWER
I20260812 08:03:30.521763 21843 consensus_queue.cc:260] T 00000000000000000000000000000000 P 408a5cecd2c242388a8cac956732949d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408a5cecd2c242388a8cac956732949d" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 32839 } }
I20260812 08:03:30.521880 21843 raft_consensus.cc:399] T 00000000000000000000000000000000 P 408a5cecd2c242388a8cac956732949d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:03:30.521924 21843 raft_consensus.cc:493] T 00000000000000000000000000000000 P 408a5cecd2c242388a8cac956732949d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:03:30.522003 21843 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 408a5cecd2c242388a8cac956732949d [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:30.522758 21843 raft_consensus.cc:515] T 00000000000000000000000000000000 P 408a5cecd2c242388a8cac956732949d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408a5cecd2c242388a8cac956732949d" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 32839 } }
I20260812 08:03:30.523052 21843 leader_election.cc:304] T 00000000000000000000000000000000 P 408a5cecd2c242388a8cac956732949d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 408a5cecd2c242388a8cac956732949d; no voters: 
I20260812 08:03:30.523325 21843 leader_election.cc:290] T 00000000000000000000000000000000 P 408a5cecd2c242388a8cac956732949d [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:03:30.523450 21849 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 408a5cecd2c242388a8cac956732949d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:30.523672 21849 raft_consensus.cc:697] T 00000000000000000000000000000000 P 408a5cecd2c242388a8cac956732949d [term 1 LEADER]: Becoming Leader. State: Replica: 408a5cecd2c242388a8cac956732949d, State: Running, Role: LEADER
I20260812 08:03:30.523965 21849 consensus_queue.cc:237] T 00000000000000000000000000000000 P 408a5cecd2c242388a8cac956732949d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408a5cecd2c242388a8cac956732949d" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 32839 } }
I20260812 08:03:30.524478 21843 sys_catalog.cc:565] T 00000000000000000000000000000000 P 408a5cecd2c242388a8cac956732949d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:30.525071 21852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 408a5cecd2c242388a8cac956732949d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "408a5cecd2c242388a8cac956732949d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408a5cecd2c242388a8cac956732949d" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 32839 } } }
I20260812 08:03:30.525195 21852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 408a5cecd2c242388a8cac956732949d [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:30.525624 21855 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:03:30.525565 21850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 408a5cecd2c242388a8cac956732949d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 408a5cecd2c242388a8cac956732949d. Latest consensus state: current_term: 1 leader_uuid: "408a5cecd2c242388a8cac956732949d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408a5cecd2c242388a8cac956732949d" member_type: VOTER last_known_addr { host: "127.0.186.126" port: 32839 } } }
I20260812 08:03:30.525779 21850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 408a5cecd2c242388a8cac956732949d [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:30.528631 21855 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:03:30.533942 21855 catalog_manager.cc:1383] Generated new cluster ID: 80a957ca27d346d28dd174369a25c223
I20260812 08:03:30.534001 21855 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:03:30.540259 21855 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:03:30.540992 21855 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:03:30.552626 21855 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 408a5cecd2c242388a8cac956732949d: Generated new TSK 0
I20260812 08:03:30.553387 21855 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:03:30.567536   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.65:0
--local_ip_for_outbound_sockets=127.0.186.65
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:32839
--builtin_ntp_servers=127.0.186.84:42325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20260812 08:03:30.674690 21872 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:03:30.674911 21872 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:30.674943 21872 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:30.675002 21872 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:30.678259 21872 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:03:30.678315 21872 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:03:30.678352 21872 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:30.678442 21872 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:03:30.678475 21872 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.65
I20260812 08:03:30.682322 21872 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:42325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.186.65
--webserver_port=0
--tserver_master_addrs=127.0.186.126:32839
--never_fsync=true
--heap_profile_path=/tmp/kudu.21872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:30.683382 21872 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:30.684499 21872 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:30.691052 21879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:30.691111 21878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:30.691234 21872 server_base.cc:1061] running on GCE node
W20260812 08:03:30.691054 21882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:30.691766 21872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:30.692409 21872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:30.693564 21872 hybrid_clock.cc:648] HybridClock initialized: now 1786521810693535 us; error 51 us; skew 500 ppm
I20260812 08:03:30.695478 21872 webserver.cc:533] Webserver started at http://127.0.186.65:41327/ using document root <none> and password file <none>
I20260812 08:03:30.696091 21872 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:30.696154 21872 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:30.696352 21872 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:30.698128 21872 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "4c43ad693825459f82d4ba3704c3fdb0"
format_stamp: "Formatted at 2026-08-12 08:03:30 on dist-test-slave-49h3"
I20260812 08:03:30.698663 21872 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "4c43ad693825459f82d4ba3704c3fdb0"
format_stamp: "Formatted at 2026-08-12 08:03:30 on dist-test-slave-49h3"
I20260812 08:03:30.702208 21872 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.001s
I20260812 08:03:30.704375 21891 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:30.705377 21872 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 08:03:30.705506 21872 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "4c43ad693825459f82d4ba3704c3fdb0"
format_stamp: "Formatted at 2026-08-12 08:03:30 on dist-test-slave-49h3"
I20260812 08:03:30.705597 21872 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:30.720453 21872 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:30.721069 21872 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:30.721230 21872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:30.721812 21872 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:30.722838 21872 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:30.722887 21872 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:30.722939 21872 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:30.722965 21872 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:30.732671 21872 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.65:34877
I20260812 08:03:30.732697 22037 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.65:34877 every 8 connection(s)
I20260812 08:03:30.733872 21872 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:03:30.734647   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 21872
I20260812 08:03:30.734794   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:03:30.738739   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.66:0
--local_ip_for_outbound_sockets=127.0.186.66
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:32839
--builtin_ntp_servers=127.0.186.84:42325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20260812 08:03:30.743461 22038 heartbeater.cc:344] Connected to a master server at 127.0.186.126:32839
I20260812 08:03:30.743721 22038 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:30.744287 22038 heartbeater.cc:507] Master 127.0.186.126:32839 requested a full tablet report, sending...
I20260812 08:03:30.746105 21800 ts_manager.cc:194] Registered new tserver with Master: 4c43ad693825459f82d4ba3704c3fdb0 (127.0.186.65:34877)
I20260812 08:03:30.747819 21800 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.65:57239
W20260812 08:03:30.848176 22041 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:03:30.848410 22041 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:30.848436 22041 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:30.848472 22041 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:30.851681 22041 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:03:30.851739 22041 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:03:30.851773 22041 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:30.851835 22041 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:03:30.851861 22041 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.66
I20260812 08:03:30.855676 22041 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:42325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.186.66
--webserver_port=0
--tserver_master_addrs=127.0.186.126:32839
--never_fsync=true
--heap_profile_path=/tmp/kudu.22041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:30.856719 22041 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:30.857846 22041 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:30.864280 22050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:30.864249 22046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:30.864249 22048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:30.864724 22041 server_base.cc:1061] running on GCE node
I20260812 08:03:30.865108 22041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:30.865748 22041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:30.866905 22041 hybrid_clock.cc:648] HybridClock initialized: now 1786521810866887 us; error 29 us; skew 500 ppm
I20260812 08:03:30.868669 22041 webserver.cc:533] Webserver started at http://127.0.186.66:36127/ using document root <none> and password file <none>
I20260812 08:03:30.869257 22041 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:30.869319 22041 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:30.869547 22041 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:30.871263 22041 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "397b07826d51403f8bc1085aea72b174"
format_stamp: "Formatted at 2026-08-12 08:03:30 on dist-test-slave-49h3"
I20260812 08:03:30.871775 22041 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "397b07826d51403f8bc1085aea72b174"
format_stamp: "Formatted at 2026-08-12 08:03:30 on dist-test-slave-49h3"
I20260812 08:03:30.875344 22041 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:03:30.877681 22058 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:30.878728 22041 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:03:30.878844 22041 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "397b07826d51403f8bc1085aea72b174"
format_stamp: "Formatted at 2026-08-12 08:03:30 on dist-test-slave-49h3"
I20260812 08:03:30.878940 22041 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:30.915139 22041 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:30.915876 22041 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:30.916059 22041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:30.916687 22041 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:30.917853 22041 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:30.917907 22041 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:30.917965 22041 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:30.917991 22041 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:30.926841 22041 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.66:33729
I20260812 08:03:30.926872 22201 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.66:33729 every 8 connection(s)
I20260812 08:03:30.927800 22041 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:03:30.935863   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 22041
I20260812 08:03:30.935971   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:03:30.936465 22202 heartbeater.cc:344] Connected to a master server at 127.0.186.126:32839
I20260812 08:03:30.936710 22202 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:30.937248 22202 heartbeater.cc:507] Master 127.0.186.126:32839 requested a full tablet report, sending...
I20260812 08:03:30.938225   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.67:0
--local_ip_for_outbound_sockets=127.0.186.67
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:32839
--builtin_ntp_servers=127.0.186.84:42325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20260812 08:03:30.938318 21800 ts_manager.cc:194] Registered new tserver with Master: 397b07826d51403f8bc1085aea72b174 (127.0.186.66:33729)
I20260812 08:03:30.938856 21800 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.66:56337
W20260812 08:03:31.045677 22205 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:03:31.045925 22205 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:31.045966 22205 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:31.046048 22205 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:31.049351 22205 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:03:31.049413 22205 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:03:31.049479 22205 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:31.049564 22205 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:03:31.049598 22205 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.67
I20260812 08:03:31.053602 22205 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:42325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.186.67
--webserver_port=0
--tserver_master_addrs=127.0.186.126:32839
--never_fsync=true
--heap_profile_path=/tmp/kudu.22205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:31.054813 22205 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:31.055986 22205 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:31.062341 22214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:31.062340 22211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:31.062373 22212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:31.062547 22205 server_base.cc:1061] running on GCE node
I20260812 08:03:31.062891 22205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:31.063436 22205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:31.064590 22205 hybrid_clock.cc:648] HybridClock initialized: now 1786521811064575 us; error 30 us; skew 500 ppm
I20260812 08:03:31.066244 22205 webserver.cc:533] Webserver started at http://127.0.186.67:39681/ using document root <none> and password file <none>
I20260812 08:03:31.066772 22205 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:31.066824 22205 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:31.066994 22205 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:31.068480 22205 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "2ed16f636eb243e2895d251a6f3d0681"
format_stamp: "Formatted at 2026-08-12 08:03:31 on dist-test-slave-49h3"
I20260812 08:03:31.068940 22205 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "2ed16f636eb243e2895d251a6f3d0681"
format_stamp: "Formatted at 2026-08-12 08:03:31 on dist-test-slave-49h3"
I20260812 08:03:31.072032 22205 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:03:31.074007 22223 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:31.074892 22205 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:31.074987 22205 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "2ed16f636eb243e2895d251a6f3d0681"
format_stamp: "Formatted at 2026-08-12 08:03:31 on dist-test-slave-49h3"
I20260812 08:03:31.075078 22205 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:31.101996 22205 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:31.102720 22205 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:31.102913 22205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:31.103525 22205 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:31.104655 22205 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:31.104704 22205 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:31.104753 22205 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:31.104781 22205 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:31.114287 22205 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.67:46531
I20260812 08:03:31.114318 22369 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.67:46531 every 8 connection(s)
I20260812 08:03:31.115336 22205 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:03:31.123873 22370 heartbeater.cc:344] Connected to a master server at 127.0.186.126:32839
I20260812 08:03:31.124114 22370 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:31.124608 22370 heartbeater.cc:507] Master 127.0.186.126:32839 requested a full tablet report, sending...
I20260812 08:03:31.124892   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 22205
I20260812 08:03:31.124997   745 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:03:31.125737 21800 ts_manager.cc:194] Registered new tserver with Master: 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67:46531)
I20260812 08:03:31.126492 21800 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.67:33019
I20260812 08:03:31.127434   745 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:03:31.135847 21799 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50390:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
W20260812 08:03:31.138296 21799 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 08:03:31.154829 21951 tablet_service.cc:1511] Processing CreateTablet for tablet 4f9ddcbef96949abbbebf26348255dc6 (DEFAULT_TABLE table=TestTable [id=5b95af7d1f8f46b3b3bd12e14ec7f0da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:31.154829 22115 tablet_service.cc:1511] Processing CreateTablet for tablet 4f9ddcbef96949abbbebf26348255dc6 (DEFAULT_TABLE table=TestTable [id=5b95af7d1f8f46b3b3bd12e14ec7f0da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:31.154829 22286 tablet_service.cc:1511] Processing CreateTablet for tablet 4f9ddcbef96949abbbebf26348255dc6 (DEFAULT_TABLE table=TestTable [id=5b95af7d1f8f46b3b3bd12e14ec7f0da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:31.155957 22286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f9ddcbef96949abbbebf26348255dc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:31.155957 22115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f9ddcbef96949abbbebf26348255dc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:31.156365 21951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f9ddcbef96949abbbebf26348255dc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:31.161545 22389 tablet_bootstrap.cc:492] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0: Bootstrap starting.
I20260812 08:03:31.162590 22392 tablet_bootstrap.cc:492] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174: Bootstrap starting.
I20260812 08:03:31.163692 22389 tablet_bootstrap.cc:654] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:31.164305 22392 tablet_bootstrap.cc:654] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:31.164417 22389 log.cc:826] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:31.164785 22390 tablet_bootstrap.cc:492] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: Bootstrap starting.
I20260812 08:03:31.165123 22392 log.cc:826] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:31.166383 22389 tablet_bootstrap.cc:492] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0: No bootstrap required, opened a new log
I20260812 08:03:31.166572 22392 tablet_bootstrap.cc:492] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174: No bootstrap required, opened a new log
I20260812 08:03:31.166591 22389 ts_tablet_manager.cc:1403] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:03:31.166759 22392 ts_tablet_manager.cc:1403] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174: Time spent bootstrapping tablet: real 0.004s	user 0.003s	sys 0.000s
I20260812 08:03:31.166977 22390 tablet_bootstrap.cc:654] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:31.167784 22390 log.cc:826] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:31.169289 22392 raft_consensus.cc:359] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 46531 } }
I20260812 08:03:31.169534 22392 raft_consensus.cc:385] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:31.169601 22392 raft_consensus.cc:740] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 397b07826d51403f8bc1085aea72b174, State: Initialized, Role: FOLLOWER
I20260812 08:03:31.169610 22390 tablet_bootstrap.cc:492] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: No bootstrap required, opened a new log
I20260812 08:03:31.169600 22389 raft_consensus.cc:359] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 46531 } }
I20260812 08:03:31.169806 22390 ts_tablet_manager.cc:1403] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: Time spent bootstrapping tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 08:03:31.169813 22389 raft_consensus.cc:385] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:31.169876 22389 raft_consensus.cc:740] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c43ad693825459f82d4ba3704c3fdb0, State: Initialized, Role: FOLLOWER
I20260812 08:03:31.170113 22392 consensus_queue.cc:260] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 46531 } }
I20260812 08:03:31.170305 22389 consensus_queue.cc:260] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 46531 } }
I20260812 08:03:31.171075 22392 ts_tablet_manager.cc:1434] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174: Time spent starting tablet: real 0.004s	user 0.000s	sys 0.005s
I20260812 08:03:31.171188 22389 ts_tablet_manager.cc:1434] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 08:03:31.171502 22202 heartbeater.cc:499] Master 127.0.186.126:32839 was elected leader, sending a full tablet report...
I20260812 08:03:31.171566 22038 heartbeater.cc:499] Master 127.0.186.126:32839 was elected leader, sending a full tablet report...
I20260812 08:03:31.173504 22390 raft_consensus.cc:359] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 46531 } }
I20260812 08:03:31.173687 22390 raft_consensus.cc:385] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:31.173740 22390 raft_consensus.cc:740] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ed16f636eb243e2895d251a6f3d0681, State: Initialized, Role: FOLLOWER
I20260812 08:03:31.174165 22390 consensus_queue.cc:260] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 46531 } }
I20260812 08:03:31.175032 22370 heartbeater.cc:499] Master 127.0.186.126:32839 was elected leader, sending a full tablet report...
I20260812 08:03:31.175071 22390 ts_tablet_manager.cc:1434] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.003s
W20260812 08:03:31.216279 22371 tablet.cc:2406] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:03:31.229128 22203 tablet.cc:2406] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:03:31.235641 22039 tablet.cc:2406] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:03:31.453006 22398 raft_consensus.cc:493] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:31.453223 22398 raft_consensus.cc:515] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 46531 } }
I20260812 08:03:31.454401 22398 leader_election.cc:290] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c43ad693825459f82d4ba3704c3fdb0 (127.0.186.65:34877), 397b07826d51403f8bc1085aea72b174 (127.0.186.66:33729)
I20260812 08:03:31.459582 22138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9ddcbef96949abbbebf26348255dc6" candidate_uuid: "2ed16f636eb243e2895d251a6f3d0681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "397b07826d51403f8bc1085aea72b174" is_pre_election: true
I20260812 08:03:31.459580 21978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9ddcbef96949abbbebf26348255dc6" candidate_uuid: "2ed16f636eb243e2895d251a6f3d0681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c43ad693825459f82d4ba3704c3fdb0" is_pre_election: true
I20260812 08:03:31.459929 21978 raft_consensus.cc:2468] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ed16f636eb243e2895d251a6f3d0681 in term 0.
I20260812 08:03:31.459930 22138 raft_consensus.cc:2468] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ed16f636eb243e2895d251a6f3d0681 in term 0.
I20260812 08:03:31.460485 22227 leader_election.cc:304] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ed16f636eb243e2895d251a6f3d0681, 4c43ad693825459f82d4ba3704c3fdb0; no voters: 
I20260812 08:03:31.460742 22398 raft_consensus.cc:2804] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:03:31.460834 22398 raft_consensus.cc:493] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:03:31.460875 22398 raft_consensus.cc:3060] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:31.461792 22398 raft_consensus.cc:515] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 46531 } }
I20260812 08:03:31.462158 22398 leader_election.cc:290] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [CANDIDATE]: Term 1 election: Requested vote from peers 4c43ad693825459f82d4ba3704c3fdb0 (127.0.186.65:34877), 397b07826d51403f8bc1085aea72b174 (127.0.186.66:33729)
I20260812 08:03:31.462579 21978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9ddcbef96949abbbebf26348255dc6" candidate_uuid: "2ed16f636eb243e2895d251a6f3d0681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c43ad693825459f82d4ba3704c3fdb0"
I20260812 08:03:31.462605 22138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9ddcbef96949abbbebf26348255dc6" candidate_uuid: "2ed16f636eb243e2895d251a6f3d0681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "397b07826d51403f8bc1085aea72b174"
I20260812 08:03:31.462718 21978 raft_consensus.cc:3060] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:31.462721 22138 raft_consensus.cc:3060] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:31.463991 22138 raft_consensus.cc:2468] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ed16f636eb243e2895d251a6f3d0681 in term 1.
I20260812 08:03:31.464002 21978 raft_consensus.cc:2468] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ed16f636eb243e2895d251a6f3d0681 in term 1.
I20260812 08:03:31.464350 22227 leader_election.cc:304] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ed16f636eb243e2895d251a6f3d0681, 4c43ad693825459f82d4ba3704c3fdb0; no voters: 
I20260812 08:03:31.464552 22398 raft_consensus.cc:2804] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:31.464820 22398 raft_consensus.cc:697] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 1 LEADER]: Becoming Leader. State: Replica: 2ed16f636eb243e2895d251a6f3d0681, State: Running, Role: LEADER
I20260812 08:03:31.465168 22398 consensus_queue.cc:237] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 46531 } }
I20260812 08:03:31.468442 21799 catalog_manager.cc:5719] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67). New cstate: current_term: 1 leader_uuid: "2ed16f636eb243e2895d251a6f3d0681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 46531 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:31.515235   745 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:03:31.516767   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 4c43ad693825459f82d4ba3704c3fdb0 to finish bootstrapping
I20260812 08:03:31.522564   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 397b07826d51403f8bc1085aea72b174 to finish bootstrapping
I20260812 08:03:31.527508   745 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 2ed16f636eb243e2895d251a6f3d0681 to finish bootstrapping
I20260812 08:03:31.544241   745 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 1.1 0.0
I20260812 08:03:31.647377   745 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 1.1 0.0
I20260812 08:03:31.850741   745 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 1.1 0.0
I20260812 08:03:31.876151 22398 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:31.882262 22407 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:32.154062   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 22205
I20260812 08:03:33.382165 22413 raft_consensus.cc:493] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ed16f636eb243e2895d251a6f3d0681)
I20260812 08:03:33.382313 22413 raft_consensus.cc:515] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 46531 } }
I20260812 08:03:33.383505 22413 leader_election.cc:290] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4c43ad693825459f82d4ba3704c3fdb0 (127.0.186.65:34877), 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67:46531)
I20260812 08:03:33.384105 22416 raft_consensus.cc:493] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ed16f636eb243e2895d251a6f3d0681)
I20260812 08:03:33.384238 22416 raft_consensus.cc:515] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 46531 } }
W20260812 08:03:33.384277 22059 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.67:46531: connect: Connection refused (error 111)
W20260812 08:03:33.385219 22059 leader_election.cc:336] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67:46531): Network error: Client connection negotiation failed: client connection to 127.0.186.67:46531: connect: Connection refused (error 111)
I20260812 08:03:33.386231 22416 leader_election.cc:290] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 397b07826d51403f8bc1085aea72b174 (127.0.186.66:33729), 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67:46531)
W20260812 08:03:33.386399 21892 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.67:46531: connect: Connection refused (error 111)
I20260812 08:03:33.387840 21978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9ddcbef96949abbbebf26348255dc6" candidate_uuid: "397b07826d51403f8bc1085aea72b174" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4c43ad693825459f82d4ba3704c3fdb0" is_pre_election: true
I20260812 08:03:33.387975 21978 raft_consensus.cc:2468] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 397b07826d51403f8bc1085aea72b174 in term 1.
W20260812 08:03:33.387984 21892 leader_election.cc:336] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67:46531): Network error: Client connection negotiation failed: client connection to 127.0.186.67:46531: connect: Connection refused (error 111)
I20260812 08:03:33.388329 22061 leader_election.cc:304] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 397b07826d51403f8bc1085aea72b174, 4c43ad693825459f82d4ba3704c3fdb0; no voters: 2ed16f636eb243e2895d251a6f3d0681
I20260812 08:03:33.388587 22413 raft_consensus.cc:2804] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 08:03:33.388652 22413 raft_consensus.cc:493] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2ed16f636eb243e2895d251a6f3d0681)
I20260812 08:03:33.388695 22413 raft_consensus.cc:3060] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:03:33.389757 22413 raft_consensus.cc:515] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 46531 } }
I20260812 08:03:33.389830 22138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9ddcbef96949abbbebf26348255dc6" candidate_uuid: "4c43ad693825459f82d4ba3704c3fdb0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "397b07826d51403f8bc1085aea72b174" is_pre_election: true
I20260812 08:03:33.389971 22138 raft_consensus.cc:2393] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4c43ad693825459f82d4ba3704c3fdb0 in current term 2: Already voted for candidate 397b07826d51403f8bc1085aea72b174 in this term.
I20260812 08:03:33.390161 22413 leader_election.cc:290] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [CANDIDATE]: Term 2 election: Requested vote from peers 4c43ad693825459f82d4ba3704c3fdb0 (127.0.186.65:34877), 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67:46531)
I20260812 08:03:33.390404 21978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9ddcbef96949abbbebf26348255dc6" candidate_uuid: "397b07826d51403f8bc1085aea72b174" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4c43ad693825459f82d4ba3704c3fdb0"
I20260812 08:03:33.390453 21896 leader_election.cc:304] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4c43ad693825459f82d4ba3704c3fdb0; no voters: 2ed16f636eb243e2895d251a6f3d0681, 397b07826d51403f8bc1085aea72b174
I20260812 08:03:33.390523 21978 raft_consensus.cc:3060] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 1 FOLLOWER]: Advancing to term 2
W20260812 08:03:33.391145 22059 leader_election.cc:336] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67:46531): Network error: Client connection negotiation failed: client connection to 127.0.186.67:46531: connect: Connection refused (error 111)
I20260812 08:03:33.391449 21978 raft_consensus.cc:2468] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 397b07826d51403f8bc1085aea72b174 in term 2.
I20260812 08:03:33.391553 22416 raft_consensus.cc:2749] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 08:03:33.391741 22061 leader_election.cc:304] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 397b07826d51403f8bc1085aea72b174, 4c43ad693825459f82d4ba3704c3fdb0; no voters: 2ed16f636eb243e2895d251a6f3d0681
I20260812 08:03:33.391911 22413 raft_consensus.cc:2804] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:03:33.392124 22413 raft_consensus.cc:697] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 2 LEADER]: Becoming Leader. State: Replica: 397b07826d51403f8bc1085aea72b174, State: Running, Role: LEADER
I20260812 08:03:33.392403 22413 consensus_queue.cc:237] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 46531 } }
I20260812 08:03:33.394891 21800 catalog_manager.cc:5719] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 reported cstate change: term changed from 1 to 2, leader changed from 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) to 397b07826d51403f8bc1085aea72b174 (127.0.186.66). New cstate: current_term: 2 leader_uuid: "397b07826d51403f8bc1085aea72b174" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 46531 } health_report { overall_health: UNKNOWN } } }
W20260812 08:03:33.809857 22059 consensus_peers.cc:597] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 -> Peer 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67:46531): Couldn't send request to peer 2ed16f636eb243e2895d251a6f3d0681. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:46531: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 08:03:33.901961 21978 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 08:03:33.902514 22413 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260812 08:03:36.442836 22059 consensus_peers.cc:597] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 -> Peer 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67:46531): Couldn't send request to peer 2ed16f636eb243e2895d251a6f3d0681. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:46531: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260812 08:03:38.448061 22413 consensus_queue.cc:579] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Leader has been unable to successfully communicate with peer 2ed16f636eb243e2895d251a6f3d0681 for more than 5 seconds (5.055s)
W20260812 08:03:38.449026 22059 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.67:46531: connect: Connection refused (error 111) [suppressed 10 similar messages]
W20260812 08:03:38.450614 21799 catalog_manager.cc:5397] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 4f9ddcbef96949abbbebf26348255dc6 with cas_config_opid_index -1: no extra replica candidate found for tablet 4f9ddcbef96949abbbebf26348255dc6 (table TestTable [id=5b95af7d1f8f46b3b3bd12e14ec7f0da]): Not found: could not select location for extra replica: not enough tablet servers to satisfy replica placement policy: the total number of registered tablet servers (3) does not allow for adding an extra replica; consider bringing up more to have at least 4 tablet servers up and running
W20260812 08:03:39.008883 22059 consensus_peers.cc:597] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 -> Peer 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67:46531): Couldn't send request to peer 2ed16f636eb243e2895d251a6f3d0681. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:46531: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260812 08:03:41.636543 22059 consensus_peers.cc:597] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 -> Peer 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67:46531): Couldn't send request to peer 2ed16f636eb243e2895d251a6f3d0681. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:46531: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260812 08:03:43.537247 22059 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.67:46531: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260812 08:03:44.039026 22059 consensus_peers.cc:597] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 -> Peer 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67:46531): Couldn't send request to peer 2ed16f636eb243e2895d251a6f3d0681. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:46531: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260812 08:03:46.279495 22059 consensus_peers.cc:597] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 -> Peer 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67:46531): Couldn't send request to peer 2ed16f636eb243e2895d251a6f3d0681. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:46531: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260812 08:03:48.704550   745 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
I20260812 08:03:48.704871   745 raft_consensus-itest-base.cc:268] Waiting until we've written at least 4MB...
W20260812 08:03:48.728978 22059 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.67:46531: connect: Connection refused (error 111) [suppressed 10 similar messages]
W20260812 08:03:48.730542 22059 consensus_peers.cc:597] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 -> Peer 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67:46531): Couldn't send request to peer 2ed16f636eb243e2895d251a6f3d0681. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:46531: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260812 08:03:48.792207 22203 maintenance_manager.cc:419] P 397b07826d51403f8bc1085aea72b174: Scheduling FlushMRSOp(4f9ddcbef96949abbbebf26348255dc6): 3681818 bytes log retention, and flush 5079305 bytes memory
I20260812 08:03:48.795591 22039 maintenance_manager.cc:419] P 4c43ad693825459f82d4ba3704c3fdb0: Scheduling FlushMRSOp(4f9ddcbef96949abbbebf26348255dc6): 3550247 bytes log retention, and flush 5079305 bytes memory
I20260812 08:03:48.820778   745 raft_consensus-itest-base.cc:274] Waiting for log GC on 397b07826d51403f8bc1085aea72b174
I20260812 08:03:48.855295 22064 maintenance_manager.cc:643] P 397b07826d51403f8bc1085aea72b174: FlushMRSOp(4f9ddcbef96949abbbebf26348255dc6) complete. Timing: real 0.063s	user 0.054s	sys 0.001s Metrics: {"bytes_written":136001,"cfile_init":1,"compiler_manager_pool.queue_time_us":229,"drs_written":1,"lbm_read_time_us":75,"lbm_reads_lt_1ms":4,"lbm_write_time_us":696,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":33,"thread_start_us":248,"threads_started":2}
I20260812 08:03:48.855319 21897 maintenance_manager.cc:643] P 4c43ad693825459f82d4ba3704c3fdb0: FlushMRSOp(4f9ddcbef96949abbbebf26348255dc6) complete. Timing: real 0.059s	user 0.050s	sys 0.005s Metrics: {"bytes_written":136001,"cfile_init":1,"compiler_manager_pool.queue_time_us":231,"drs_written":1,"lbm_read_time_us":76,"lbm_reads_lt_1ms":4,"lbm_write_time_us":765,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":33,"thread_start_us":262,"threads_started":2}
I20260812 08:03:48.856410 22039 maintenance_manager.cc:419] P 4c43ad693825459f82d4ba3704c3fdb0: Scheduling LogGCOp(4f9ddcbef96949abbbebf26348255dc6): free 2629877 bytes of WAL
I20260812 08:03:48.856408 22203 maintenance_manager.cc:419] P 397b07826d51403f8bc1085aea72b174: Scheduling LogGCOp(4f9ddcbef96949abbbebf26348255dc6): free 2761400 bytes of WAL
I20260812 08:03:48.856673 21897 log_reader.cc:385] T 4f9ddcbef96949abbbebf26348255dc6: removed 3 log segments from log reader
I20260812 08:03:48.856704 22064 log_reader.cc:385] T 4f9ddcbef96949abbbebf26348255dc6: removed 3 log segments from log reader
I20260812 08:03:48.856799 22064 log.cc:1079] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/wals/4f9ddcbef96949abbbebf26348255dc6/wal-000000001 (ops 1-9)
I20260812 08:03:48.856797 21897 log.cc:1079] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/4f9ddcbef96949abbbebf26348255dc6/wal-000000001 (ops 1-8)
I20260812 08:03:48.856868 21897 log.cc:1079] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/4f9ddcbef96949abbbebf26348255dc6/wal-000000002 (ops 9-15)
I20260812 08:03:48.856910 22064 log.cc:1079] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/wals/4f9ddcbef96949abbbebf26348255dc6/wal-000000002 (ops 10-16)
I20260812 08:03:48.856952 21897 log.cc:1079] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/4f9ddcbef96949abbbebf26348255dc6/wal-000000003 (ops 16-22)
I20260812 08:03:48.857002 22064 log.cc:1079] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/wals/4f9ddcbef96949abbbebf26348255dc6/wal-000000003 (ops 17-23)
I20260812 08:03:48.858018 22064 maintenance_manager.cc:643] P 397b07826d51403f8bc1085aea72b174: LogGCOp(4f9ddcbef96949abbbebf26348255dc6) complete. Timing: real 0.001s	user 0.000s	sys 0.001s Metrics: {}
I20260812 08:03:48.858035 21897 maintenance_manager.cc:643] P 4c43ad693825459f82d4ba3704c3fdb0: LogGCOp(4f9ddcbef96949abbbebf26348255dc6) complete. Timing: real 0.001s	user 0.000s	sys 0.001s Metrics: {}
I20260812 08:03:48.858383 22203 maintenance_manager.cc:419] P 397b07826d51403f8bc1085aea72b174: Scheduling UndoDeltaBlockGCOp(4f9ddcbef96949abbbebf26348255dc6): 506 bytes on disk
I20260812 08:03:48.858681 22039 maintenance_manager.cc:419] P 4c43ad693825459f82d4ba3704c3fdb0: Scheduling UndoDeltaBlockGCOp(4f9ddcbef96949abbbebf26348255dc6): 506 bytes on disk
I20260812 08:03:48.859037 22064 maintenance_manager.cc:643] P 397b07826d51403f8bc1085aea72b174: UndoDeltaBlockGCOp(4f9ddcbef96949abbbebf26348255dc6) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":81,"lbm_reads_lt_1ms":4}
I20260812 08:03:48.859881 21897 maintenance_manager.cc:643] P 4c43ad693825459f82d4ba3704c3fdb0: UndoDeltaBlockGCOp(4f9ddcbef96949abbbebf26348255dc6) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":65,"lbm_reads_lt_1ms":4}
I20260812 08:03:48.861902   745 raft_consensus-itest-base.cc:279] Log GC complete on 397b07826d51403f8bc1085aea72b174
I20260812 08:03:49.175487 22450 consensus_queue.cc:753] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: The logs necessary to catch up peer 2ed16f636eb243e2895d251a6f3d0681 have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 8..35: Segment 1 which contained index 8 has been GCed)
I20260812 08:03:49.175700 22450 consensus_queue.cc:579] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: The logs necessary to catch up peer 2ed16f636eb243e2895d251a6f3d0681 have been garbage collected. The replica will never be able to catch up
I20260812 08:03:49.178958 22138 consensus_queue.cc:237] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 39, Committed index: 39, Last appended: 2.39, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } }
I20260812 08:03:49.180349 21977 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 2 index: 39. Preceding OpId from leader: term: 2 index: 40. (index mismatch)
I20260812 08:03:49.180775 22450 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 39, Time since last communication: 0.000s
I20260812 08:03:49.182614 22491 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 2 LEADER]: Committing config change with OpId 2.40: config changed from index -1 to 40, VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New config: { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } }
I20260812 08:03:49.183099 21977 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Committing config change with OpId 2.40: config changed from index -1 to 40, VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New config: { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } }
I20260812 08:03:49.184593 21784 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 4f9ddcbef96949abbbebf26348255dc6 with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:49.185316 21796 catalog_manager.cc:5719] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 reported cstate change: config changed from index -1 to 40, VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New cstate: current_term: 2 leader_uuid: "397b07826d51403f8bc1085aea72b174" committed_config { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } health_report { overall_health: HEALTHY } } }
W20260812 08:03:49.188618 21780 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.67:46531: connect: Connection refused (error 111)
I20260812 08:03:49.188922 22138 consensus_queue.cc:237] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 2.40, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } }
W20260812 08:03:49.189471 21780 catalog_manager.cc:4777] TS 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67:46531): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 4f9ddcbef96949abbbebf26348255dc6: Network error: Client connection negotiation failed: client connection to 127.0.186.67:46531: connect: Connection refused (error 111)
W20260812 08:03:49.190833 22059 consensus_peers.cc:597] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 -> Peer 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67:46531): Couldn't send request to peer 2ed16f636eb243e2895d251a6f3d0681. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.67:46531: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 08:03:49.190913 21977 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 2 index: 40. Preceding OpId from leader: term: 2 index: 41. (index mismatch)
I20260812 08:03:49.191272 22450 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 41, Last known committed idx: 40, Time since last communication: 0.000s
I20260812 08:03:49.192620 22491 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 2 LEADER]: Committing config change with OpId 2.41: config changed from index 40 to 41, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New config: { opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } } }
I20260812 08:03:49.193024 21977 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Committing config change with OpId 2.41: config changed from index 40 to 41, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New config: { opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } } }
I20260812 08:03:49.194557 21784 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 4f9ddcbef96949abbbebf26348255dc6 with cas_config_opid_index 40: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:49.194850 21796 catalog_manager.cc:5719] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 reported cstate change: config changed from index 40 to 41, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New cstate: current_term: 2 leader_uuid: "397b07826d51403f8bc1085aea72b174" committed_config { opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260812 08:03:50.363075 21780 catalog_manager.cc:4777] TS 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67:46531): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 4f9ddcbef96949abbbebf26348255dc6: Network error: Client connection negotiation failed: client connection to 127.0.186.67:46531: connect: Connection refused (error 111)
I20260812 08:03:50.863386   745 raft_consensus-itest-base.cc:295] Servers converged with original term 2
I20260812 08:03:50.868106   745 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskamFhYX/build/debug/bin/kudu
/tmp/dist-test-taskamFhYX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.67:46531
--local_ip_for_outbound_sockets=127.0.186.67
--tserver_master_addrs=127.0.186.126:32839
--webserver_port=39681
--webserver_interface=127.0.186.67
--builtin_ntp_servers=127.0.186.84:42325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20260812 08:03:50.975114 22528 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:03:50.975402 22528 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:50.975445 22528 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:50.975510 22528 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:50.978958 22528 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:03:50.979022 22528 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:03:50.979075 22528 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:50.979171 22528 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:03:50.979205 22528 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.67
I20260812 08:03:50.983158 22528 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.84:42325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=5
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.67:46531
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.186.67
--webserver_port=39681
--tserver_master_addrs=127.0.186.126:32839
--never_fsync=true
--heap_profile_path=/tmp/kudu.22528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:50.984313 22528 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:50.985538 22528 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:50.991936 22535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:50.991933 22538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:50.991938 22534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:50.992175 22528 server_base.cc:1061] running on GCE node
I20260812 08:03:50.992614 22528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:50.993182 22528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:50.994336 22528 hybrid_clock.cc:648] HybridClock initialized: now 1786521830994321 us; error 26 us; skew 500 ppm
I20260812 08:03:50.996230 22528 webserver.cc:533] Webserver started at http://127.0.186.67:39681/ using document root <none> and password file <none>
I20260812 08:03:50.996833 22528 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:50.996917 22528 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:51.000355 22528 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:51.002238 22546 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:51.003175 22528 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:51.003283 22528 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "2ed16f636eb243e2895d251a6f3d0681"
format_stamp: "Formatted at 2026-08-12 08:03:31 on dist-test-slave-49h3"
I20260812 08:03:51.003685 22528 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:51.019430 22528 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:51.020013 22528 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:51.020164 22528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:51.020695 22528 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:51.022060 22554 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:03:51.026005 22528 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:51.026072 22528 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.001s	sys 0.000s
I20260812 08:03:51.026135 22528 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:51.027565 22528 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:51.027611 22528 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
I20260812 08:03:51.027923 22554 tablet_bootstrap.cc:492] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: Bootstrap starting.
I20260812 08:03:51.035383 22554 log.cc:826] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:51.038177 22528 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.67:46531
I20260812 08:03:51.038200 22695 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.186.67:46531 every 8 connection(s)
I20260812 08:03:51.038331 22554 tablet_bootstrap.cc:492] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:03:51.038692 22554 tablet_bootstrap.cc:492] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: Bootstrap complete.
I20260812 08:03:51.039121 22554 ts_tablet_manager.cc:1403] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: Time spent bootstrapping tablet: real 0.011s	user 0.005s	sys 0.004s
I20260812 08:03:51.039250 22528 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:03:51.042352 22554 raft_consensus.cc:359] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 46531 } }
I20260812 08:03:51.042701 22554 raft_consensus.cc:740] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ed16f636eb243e2895d251a6f3d0681, State: Initialized, Role: FOLLOWER
I20260812 08:03:51.043111 22554 consensus_queue.cc:260] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 46531 } }
I20260812 08:03:51.043892 22554 ts_tablet_manager.cc:1434] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260812 08:03:51.044837   745 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskamFhYX/build/debug/bin/kudu as pid 22528
I20260812 08:03:51.046989 22696 heartbeater.cc:344] Connected to a master server at 127.0.186.126:32839
I20260812 08:03:51.047195 22696 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:51.047648 22696 heartbeater.cc:507] Master 127.0.186.126:32839 requested a full tablet report, sending...
I20260812 08:03:51.049263 21797 ts_manager.cc:194] Re-registered known tserver with Master: 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67:46531)
I20260812 08:03:51.050034 21797 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.186.67:43845
I20260812 08:03:51.051555 22696 heartbeater.cc:499] Master 127.0.186.126:32839 was elected leader, sending a full tablet report...
I20260812 08:03:51.275686 22638 raft_consensus.cc:3060] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:03:51.278102 22638 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 40. (index mismatch)
I20260812 08:03:51.279138 22521 consensus_queue.cc:1050] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Got LMP mismatch error from peer: Peer: permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 41, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:51.279402 22521 consensus_queue.cc:753] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: The logs necessary to catch up peer 2ed16f636eb243e2895d251a6f3d0681 have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..40: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:51.279455 22521 consensus_queue.cc:579] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: The logs necessary to catch up peer 2ed16f636eb243e2895d251a6f3d0681 have been garbage collected. The replica will never be able to catch up
I20260812 08:03:51.281868 22138 consensus_queue.cc:237] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 41, Committed index: 41, Last appended: 2.41, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } }
I20260812 08:03:51.283103 21977 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 2 index: 41. Preceding OpId from leader: term: 2 index: 42. (index mismatch)
I20260812 08:03:51.283510 22521 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 42, Last known committed idx: 41, Time since last communication: 0.000s
I20260812 08:03:51.285236 22703 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 2 LEADER]: Committing config change with OpId 2.42: config changed from index 41 to 42, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New config: { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } }
I20260812 08:03:51.285734 21977 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Committing config change with OpId 2.42: config changed from index 41 to 42, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New config: { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } }
I20260812 08:03:51.286741 21784 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 4f9ddcbef96949abbbebf26348255dc6 with cas_config_opid_index 41: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:51.287676 21796 catalog_manager.cc:5719] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 reported cstate change: config changed from index 41 to 42, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New cstate: current_term: 2 leader_uuid: "397b07826d51403f8bc1085aea72b174" committed_config { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:51.291271 22138 consensus_queue.cc:237] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 42, Committed index: 42, Last appended: 2.42, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } }
I20260812 08:03:51.292469 21977 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 2 index: 42. Preceding OpId from leader: term: 2 index: 43. (index mismatch)
I20260812 08:03:51.292527 22638 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 43. (index mismatch)
I20260812 08:03:51.292902 22703 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 43, Last known committed idx: 42, Time since last communication: 0.000s
I20260812 08:03:51.293078 22521 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 43, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:51.293298 22521 consensus_queue.cc:753] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: The logs necessary to catch up peer 2ed16f636eb243e2895d251a6f3d0681 have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..42: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:51.293352 22521 consensus_queue.cc:579] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: The logs necessary to catch up peer 2ed16f636eb243e2895d251a6f3d0681 have been garbage collected. The replica will never be able to catch up
I20260812 08:03:51.294611 22703 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 2 LEADER]: Committing config change with OpId 2.43: config changed from index 42 to 43, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New config: { opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } } }
I20260812 08:03:51.295153 21977 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Committing config change with OpId 2.43: config changed from index 42 to 43, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New config: { opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } } }
I20260812 08:03:51.296322 21784 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 4f9ddcbef96949abbbebf26348255dc6 with cas_config_opid_index 42: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:51.296783 21796 catalog_manager.cc:5719] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 reported cstate change: config changed from index 42 to 43, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New cstate: current_term: 2 leader_uuid: "397b07826d51403f8bc1085aea72b174" committed_config { opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } health_report { overall_health: FAILED_UNRECOVERABLE } } }
I20260812 08:03:51.299350 22138 logging.cc:436] LogThrottler TrackedPeer WAL GC: suppressed but not reported on 1 messages since previous log ~0 seconds ago
I20260812 08:03:51.299520 22138 consensus_queue.cc:237] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 43, Committed index: 43, Last appended: 2.43, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } }
I20260812 08:03:51.300521 21977 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 2 index: 43. Preceding OpId from leader: term: 2 index: 44. (index mismatch)
I20260812 08:03:51.300868 22521 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 44, Last known committed idx: 43, Time since last communication: 0.000s
I20260812 08:03:51.302438 22703 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 2 LEADER]: Committing config change with OpId 2.44: config changed from index 43 to 44, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } }
I20260812 08:03:51.302898 21977 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Committing config change with OpId 2.44: config changed from index 43 to 44, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } }
I20260812 08:03:51.303903 21784 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 4f9ddcbef96949abbbebf26348255dc6 with cas_config_opid_index 43: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:51.305078 21797 catalog_manager.cc:5719] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 reported cstate change: config changed from index 43 to 44, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New cstate: current_term: 2 leader_uuid: "397b07826d51403f8bc1085aea72b174" committed_config { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:51.307719 22138 consensus_queue.cc:237] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 44, Committed index: 44, Last appended: 2.44, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } }
I20260812 08:03:51.308844 22638 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 45. (index mismatch)
I20260812 08:03:51.308848 21977 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 2 index: 44. Preceding OpId from leader: term: 2 index: 45. (index mismatch)
I20260812 08:03:51.309243 22521 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 45, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:51.309413 22703 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 45, Last known committed idx: 44, Time since last communication: 0.000s
I20260812 08:03:51.309490 22521 consensus_queue.cc:753] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: The logs necessary to catch up peer 2ed16f636eb243e2895d251a6f3d0681 have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..44: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:51.309609 22521 consensus_queue.cc:579] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: The logs necessary to catch up peer 2ed16f636eb243e2895d251a6f3d0681 have been garbage collected. The replica will never be able to catch up
I20260812 08:03:51.311051 22711 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 2 LEADER]: Committing config change with OpId 2.45: config changed from index 44 to 45, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } } }
I20260812 08:03:51.311497 21977 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Committing config change with OpId 2.45: config changed from index 44 to 45, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } } }
I20260812 08:03:51.312232 21784 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 4f9ddcbef96949abbbebf26348255dc6 with cas_config_opid_index 44: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:51.313028 21797 catalog_manager.cc:5719] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 reported cstate change: config changed from index 44 to 45, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New cstate: current_term: 2 leader_uuid: "397b07826d51403f8bc1085aea72b174" committed_config { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } health_report { overall_health: FAILED_UNRECOVERABLE } } }
I20260812 08:03:51.315481 22138 logging.cc:436] LogThrottler TrackedPeer WAL GC: suppressed but not reported on 1 messages since previous log ~0 seconds ago
I20260812 08:03:51.315642 22138 consensus_queue.cc:237] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 45, Committed index: 45, Last appended: 2.45, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 46 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } }
I20260812 08:03:51.316612 21977 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 2 index: 45. Preceding OpId from leader: term: 2 index: 46. (index mismatch)
I20260812 08:03:51.316989 22711 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 46, Last known committed idx: 45, Time since last communication: 0.000s
I20260812 08:03:51.318365 22703 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 2 LEADER]: Committing config change with OpId 2.46: config changed from index 45 to 46, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New config: { opid_index: 46 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } }
I20260812 08:03:51.318894 21977 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Committing config change with OpId 2.46: config changed from index 45 to 46, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New config: { opid_index: 46 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } }
I20260812 08:03:51.319605 21784 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 4f9ddcbef96949abbbebf26348255dc6 with cas_config_opid_index 45: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:51.320202 21797 catalog_manager.cc:5719] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 reported cstate change: config changed from index 45 to 46, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New cstate: current_term: 2 leader_uuid: "397b07826d51403f8bc1085aea72b174" committed_config { opid_index: 46 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:51.323040 22138 consensus_queue.cc:237] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 46, Committed index: 46, Last appended: 2.46, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 47 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } }
I20260812 08:03:51.324087 21977 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 2 index: 46. Preceding OpId from leader: term: 2 index: 47. (index mismatch)
I20260812 08:03:51.324182 22638 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 47. (index mismatch)
I20260812 08:03:51.324430 22711 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 47, Last known committed idx: 46, Time since last communication: 0.000s
I20260812 08:03:51.324671 22521 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 47, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:51.324888 22521 consensus_queue.cc:753] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: The logs necessary to catch up peer 2ed16f636eb243e2895d251a6f3d0681 have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..46: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:51.324954 22521 consensus_queue.cc:579] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: The logs necessary to catch up peer 2ed16f636eb243e2895d251a6f3d0681 have been garbage collected. The replica will never be able to catch up
I20260812 08:03:51.326042 22521 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 2 LEADER]: Committing config change with OpId 2.47: config changed from index 46 to 47, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New config: { opid_index: 47 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } } }
I20260812 08:03:51.326483 21977 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Committing config change with OpId 2.47: config changed from index 46 to 47, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New config: { opid_index: 47 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } } }
I20260812 08:03:51.327591 21784 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 4f9ddcbef96949abbbebf26348255dc6 with cas_config_opid_index 46: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:51.328204 21797 catalog_manager.cc:5719] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 reported cstate change: config changed from index 46 to 47, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New cstate: current_term: 2 leader_uuid: "397b07826d51403f8bc1085aea72b174" committed_config { opid_index: 47 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } health_report { overall_health: FAILED_UNRECOVERABLE } } }
I20260812 08:03:51.330523 22138 logging.cc:436] LogThrottler TrackedPeer WAL GC: suppressed but not reported on 1 messages since previous log ~0 seconds ago
I20260812 08:03:51.330713 22138 consensus_queue.cc:237] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 47, Committed index: 47, Last appended: 2.47, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 48 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } }
I20260812 08:03:51.331758 21977 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 2 index: 47. Preceding OpId from leader: term: 2 index: 48. (index mismatch)
I20260812 08:03:51.332131 22703 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 48, Last known committed idx: 47, Time since last communication: 0.000s
I20260812 08:03:51.333555 22521 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 2 LEADER]: Committing config change with OpId 2.48: config changed from index 47 to 48, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New config: { opid_index: 48 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } }
I20260812 08:03:51.334024 21977 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Committing config change with OpId 2.48: config changed from index 47 to 48, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New config: { opid_index: 48 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } }
I20260812 08:03:51.334877 21784 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 4f9ddcbef96949abbbebf26348255dc6 with cas_config_opid_index 47: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:51.335521 21796 catalog_manager.cc:5719] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 reported cstate change: config changed from index 47 to 48, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New cstate: current_term: 2 leader_uuid: "397b07826d51403f8bc1085aea72b174" committed_config { opid_index: 48 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:51.338281 22138 consensus_queue.cc:237] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 48, Committed index: 48, Last appended: 2.48, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } }
I20260812 08:03:51.339488 21977 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 2 index: 48. Preceding OpId from leader: term: 2 index: 49. (index mismatch)
I20260812 08:03:51.339521 22638 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 49. (index mismatch)
I20260812 08:03:51.339864 22521 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 49, Last known committed idx: 48, Time since last communication: 0.000s
I20260812 08:03:51.340046 22703 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 49, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:51.340260 22703 consensus_queue.cc:753] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: The logs necessary to catch up peer 2ed16f636eb243e2895d251a6f3d0681 have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..48: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:51.340317 22703 consensus_queue.cc:579] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: The logs necessary to catch up peer 2ed16f636eb243e2895d251a6f3d0681 have been garbage collected. The replica will never be able to catch up
I20260812 08:03:51.341619 22703 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 2 LEADER]: Committing config change with OpId 2.49: config changed from index 48 to 49, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New config: { opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } } }
I20260812 08:03:51.342082 21977 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Committing config change with OpId 2.49: config changed from index 48 to 49, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New config: { opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } } }
I20260812 08:03:51.343443 21784 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 4f9ddcbef96949abbbebf26348255dc6 with cas_config_opid_index 48: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:51.344177 21799 catalog_manager.cc:5719] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 reported cstate change: config changed from index 48 to 49, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New cstate: current_term: 2 leader_uuid: "397b07826d51403f8bc1085aea72b174" committed_config { opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } } }
I20260812 08:03:51.346793 22138 logging.cc:436] LogThrottler TrackedPeer WAL GC: suppressed but not reported on 1 messages since previous log ~0 seconds ago
I20260812 08:03:51.346930 22138 consensus_queue.cc:237] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 49, Committed index: 49, Last appended: 2.49, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 50 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } }
I20260812 08:03:51.347766 21977 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 2 index: 49. Preceding OpId from leader: term: 2 index: 50. (index mismatch)
I20260812 08:03:51.348078 22521 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 50, Last known committed idx: 49, Time since last communication: 0.000s
I20260812 08:03:51.349480 22703 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 2 LEADER]: Committing config change with OpId 2.50: config changed from index 49 to 50, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New config: { opid_index: 50 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } }
I20260812 08:03:51.349978 21977 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Committing config change with OpId 2.50: config changed from index 49 to 50, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New config: { opid_index: 50 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } }
I20260812 08:03:51.350778 21784 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 4f9ddcbef96949abbbebf26348255dc6 with cas_config_opid_index 49: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:51.351755 21796 catalog_manager.cc:5719] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 reported cstate change: config changed from index 49 to 50, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New cstate: current_term: 2 leader_uuid: "397b07826d51403f8bc1085aea72b174" committed_config { opid_index: 50 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:51.354494 22138 consensus_queue.cc:237] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 50, Committed index: 50, Last appended: 2.50, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 51 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } }
I20260812 08:03:51.355711 21977 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 2 index: 50. Preceding OpId from leader: term: 2 index: 51. (index mismatch)
I20260812 08:03:51.355721 22638 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 51. (index mismatch)
I20260812 08:03:51.356096 22703 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 51, Last known committed idx: 50, Time since last communication: 0.000s
I20260812 08:03:51.356279 22521 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 51, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:51.356479 22521 consensus_queue.cc:753] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: The logs necessary to catch up peer 2ed16f636eb243e2895d251a6f3d0681 have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..50: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:51.356552 22521 consensus_queue.cc:579] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: The logs necessary to catch up peer 2ed16f636eb243e2895d251a6f3d0681 have been garbage collected. The replica will never be able to catch up
I20260812 08:03:51.357993 22711 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 2 LEADER]: Committing config change with OpId 2.51: config changed from index 50 to 51, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New config: { opid_index: 51 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } } }
I20260812 08:03:51.358438 21977 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Committing config change with OpId 2.51: config changed from index 50 to 51, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New config: { opid_index: 51 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } } }
I20260812 08:03:51.359578 21784 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 4f9ddcbef96949abbbebf26348255dc6 with cas_config_opid_index 50: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:51.360154 21796 catalog_manager.cc:5719] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 reported cstate change: config changed from index 50 to 51, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New cstate: current_term: 2 leader_uuid: "397b07826d51403f8bc1085aea72b174" committed_config { opid_index: 51 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } health_report { overall_health: FAILED_UNRECOVERABLE } } }
I20260812 08:03:51.362730 22138 logging.cc:436] LogThrottler TrackedPeer WAL GC: suppressed but not reported on 1 messages since previous log ~0 seconds ago
I20260812 08:03:51.362895 22138 consensus_queue.cc:237] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 51, Committed index: 51, Last appended: 2.51, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 52 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } }
I20260812 08:03:51.363795 21977 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 2 index: 51. Preceding OpId from leader: term: 2 index: 52. (index mismatch)
I20260812 08:03:51.364127 22703 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 52, Last known committed idx: 51, Time since last communication: 0.000s
I20260812 08:03:51.365504 22711 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 2 LEADER]: Committing config change with OpId 2.52: config changed from index 51 to 52, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New config: { opid_index: 52 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } }
I20260812 08:03:51.366209 21977 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Committing config change with OpId 2.52: config changed from index 51 to 52, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New config: { opid_index: 52 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } }
I20260812 08:03:51.366882 21784 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 4f9ddcbef96949abbbebf26348255dc6 with cas_config_opid_index 51: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:51.367671 21799 catalog_manager.cc:5719] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 reported cstate change: config changed from index 51 to 52, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New cstate: current_term: 2 leader_uuid: "397b07826d51403f8bc1085aea72b174" committed_config { opid_index: 52 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:51.370565 22203 maintenance_manager.cc:419] P 397b07826d51403f8bc1085aea72b174: Scheduling LogGCOp(4f9ddcbef96949abbbebf26348255dc6): free 920418 bytes of WAL
I20260812 08:03:51.370616 22039 maintenance_manager.cc:419] P 4c43ad693825459f82d4ba3704c3fdb0: Scheduling LogGCOp(4f9ddcbef96949abbbebf26348255dc6): free 1709345 bytes of WAL
I20260812 08:03:51.370574 22138 consensus_queue.cc:237] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 52, Committed index: 52, Last appended: 2.52, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } }
I20260812 08:03:51.370775 22064 maintenance_manager.cc:643] P 397b07826d51403f8bc1085aea72b174: LogGCOp(4f9ddcbef96949abbbebf26348255dc6) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"spinlock_wait_cycles":28416}
I20260812 08:03:51.371017 21897 log_reader.cc:385] T 4f9ddcbef96949abbbebf26348255dc6: removed 2 log segments from log reader
I20260812 08:03:51.371093 21897 log.cc:1079] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/4f9ddcbef96949abbbebf26348255dc6/wal-000000004 (ops 23-29)
I20260812 08:03:51.371137 21897 log.cc:1079] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/wal/wals/4f9ddcbef96949abbbebf26348255dc6/wal-000000005 (ops 30-35)
I20260812 08:03:51.371727 21897 maintenance_manager.cc:643] P 4c43ad693825459f82d4ba3704c3fdb0: LogGCOp(4f9ddcbef96949abbbebf26348255dc6) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 08:03:51.371799 22638 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 53. (index mismatch)
I20260812 08:03:51.371799 21977 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 2 index: 52. Preceding OpId from leader: term: 2 index: 53. (index mismatch)
I20260812 08:03:51.372180 22703 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 53, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:51.372334 22711 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 53, Last known committed idx: 52, Time since last communication: 0.000s
I20260812 08:03:51.372591 22703 consensus_queue.cc:753] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: The logs necessary to catch up peer 2ed16f636eb243e2895d251a6f3d0681 have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..52: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:51.372654 22703 consensus_queue.cc:579] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: The logs necessary to catch up peer 2ed16f636eb243e2895d251a6f3d0681 have been garbage collected. The replica will never be able to catch up
I20260812 08:03:51.373816 22703 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 2 LEADER]: Committing config change with OpId 2.53: config changed from index 52 to 53, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New config: { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } } }
I20260812 08:03:51.374333 21977 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Committing config change with OpId 2.53: config changed from index 52 to 53, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New config: { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } } }
I20260812 08:03:51.375213 21784 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 4f9ddcbef96949abbbebf26348255dc6 with cas_config_opid_index 52: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:51.376111 21799 catalog_manager.cc:5719] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 reported cstate change: config changed from index 52 to 53, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New cstate: current_term: 2 leader_uuid: "397b07826d51403f8bc1085aea72b174" committed_config { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } } }
I20260812 08:03:51.378520 22138 logging.cc:436] LogThrottler TrackedPeer WAL GC: suppressed but not reported on 1 messages since previous log ~0 seconds ago
I20260812 08:03:51.378645 22138 consensus_queue.cc:237] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 53, Committed index: 53, Last appended: 2.53, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 54 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } }
I20260812 08:03:51.379552 21977 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 2 index: 53. Preceding OpId from leader: term: 2 index: 54. (index mismatch)
I20260812 08:03:51.379911 22711 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 54, Last known committed idx: 53, Time since last communication: 0.000s
I20260812 08:03:51.381238 22703 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 2 LEADER]: Committing config change with OpId 2.54: config changed from index 53 to 54, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New config: { opid_index: 54 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } }
I20260812 08:03:51.381732 21977 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Committing config change with OpId 2.54: config changed from index 53 to 54, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New config: { opid_index: 54 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } }
I20260812 08:03:51.382467 21784 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 4f9ddcbef96949abbbebf26348255dc6 with cas_config_opid_index 53: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:51.383181 21796 catalog_manager.cc:5719] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 reported cstate change: config changed from index 53 to 54, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New cstate: current_term: 2 leader_uuid: "397b07826d51403f8bc1085aea72b174" committed_config { opid_index: 54 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:51.385571 22138 consensus_queue.cc:237] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 54, Committed index: 54, Last appended: 2.54, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 55 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } }
I20260812 08:03:51.386754 22638 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 55. (index mismatch)
I20260812 08:03:51.386768 21977 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 2 index: 54. Preceding OpId from leader: term: 2 index: 55. (index mismatch)
I20260812 08:03:51.387171 22703 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 55, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:51.387370 22711 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 55, Last known committed idx: 54, Time since last communication: 0.000s
I20260812 08:03:51.387421 22703 consensus_queue.cc:753] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: The logs necessary to catch up peer 2ed16f636eb243e2895d251a6f3d0681 have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..54: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:51.387492 22703 consensus_queue.cc:579] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: The logs necessary to catch up peer 2ed16f636eb243e2895d251a6f3d0681 have been garbage collected. The replica will never be able to catch up
I20260812 08:03:51.388904 22521 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 2 LEADER]: Committing config change with OpId 2.55: config changed from index 54 to 55, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New config: { opid_index: 55 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } } }
I20260812 08:03:51.389362 21977 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Committing config change with OpId 2.55: config changed from index 54 to 55, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New config: { opid_index: 55 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } } }
I20260812 08:03:51.390650 21784 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 4f9ddcbef96949abbbebf26348255dc6 with cas_config_opid_index 54: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:51.390763 21796 catalog_manager.cc:5719] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 reported cstate change: config changed from index 54 to 55, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New cstate: current_term: 2 leader_uuid: "397b07826d51403f8bc1085aea72b174" committed_config { opid_index: 55 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } health_report { overall_health: FAILED_UNRECOVERABLE } } }
I20260812 08:03:51.392966 22138 logging.cc:436] LogThrottler TrackedPeer WAL GC: suppressed but not reported on 1 messages since previous log ~0 seconds ago
I20260812 08:03:51.393110 22138 consensus_queue.cc:237] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 55, Committed index: 55, Last appended: 2.55, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } }
I20260812 08:03:51.394515 21977 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 2 index: 55. Preceding OpId from leader: term: 2 index: 56. (index mismatch)
I20260812 08:03:51.394863 22711 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 56, Last known committed idx: 55, Time since last communication: 0.000s
I20260812 08:03:51.396054 22703 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 2 LEADER]: Committing config change with OpId 2.56: config changed from index 55 to 56, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New config: { opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } }
I20260812 08:03:51.396490 21977 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Committing config change with OpId 2.56: config changed from index 55 to 56, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New config: { opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } }
I20260812 08:03:51.397477 21784 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 4f9ddcbef96949abbbebf26348255dc6 with cas_config_opid_index 55: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:51.397843 21799 catalog_manager.cc:5719] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 reported cstate change: config changed from index 55 to 56, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New cstate: current_term: 2 leader_uuid: "397b07826d51403f8bc1085aea72b174" committed_config { opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:51.400279 22138 consensus_queue.cc:237] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 56, Committed index: 56, Last appended: 2.56, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 57 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } }
I20260812 08:03:51.401562 21977 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 2 index: 56. Preceding OpId from leader: term: 2 index: 57. (index mismatch)
I20260812 08:03:51.401590 22638 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 57. (index mismatch)
I20260812 08:03:51.401950 22703 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 57, Last known committed idx: 56, Time since last communication: 0.000s
I20260812 08:03:51.402107 22711 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 57, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:51.402312 22711 consensus_queue.cc:753] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: The logs necessary to catch up peer 2ed16f636eb243e2895d251a6f3d0681 have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 2..56: Segment 1 which contained index 2 has been GCed)
I20260812 08:03:51.402359 22711 consensus_queue.cc:579] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: The logs necessary to catch up peer 2ed16f636eb243e2895d251a6f3d0681 have been garbage collected. The replica will never be able to catch up
I20260812 08:03:51.403532 22703 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 2 LEADER]: Committing config change with OpId 2.57: config changed from index 56 to 57, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New config: { opid_index: 57 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } } }
I20260812 08:03:51.404100 21977 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Committing config change with OpId 2.57: config changed from index 56 to 57, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New config: { opid_index: 57 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } } }
I20260812 08:03:51.405052 21784 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 4f9ddcbef96949abbbebf26348255dc6 with cas_config_opid_index 56: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:51.405805 21799 catalog_manager.cc:5719] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 reported cstate change: config changed from index 56 to 57, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New cstate: current_term: 2 leader_uuid: "397b07826d51403f8bc1085aea72b174" committed_config { opid_index: 57 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } health_report { overall_health: FAILED_UNRECOVERABLE } } }
I20260812 08:03:51.407913 22138 logging.cc:436] LogThrottler TrackedPeer WAL GC: suppressed but not reported on 1 messages since previous log ~0 seconds ago
I20260812 08:03:51.408061 22138 consensus_queue.cc:237] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 57, Committed index: 57, Last appended: 2.57, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 58 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } }
I20260812 08:03:51.408938 21977 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 2 index: 57. Preceding OpId from leader: term: 2 index: 58. (index mismatch)
I20260812 08:03:51.409300 22711 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 58, Last known committed idx: 57, Time since last communication: 0.000s
I20260812 08:03:51.410849 22703 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 2 LEADER]: Committing config change with OpId 2.58: config changed from index 57 to 58, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New config: { opid_index: 58 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } }
I20260812 08:03:51.412276 21784 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 4f9ddcbef96949abbbebf26348255dc6 with cas_config_opid_index 57: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:03:51.412309 21977 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Committing config change with OpId 2.58: config changed from index 57 to 58, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New config: { opid_index: 58 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } }
I20260812 08:03:51.413058 21796 catalog_manager.cc:5719] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 reported cstate change: config changed from index 57 to 58, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) evicted. New cstate: current_term: 2 leader_uuid: "397b07826d51403f8bc1085aea72b174" committed_config { opid_index: 58 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:51.414945 22610 tablet_service.cc:1558] Processing DeleteTablet for tablet 4f9ddcbef96949abbbebf26348255dc6 with delete_type TABLET_DATA_TOMBSTONED (TS 2ed16f636eb243e2895d251a6f3d0681 not found in new config with opid_index 40) from {username='slave'} at 127.0.0.1:52542
I20260812 08:03:51.415760 22714 tablet_replica.cc:333] stopping tablet replica
I20260812 08:03:51.416064 22714 raft_consensus.cc:2243] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 08:03:51.416390 22714 raft_consensus.cc:2272] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 08:03:51.416564 22138 consensus_queue.cc:237] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 58, Committed index: 58, Last appended: 2.58, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } }
I20260812 08:03:51.417311 22714 ts_tablet_manager.cc:1916] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 08:03:51.417639 21977 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 2 index: 58. Preceding OpId from leader: term: 2 index: 59. (index mismatch)
I20260812 08:03:51.418022 22521 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 59, Last known committed idx: 58, Time since last communication: 0.000s
W20260812 08:03:51.418219 22059 consensus_peers.cc:597] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 -> Peer 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67:46531): Couldn't send request to peer 2ed16f636eb243e2895d251a6f3d0681. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:03:51.419495 22703 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 2 LEADER]: Committing config change with OpId 2.59: config changed from index 58 to 59, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } } }
I20260812 08:03:51.419909 21977 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Committing config change with OpId 2.59: config changed from index 58 to 59, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } } }
I20260812 08:03:51.420189 22714 ts_tablet_manager.cc:1929] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.1
I20260812 08:03:51.420293 22714 log.cc:1199] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: Deleting WAL directory at /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/4f9ddcbef96949abbbebf26348255dc6
I20260812 08:03:51.420878 21784 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 4f9ddcbef96949abbbebf26348255dc6 with cas_config_opid_index 58: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:03:51.420881 21780 catalog_manager.cc:5050] TS 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67:46531): tablet 4f9ddcbef96949abbbebf26348255dc6 (table TestTable [id=5b95af7d1f8f46b3b3bd12e14ec7f0da]) successfully deleted
I20260812 08:03:51.421849 21796 catalog_manager.cc:5719] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 reported cstate change: config changed from index 58 to 59, NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) added. New cstate: current_term: 2 leader_uuid: "397b07826d51403f8bc1085aea72b174" committed_config { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } } }
I20260812 08:03:51.891552 22715 ts_tablet_manager.cc:933] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: Initiating tablet copy from peer 397b07826d51403f8bc1085aea72b174 (127.0.186.66:33729)
I20260812 08:03:51.892163 22715 tablet_copy_client.cc:323] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: tablet copy: Beginning tablet copy session from remote peer at address 127.0.186.66:33729
I20260812 08:03:51.895875 22162 tablet_copy_service.cc:140] P 397b07826d51403f8bc1085aea72b174: Received BeginTabletCopySession request for tablet 4f9ddcbef96949abbbebf26348255dc6 from peer 2ed16f636eb243e2895d251a6f3d0681 ({username='slave'} at 127.0.186.67:52511)
I20260812 08:03:51.896021 22162 tablet_copy_service.cc:161] P 397b07826d51403f8bc1085aea72b174: Beginning new tablet copy session on tablet 4f9ddcbef96949abbbebf26348255dc6 from peer 2ed16f636eb243e2895d251a6f3d0681 at {username='slave'} at 127.0.186.67:52511: session id = 2ed16f636eb243e2895d251a6f3d0681-4f9ddcbef96949abbbebf26348255dc6
I20260812 08:03:51.897509 22162 tablet_copy_source_session.cc:215] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174: Tablet Copy: opened 5 blocks and 3 log segments
I20260812 08:03:51.899053 22715 ts_tablet_manager.cc:1916] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260812 08:03:51.900929 22715 ts_tablet_manager.cc:1929] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.1
I20260812 08:03:51.901255 22715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f9ddcbef96949abbbebf26348255dc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:51.902305 22715 tablet_copy_client.cc:806] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: tablet copy: Starting download of 5 data blocks...
I20260812 08:03:51.909291 22715 tablet_copy_client.cc:670] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: tablet copy: Starting download of 3 WAL segments...
I20260812 08:03:51.917310 22715 tablet_copy_client.cc:538] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 08:03:51.919071 22715 tablet_bootstrap.cc:492] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: Bootstrap starting.
I20260812 08:03:51.931284 22715 tablet_bootstrap.cc:492] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: Bootstrap replayed 1/3 log segments. Stats: ops{read=7 overwritten=0 applied=4 ignored=4} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=2. Pending: 3 replicates
I20260812 08:03:51.936398 22715 tablet_bootstrap.cc:492] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: Bootstrap replayed 2/3 log segments. Stats: ops{read=14 overwritten=0 applied=12 ignored=12} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=2. Pending: 2 replicates
I20260812 08:03:51.945878 22715 tablet_bootstrap.cc:492] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: Bootstrap replayed 3/3 log segments. Stats: ops{read=36 overwritten=0 applied=36 ignored=12} inserts{seen=4 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=2. Pending: 0 replicates
I20260812 08:03:51.946236 22715 tablet_bootstrap.cc:492] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: Bootstrap complete.
I20260812 08:03:51.946775 22715 ts_tablet_manager.cc:1403] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: Time spent bootstrapping tablet: real 0.028s	user 0.018s	sys 0.009s
I20260812 08:03:51.947363 22715 raft_consensus.cc:359] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } }
I20260812 08:03:51.947469 22715 raft_consensus.cc:740] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 2ed16f636eb243e2895d251a6f3d0681, State: Initialized, Role: LEARNER
I20260812 08:03:51.947587 22715 consensus_queue.cc:260] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 59, Last appended: 2.59, Last appended by leader: 59, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: NON_VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: true } }
I20260812 08:03:51.947914 22715 ts_tablet_manager.cc:1434] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: Time spent starting tablet: real 0.001s	user 0.002s	sys 0.001s
I20260812 08:03:51.948727 22160 tablet_copy_service.cc:342] P 397b07826d51403f8bc1085aea72b174: Request end of tablet copy session 2ed16f636eb243e2895d251a6f3d0681-4f9ddcbef96949abbbebf26348255dc6 received from {username='slave'} at 127.0.186.67:52511
I20260812 08:03:51.948874 22160 tablet_copy_service.cc:434] P 397b07826d51403f8bc1085aea72b174: ending tablet copy session 2ed16f636eb243e2895d251a6f3d0681-4f9ddcbef96949abbbebf26348255dc6 on tablet 4f9ddcbef96949abbbebf26348255dc6 with peer 2ed16f636eb243e2895d251a6f3d0681
I20260812 08:03:52.043169 22697 maintenance_manager.cc:419] P 2ed16f636eb243e2895d251a6f3d0681: Scheduling UndoDeltaBlockGCOp(4f9ddcbef96949abbbebf26348255dc6): 506 bytes on disk
I20260812 08:03:52.044155 22552 maintenance_manager.cc:643] P 2ed16f636eb243e2895d251a6f3d0681: UndoDeltaBlockGCOp(4f9ddcbef96949abbbebf26348255dc6) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":64,"lbm_reads_lt_1ms":4}
I20260812 08:03:52.509354 22638 raft_consensus.cc:1217] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.58->[2.59-2.59]   Dedup: 2.59->[]
I20260812 08:03:52.576783 22203 maintenance_manager.cc:419] P 397b07826d51403f8bc1085aea72b174: Scheduling LogGCOp(4f9ddcbef96949abbbebf26348255dc6): free 920418 bytes of WAL
I20260812 08:03:52.577083 22064 log_reader.cc:385] T 4f9ddcbef96949abbbebf26348255dc6: removed 1 log segments from log reader
I20260812 08:03:52.577138 22064 log.cc:1079] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/wal/wals/4f9ddcbef96949abbbebf26348255dc6/wal-000000004 (ops 24-30)
I20260812 08:03:52.577486 22064 maintenance_manager.cc:643] P 397b07826d51403f8bc1085aea72b174: LogGCOp(4f9ddcbef96949abbbebf26348255dc6) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {}
I20260812 08:03:52.928150 22730 raft_consensus.cc:1064] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174: attempting to promote NON_VOTER 2ed16f636eb243e2895d251a6f3d0681 to VOTER
I20260812 08:03:52.928714 22730 consensus_queue.cc:237] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 59, Committed index: 59, Last appended: 2.59, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: false } }
I20260812 08:03:52.930267 22638 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 2 LEARNER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 2 index: 59. Preceding OpId from leader: term: 2 index: 60. (index mismatch)
I20260812 08:03:52.930325 21977 raft_consensus.cc:1275] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Refusing update from remote peer 397b07826d51403f8bc1085aea72b174: Log matching property violated. Preceding OpId in replica: term: 2 index: 59. Preceding OpId from leader: term: 2 index: 60. (index mismatch)
I20260812 08:03:52.930812 22729 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 60, Last known committed idx: 59, Time since last communication: 0.000s
I20260812 08:03:52.931001 22730 consensus_queue.cc:1048] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 60, Last known committed idx: 59, Time since last communication: 0.000s
I20260812 08:03:52.932758 22729 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 [term 2 LEADER]: Committing config change with OpId 2.60: config changed from index 59 to 60, 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) changed from NON_VOTER to VOTER. New config: { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: false } } }
I20260812 08:03:52.933264 21977 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 4c43ad693825459f82d4ba3704c3fdb0 [term 2 FOLLOWER]: Committing config change with OpId 2.60: config changed from index 59 to 60, 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) changed from NON_VOTER to VOTER. New config: { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: false } } }
I20260812 08:03:52.935683 21796 catalog_manager.cc:5719] T 4f9ddcbef96949abbbebf26348255dc6 P 397b07826d51403f8bc1085aea72b174 reported cstate change: config changed from index 59 to 60, 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "397b07826d51403f8bc1085aea72b174" committed_config { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 08:03:52.936949 22638 raft_consensus.cc:2955] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681 [term 2 FOLLOWER]: Committing config change with OpId 2.60: config changed from index 59 to 60, 2ed16f636eb243e2895d251a6f3d0681 (127.0.186.67) changed from NON_VOTER to VOTER. New config: { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "4c43ad693825459f82d4ba3704c3fdb0" member_type: VOTER last_known_addr { host: "127.0.186.65" port: 34877 } } peers { permanent_uuid: "397b07826d51403f8bc1085aea72b174" member_type: VOTER last_known_addr { host: "127.0.186.66" port: 33729 } } peers { permanent_uuid: "2ed16f636eb243e2895d251a6f3d0681" member_type: VOTER last_known_addr { host: "127.0.186.67" port: 46531 } attrs { promote: false } } }
I20260812 08:03:52.948688 22697 maintenance_manager.cc:419] P 2ed16f636eb243e2895d251a6f3d0681: Scheduling LogGCOp(4f9ddcbef96949abbbebf26348255dc6): free 920350 bytes of WAL
I20260812 08:03:52.948983 22552 log_reader.cc:385] T 4f9ddcbef96949abbbebf26348255dc6: removed 1 log segments from log reader
I20260812 08:03:52.949064 22552 log.cc:1079] T 4f9ddcbef96949abbbebf26348255dc6 P 2ed16f636eb243e2895d251a6f3d0681: Deleting log segment in path: /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/wal/wals/4f9ddcbef96949abbbebf26348255dc6/wal-000000001 (ops 24-30)
I20260812 08:03:52.949467 22552 maintenance_manager.cc:643] P 2ed16f636eb243e2895d251a6f3d0681: LogGCOp(4f9ddcbef96949abbbebf26348255dc6) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 08:03:53.000223 22115 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260812 08:03:53.000564 21951 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:03:53.001067 22610 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |       Address       | Status
----------------------------------+---------------------+---------
 408a5cecd2c242388a8cac956732949d | 127.0.186.126:32839 | HEALTHY

Unusual flags for Master:
                   Flag                   |                                                                                               Value                                                                                                |         Tags          |         Master
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
 ipki_ca_key_size                         | 768                                                                                                                                                                                                | experimental          | all 1 server(s) checked
 ipki_server_key_size                     | 768                                                                                                                                                                                                | experimental          | all 1 server(s) checked
 never_fsync                              | true                                                                                                                                                                                               | unsafe,advanced       | all 1 server(s) checked
 openssl_security_level_override          | 0                                                                                                                                                                                                  | unsafe,hidden         | all 1 server(s) checked
 raft_prepare_replacement_before_eviction | true                                                                                                                                                                                               | experimental,advanced | all 1 server(s) checked
 rpc_reuseport                            | true                                                                                                                                                                                               | experimental          | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports         | true                                                                                                                                                                                               | unsafe                | all 1 server(s) checked
 server_dump_info_format                  | pb                                                                                                                                                                                                 | hidden                | all 1 server(s) checked
 server_dump_info_path                    | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden                | all 1 server(s) checked
 tsk_num_rsa_bits                         | 512                                                                                                                                                                                                | experimental          | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |       Value        |         Master
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.0.186.84:42325 | all 1 server(s) checked
 time_source         | builtin            | all 1 server(s) checked

Tablet Server Summary
               UUID               |      Address       | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
 2ed16f636eb243e2895d251a6f3d0681 | 127.0.186.67:46531 | HEALTHY | <none>   |       0        |       0
 397b07826d51403f8bc1085aea72b174 | 127.0.186.66:33729 | HEALTHY | <none>   |       1        |       0
 4c43ad693825459f82d4ba3704c3fdb0 | 127.0.186.65:34877 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
                   Flag                   |                                                                                             Value                                                                                              |             Tags              |      Tablet Server
------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------+-------------------------
 ipki_server_key_size                     | 768                                                                                                                                                                                            | experimental                  | all 3 server(s) checked
 local_ip_for_outbound_sockets            | 127.0.186.65                                                                                                                                                                                   | experimental                  | 127.0.186.65:34877
 local_ip_for_outbound_sockets            | 127.0.186.66                                                                                                                                                                                   | experimental                  | 127.0.186.66:33729
 local_ip_for_outbound_sockets            | 127.0.186.67                                                                                                                                                                                   | experimental                  | 127.0.186.67:46531
 log_compression_codec                    | no_compression                                                                                                                                                                                 | experimental                  | all 3 server(s) checked
 log_max_segments_to_retain               | 3                                                                                                                                                                                              | experimental,runtime,advanced | all 3 server(s) checked
 log_target_replay_size_mb                | 1                                                                                                                                                                                              | experimental                  | all 3 server(s) checked
 maintenance_manager_polling_interval_ms  | 100                                                                                                                                                                                            | hidden                        | all 3 server(s) checked
 max_cell_size_bytes                      | 1000000                                                                                                                                                                                        | unsafe                        | all 3 server(s) checked
 never_fsync                              | true                                                                                                                                                                                           | unsafe,advanced               | all 3 server(s) checked
 openssl_security_level_override          | 0                                                                                                                                                                                              | unsafe,hidden                 | all 3 server(s) checked
 raft_prepare_replacement_before_eviction | true                                                                                                                                                                                           | experimental,advanced         | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports         | true                                                                                                                                                                                           | unsafe                        | all 3 server(s) checked
 server_dump_info_format                  | pb                                                                                                                                                                                             | hidden                        | all 3 server(s) checked
 server_dump_info_path                    | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden                        | 127.0.186.65:34877
 server_dump_info_path                    | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden                        | 127.0.186.66:33729
 server_dump_info_path                    | /tmp/dist-test-taskamFhYX/test-tmp/raft_consensus_nonvoter-itest.0.ReplicaBehindWalGcThresholdITest.ReplicaReplacement_5.1786521609225256-745-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden                        | 127.0.186.67:46531

Flags of checked categories for Tablet Server:
        Flag         |       Value        |      Tablet Server
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.0.186.84:42325 | all 3 server(s) checked
 time_source         | builtin            | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
   Name    | RF | Status  | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-----------+----+---------+---------------+---------+------------+------------------+-------------
 TestTable | 3  | HEALTHY | 1             | 1       | 0          | 0                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

OK
I20260812 08:03:53.025424   745 log_verifier.cc:126] Checking tablet 4f9ddcbef96949abbbebf26348255dc6
I20260812 08:03:53.035692   745 log_verifier.cc:177] Verified matching terms for 33 ops in tablet 4f9ddcbef96949abbbebf26348255dc6
I20260812 08:03:53.036602   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 21872
I20260812 08:03:53.050487   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 22041
I20260812 08:03:53.063918   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 22528
I20260812 08:03:53.072890   745 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskamFhYX/build/debug/bin/kudu with pid 21760
2026-08-12T08:03:53Z chronyd exiting
[       OK ] ReplicaBehindWalGcThresholdITest.ReplicaReplacement/5 (22766 ms)
[----------] 6 tests from ReplicaBehindWalGcThresholdITest (91219 ms total)

[----------] Global test environment tear-down
[==========] 23 tests from 3 test suites ran. (223861 ms total)
[  PASSED  ] 23 tests.
