Note: This is test shard 6 of 6.
[==========] Running 7 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from RaftConsensusITest
[ RUN      ] RaftConsensusITest.InsertUniqueKeysWithCrashyNodes
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260812 08:02:57.614426 18601 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:02:57.614564 18601 ts_itest-base.cc:116] --------------
I20260812 08:02:57.614578 18601 ts_itest-base.cc:117] 7 tablet servers
I20260812 08:02:57.614590 18601 ts_itest-base.cc:118] 7 replicas per TS
I20260812 08:02:57.614600 18601 ts_itest-base.cc:119] --------------
2026-08-12T08:02:57Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:02:57Z Disabled control of system clock
I20260812 08:02:57.627185 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.42.126:35213
--webserver_interface=127.18.42.126
--webserver_port=0
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.42.126:35213 with env {}
W20260812 08:02:57.751022 18622 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:57.751343 18622 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:57.751511 18622 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:57.755869 18622 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:02:57.755944 18622 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:57.755966 18622 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:02:57.755985 18622 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:02:57.760404 18622 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.42.126:35213
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.42.126:35213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/master-0/data/info.pb
--webserver_interface=127.18.42.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.18622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:57.761834 18622 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:57.763316 18622 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:57.771837 18634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:57.772056 18629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:57.772322 18631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:57.772578 18622 server_base.cc:1061] running on GCE node
I20260812 08:02:57.773274 18622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:57.774545 18622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:57.775749 18622 hybrid_clock.cc:648] HybridClock initialized: now 1786521777775751 us; error 63 us; skew 500 ppm
I20260812 08:02:57.778204 18622 webserver.cc:533] Webserver started at http://127.18.42.126:39723/ using document root <none> and password file <none>
I20260812 08:02:57.778955 18622 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:57.779054 18622 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:57.779320 18622 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:57.781064 18622 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/master-0/data/instance:
uuid: "af9c6f91894742328cd99747cf1207fc"
format_stamp: "Formatted at 2026-08-12 08:02:57 on dist-test-slave-zr1t"
I20260812 08:02:57.781680 18622 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/master-0/wal/instance:
uuid: "af9c6f91894742328cd99747cf1207fc"
format_stamp: "Formatted at 2026-08-12 08:02:57 on dist-test-slave-zr1t"
I20260812 08:02:57.785626 18622 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260812 08:02:57.788457 18645 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:57.789671 18622 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:02:57.789829 18622 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/master-0/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/master-0/wal
uuid: "af9c6f91894742328cd99747cf1207fc"
format_stamp: "Formatted at 2026-08-12 08:02:57 on dist-test-slave-zr1t"
I20260812 08:02:57.789978 18622 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:57.806572 18622 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:57.807585 18622 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:57.807833 18622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:57.826782 18622 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.126:35213
I20260812 08:02:57.826894 18748 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.126:35213 every 8 connection(s)
I20260812 08:02:57.828171 18622 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/master-0/data/info.pb
I20260812 08:02:57.829911 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 18622
I20260812 08:02:57.830108 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/master-0/wal/instance
I20260812 08:02:57.833280 18751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:57.842507 18751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af9c6f91894742328cd99747cf1207fc: Bootstrap starting.
I20260812 08:02:57.845791 18751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af9c6f91894742328cd99747cf1207fc: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:57.847213 18751 log.cc:826] T 00000000000000000000000000000000 P af9c6f91894742328cd99747cf1207fc: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:57.849649 18751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af9c6f91894742328cd99747cf1207fc: No bootstrap required, opened a new log
I20260812 08:02:57.854163 18751 raft_consensus.cc:359] T 00000000000000000000000000000000 P af9c6f91894742328cd99747cf1207fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af9c6f91894742328cd99747cf1207fc" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 35213 } }
I20260812 08:02:57.854427 18751 raft_consensus.cc:385] T 00000000000000000000000000000000 P af9c6f91894742328cd99747cf1207fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:57.854488 18751 raft_consensus.cc:740] T 00000000000000000000000000000000 P af9c6f91894742328cd99747cf1207fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af9c6f91894742328cd99747cf1207fc, State: Initialized, Role: FOLLOWER
I20260812 08:02:57.855252 18751 consensus_queue.cc:260] T 00000000000000000000000000000000 P af9c6f91894742328cd99747cf1207fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af9c6f91894742328cd99747cf1207fc" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 35213 } }
I20260812 08:02:57.855486 18751 raft_consensus.cc:399] T 00000000000000000000000000000000 P af9c6f91894742328cd99747cf1207fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:57.855619 18751 raft_consensus.cc:493] T 00000000000000000000000000000000 P af9c6f91894742328cd99747cf1207fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:57.855799 18751 raft_consensus.cc:3060] T 00000000000000000000000000000000 P af9c6f91894742328cd99747cf1207fc [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:57.857309 18751 raft_consensus.cc:515] T 00000000000000000000000000000000 P af9c6f91894742328cd99747cf1207fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af9c6f91894742328cd99747cf1207fc" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 35213 } }
I20260812 08:02:57.857887 18751 leader_election.cc:304] T 00000000000000000000000000000000 P af9c6f91894742328cd99747cf1207fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af9c6f91894742328cd99747cf1207fc; no voters: 
I20260812 08:02:57.858322 18751 leader_election.cc:290] T 00000000000000000000000000000000 P af9c6f91894742328cd99747cf1207fc [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:02:57.858589 18758 raft_consensus.cc:2804] T 00000000000000000000000000000000 P af9c6f91894742328cd99747cf1207fc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:57.859040 18758 raft_consensus.cc:697] T 00000000000000000000000000000000 P af9c6f91894742328cd99747cf1207fc [term 1 LEADER]: Becoming Leader. State: Replica: af9c6f91894742328cd99747cf1207fc, State: Running, Role: LEADER
I20260812 08:02:57.859378 18758 consensus_queue.cc:237] T 00000000000000000000000000000000 P af9c6f91894742328cd99747cf1207fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af9c6f91894742328cd99747cf1207fc" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 35213 } }
I20260812 08:02:57.859689 18751 sys_catalog.cc:565] T 00000000000000000000000000000000 P af9c6f91894742328cd99747cf1207fc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:02:57.862282 18760 sys_catalog.cc:455] T 00000000000000000000000000000000 P af9c6f91894742328cd99747cf1207fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader af9c6f91894742328cd99747cf1207fc. Latest consensus state: current_term: 1 leader_uuid: "af9c6f91894742328cd99747cf1207fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af9c6f91894742328cd99747cf1207fc" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 35213 } } }
I20260812 08:02:57.862507 18760 sys_catalog.cc:458] T 00000000000000000000000000000000 P af9c6f91894742328cd99747cf1207fc [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:57.863492 18770 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:02:57.863771 18759 sys_catalog.cc:455] T 00000000000000000000000000000000 P af9c6f91894742328cd99747cf1207fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af9c6f91894742328cd99747cf1207fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af9c6f91894742328cd99747cf1207fc" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 35213 } } }
I20260812 08:02:57.863879 18759 sys_catalog.cc:458] T 00000000000000000000000000000000 P af9c6f91894742328cd99747cf1207fc [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:57.867806 18770 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:02:57.875775 18770 catalog_manager.cc:1383] Generated new cluster ID: bb91aa81772f44df985203db03deff01
I20260812 08:02:57.875871 18770 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:02:57.885056 18770 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:02:57.886255 18770 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:02:57.896198 18770 catalog_manager.cc:6092] T 00000000000000000000000000000000 P af9c6f91894742328cd99747cf1207fc: Generated new TSK 0
I20260812 08:02:57.896934 18770 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:02:57.908172 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.65:0
--local_ip_for_outbound_sockets=127.18.42.65
--webserver_interface=127.18.42.65
--webserver_port=0
--tserver_master_addrs=127.18.42.126:35213
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.05
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
W20260812 08:02:58.028970 18784 flags.cc:432] Enabled unsafe flag: --fault_crash_on_leader_request_fraction=0.050000000000000003
W20260812 08:02:58.029197 18784 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:02:58.029217 18784 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:02:58.029232 18784 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:02:58.029315 18784 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:58.029366 18784 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:58.029567 18784 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:58.033083 18784 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:58.033193 18784 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.65
I20260812 08:02:58.037335 18784 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.050000000000000003
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data/info.pb
--webserver_interface=127.18.42.65
--webserver_port=0
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.18784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:58.038604 18784 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:58.039816 18784 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:02:58.042838 18784 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:02:58.047113 18800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:58.047170 18797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:58.047130 18798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:58.048059 18784 server_base.cc:1061] running on GCE node
I20260812 08:02:58.048455 18784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:58.049029 18784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:58.050210 18784 hybrid_clock.cc:648] HybridClock initialized: now 1786521778050164 us; error 58 us; skew 500 ppm
I20260812 08:02:58.052511 18784 webserver.cc:533] Webserver started at http://127.18.42.65:38137/ using document root <none> and password file <none>
I20260812 08:02:58.053153 18784 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:58.053216 18784 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:58.053435 18784 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:58.055378 18784 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data/instance:
uuid: "a5813459b1d94e089ffbe751f4310de7"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:02:58.055980 18784 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/wal/instance:
uuid: "a5813459b1d94e089ffbe751f4310de7"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:02:58.059964 18784 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:02:58.062752 18810 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:58.064083 18784 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:02:58.064237 18784 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/wal
uuid: "a5813459b1d94e089ffbe751f4310de7"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:02:58.064374 18784 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:58.087953 18784 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:58.088773 18784 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:58.088999 18784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:58.089768 18784 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:58.090924 18784 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:58.090976 18784 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:58.091017 18784 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:58.091077 18784 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:58.107434 18784 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.65:45661
I20260812 08:02:58.107451 19013 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.65:45661 every 8 connection(s)
I20260812 08:02:58.108564 18784 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data/info.pb
I20260812 08:02:58.118446 19014 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:02:58.118759 19014 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:58.118824 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 18784
I20260812 08:02:58.118952 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/wal/instance
I20260812 08:02:58.119453 19014 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:02:58.121372 18673 ts_manager.cc:194] Registered new tserver with Master: a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661)
I20260812 08:02:58.121655 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.66:0
--local_ip_for_outbound_sockets=127.18.42.66
--webserver_interface=127.18.42.66
--webserver_port=0
--tserver_master_addrs=127.18.42.126:35213
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.05
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
I20260812 08:02:58.122902 18673 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.65:39275
W20260812 08:02:58.236521 19017 flags.cc:432] Enabled unsafe flag: --fault_crash_on_leader_request_fraction=0.050000000000000003
W20260812 08:02:58.236766 19017 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:02:58.236784 19017 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:02:58.236802 19017 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:02:58.236841 19017 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:58.236860 19017 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:58.236902 19017 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:58.241026 19017 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:58.241158 19017 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.66
I20260812 08:02:58.245296 19017 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.050000000000000003
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/data/info.pb
--webserver_interface=127.18.42.66
--webserver_port=0
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.19017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:58.246706 19017 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:58.247902 19017 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:02:58.250869 19017 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:02:58.254877 19032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:58.254981 19028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:58.254884 19027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:58.255259 19017 server_base.cc:1061] running on GCE node
I20260812 08:02:58.255743 19017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:58.256393 19017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:58.257615 19017 hybrid_clock.cc:648] HybridClock initialized: now 1786521778257580 us; error 40 us; skew 500 ppm
I20260812 08:02:58.259634 19017 webserver.cc:533] Webserver started at http://127.18.42.66:42027/ using document root <none> and password file <none>
I20260812 08:02:58.260258 19017 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:58.260314 19017 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:58.260558 19017 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:58.262326 19017 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/data/instance:
uuid: "5b4c3ad48b794617893c88a077be1b8d"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:02:58.262879 19017 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/wal/instance:
uuid: "5b4c3ad48b794617893c88a077be1b8d"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:02:58.266433 19017 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.001s
I20260812 08:02:58.268752 19044 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:58.269881 19017 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 08:02:58.270011 19017 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/wal
uuid: "5b4c3ad48b794617893c88a077be1b8d"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:02:58.270133 19017 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:58.291117 19017 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:58.291883 19017 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:58.292099 19017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:58.292829 19017 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:58.294184 19017 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:58.294260 19017 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:58.294341 19017 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:58.294387 19017 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:58.311518 19017 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.66:32855
I20260812 08:02:58.311538 19247 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.66:32855 every 8 connection(s)
I20260812 08:02:58.312763 19017 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/data/info.pb
I20260812 08:02:58.321236 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 19017
I20260812 08:02:58.321369 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/wal/instance
I20260812 08:02:58.322192 19249 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:02:58.322489 19249 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:58.323119 19249 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:02:58.324049 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.67:0
--local_ip_for_outbound_sockets=127.18.42.67
--webserver_interface=127.18.42.67
--webserver_port=0
--tserver_master_addrs=127.18.42.126:35213
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.05
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
I20260812 08:02:58.324307 18673 ts_manager.cc:194] Registered new tserver with Master: 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855)
I20260812 08:02:58.325387 18673 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.66:33261
W20260812 08:02:58.444988 19255 flags.cc:432] Enabled unsafe flag: --fault_crash_on_leader_request_fraction=0.050000000000000003
W20260812 08:02:58.445226 19255 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:02:58.445245 19255 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:02:58.445261 19255 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:02:58.445307 19255 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:58.445326 19255 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:58.445370 19255 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:58.448894 19255 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:58.449024 19255 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.67
I20260812 08:02:58.453063 19255 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.050000000000000003
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data/info.pb
--webserver_interface=127.18.42.67
--webserver_port=0
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.19255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:58.454432 19255 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:58.455538 19255 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:02:58.458403 19255 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:02:58.462342 19261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:58.462342 19271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:58.462430 19260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:58.462633 19255 server_base.cc:1061] running on GCE node
I20260812 08:02:58.463014 19255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:58.463578 19255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:58.464735 19255 hybrid_clock.cc:648] HybridClock initialized: now 1786521778464722 us; error 25 us; skew 500 ppm
I20260812 08:02:58.466585 19255 webserver.cc:533] Webserver started at http://127.18.42.67:39753/ using document root <none> and password file <none>
I20260812 08:02:58.467186 19255 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:58.467243 19255 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:58.467458 19255 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:58.469020 19255 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data/instance:
uuid: "294e0e1970eb4e58bed426de35babb6b"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:02:58.469620 19255 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal/instance:
uuid: "294e0e1970eb4e58bed426de35babb6b"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:02:58.473160 19255 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.000s
I20260812 08:02:58.475591 19279 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:58.476603 19255 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:58.476716 19255 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal
uuid: "294e0e1970eb4e58bed426de35babb6b"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:02:58.476816 19255 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:58.504839 19255 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:58.505788 19255 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:58.506024 19255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:58.506763 19255 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:58.507949 19255 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:58.508035 19255 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:58.508111 19255 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:58.508164 19255 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:58.523159 19255 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.67:40205
I20260812 08:02:58.523185 19470 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.67:40205 every 8 connection(s)
I20260812 08:02:58.524296 19255 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data/info.pb
I20260812 08:02:58.533970 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 19255
I20260812 08:02:58.534122 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal/instance
I20260812 08:02:58.534221 19471 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:02:58.534511 19471 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:58.535218 19471 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:02:58.536484 18673 ts_manager.cc:194] Registered new tserver with Master: 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205)
I20260812 08:02:58.536768 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.68:0
--local_ip_for_outbound_sockets=127.18.42.68
--webserver_interface=127.18.42.68
--webserver_port=0
--tserver_master_addrs=127.18.42.126:35213
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.05
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
I20260812 08:02:58.537273 18673 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.67:33839
W20260812 08:02:58.649142 19474 flags.cc:432] Enabled unsafe flag: --fault_crash_on_leader_request_fraction=0.050000000000000003
W20260812 08:02:58.649366 19474 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:02:58.649384 19474 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:02:58.649399 19474 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:02:58.649438 19474 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:58.649457 19474 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:58.649507 19474 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:58.652905 19474 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:58.653019 19474 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.68
I20260812 08:02:58.657076 19474 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.050000000000000003
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/data/info.pb
--webserver_interface=127.18.42.68
--webserver_port=0
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.19474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.68

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:58.658211 19474 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:58.659281 19474 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:02:58.662227 19474 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:02:58.666121 19485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:58.666131 19486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:58.666138 19489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:58.666903 19474 server_base.cc:1061] running on GCE node
I20260812 08:02:58.667315 19474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:58.667871 19474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:58.669065 19474 hybrid_clock.cc:648] HybridClock initialized: now 1786521778669055 us; error 28 us; skew 500 ppm
I20260812 08:02:58.671092 19474 webserver.cc:533] Webserver started at http://127.18.42.68:36373/ using document root <none> and password file <none>
I20260812 08:02:58.671674 19474 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:58.671758 19474 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:58.671989 19474 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:58.673629 19474 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/data/instance:
uuid: "b7def2bd3a74420d9deec19b32f541ec"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:02:58.674151 19474 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/wal/instance:
uuid: "b7def2bd3a74420d9deec19b32f541ec"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:02:58.677958 19474 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.003s
I20260812 08:02:58.680404 19497 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:58.681614 19474 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:02:58.681742 19474 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/wal
uuid: "b7def2bd3a74420d9deec19b32f541ec"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:02:58.681849 19474 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:58.707207 19474 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:58.708020 19474 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:58.708257 19474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:58.709031 19474 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:58.710284 19474 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:58.710369 19474 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:58.710449 19474 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:58.710507 19474 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:58.728188 19474 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.68:36499
I20260812 08:02:58.728209 19700 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.68:36499 every 8 connection(s)
I20260812 08:02:58.729324 19474 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/data/info.pb
I20260812 08:02:58.735493 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 19474
I20260812 08:02:58.735647 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/wal/instance
I20260812 08:02:58.738178 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.69:0
--local_ip_for_outbound_sockets=127.18.42.69
--webserver_interface=127.18.42.69
--webserver_port=0
--tserver_master_addrs=127.18.42.126:35213
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.05
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
I20260812 08:02:58.739358 19702 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:02:58.739753 19702 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:58.740648 19702 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:02:58.742327 18673 ts_manager.cc:194] Registered new tserver with Master: b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499)
I20260812 08:02:58.743295 18673 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.68:38777
W20260812 08:02:58.865377 19711 flags.cc:432] Enabled unsafe flag: --fault_crash_on_leader_request_fraction=0.050000000000000003
W20260812 08:02:58.865658 19711 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:02:58.865681 19711 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:02:58.865696 19711 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:02:58.865736 19711 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:58.865760 19711 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:58.865803 19711 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:58.869751 19711 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:58.869874 19711 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.69
I20260812 08:02:58.873847 19711 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.050000000000000003
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.69:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data/info.pb
--webserver_interface=127.18.42.69
--webserver_port=0
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.19711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.69

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:58.875340 19711 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:58.877205 19711 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:02:58.881336 19711 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:02:58.885586 19720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:58.885557 19728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:58.885509 19719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:58.886713 19711 server_base.cc:1061] running on GCE node
I20260812 08:02:58.887197 19711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:58.887831 19711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:58.889038 19711 hybrid_clock.cc:648] HybridClock initialized: now 1786521778889012 us; error 40 us; skew 500 ppm
I20260812 08:02:58.891047 19711 webserver.cc:533] Webserver started at http://127.18.42.69:40235/ using document root <none> and password file <none>
I20260812 08:02:58.891652 19711 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:58.891737 19711 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:58.891963 19711 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:58.893666 19711 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data/instance:
uuid: "285e078e6cfb46ae9dbe57c20be666d5"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:02:58.894222 19711 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal/instance:
uuid: "285e078e6cfb46ae9dbe57c20be666d5"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:02:58.898025 19711 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260812 08:02:58.900758 19736 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:58.902034 19711 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:02:58.902207 19711 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
uuid: "285e078e6cfb46ae9dbe57c20be666d5"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:02:58.902351 19711 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:58.921123 19711 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:58.921988 19711 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:58.922206 19711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:58.922996 19711 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:58.924295 19711 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:58.924384 19711 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:58.924489 19711 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:58.924543 19711 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:58.940604 19711 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.69:34819
I20260812 08:02:58.940629 19927 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.69:34819 every 8 connection(s)
I20260812 08:02:58.941808 19711 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data/info.pb
I20260812 08:02:58.950316 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 19711
I20260812 08:02:58.950505 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal/instance
I20260812 08:02:58.951722 19928 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:02:58.952054 19928 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:58.952649 19928 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:02:58.952730 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.70:0
--local_ip_for_outbound_sockets=127.18.42.70
--webserver_interface=127.18.42.70
--webserver_port=0
--tserver_master_addrs=127.18.42.126:35213
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.05
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
I20260812 08:02:58.954121 18673 ts_manager.cc:194] Registered new tserver with Master: 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819)
I20260812 08:02:58.955184 18673 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.69:54897
W20260812 08:02:59.080370 19932 flags.cc:432] Enabled unsafe flag: --fault_crash_on_leader_request_fraction=0.050000000000000003
W20260812 08:02:59.080615 19932 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:02:59.080636 19932 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:02:59.080653 19932 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:02:59.080696 19932 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:59.080725 19932 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:59.080770 19932 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:59.084934 19932 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:59.085057 19932 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.70
I20260812 08:02:59.089694 19932 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.050000000000000003
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.70:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data/info.pb
--webserver_interface=127.18.42.70
--webserver_port=0
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.19932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.70

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:59.090864 19932 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:59.092078 19932 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:02:59.095157 19932 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:02:59.099082 19941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:59.099262 19943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:59.099373 19947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:59.100304 19932 server_base.cc:1061] running on GCE node
I20260812 08:02:59.100759 19932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:59.101334 19932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:59.102531 19932 hybrid_clock.cc:648] HybridClock initialized: now 1786521779102510 us; error 45 us; skew 500 ppm
I20260812 08:02:59.105105 19932 webserver.cc:533] Webserver started at http://127.18.42.70:40739/ using document root <none> and password file <none>
I20260812 08:02:59.105918 19932 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:59.106012 19932 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:59.106284 19932 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:59.114503 19932 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data/instance:
uuid: "38681904c36540a4876519ec5ffac6b6"
format_stamp: "Formatted at 2026-08-12 08:02:59 on dist-test-slave-zr1t"
I20260812 08:02:59.115196 19932 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/wal/instance:
uuid: "38681904c36540a4876519ec5ffac6b6"
format_stamp: "Formatted at 2026-08-12 08:02:59 on dist-test-slave-zr1t"
I20260812 08:02:59.123052 19932 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.005s	sys 0.003s
I20260812 08:02:59.125370 19014 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
I20260812 08:02:59.126302 19954 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:59.128160 19932 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:02:59.128306 19932 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/wal
uuid: "38681904c36540a4876519ec5ffac6b6"
format_stamp: "Formatted at 2026-08-12 08:02:59 on dist-test-slave-zr1t"
I20260812 08:02:59.128455 19932 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:59.163431 19932 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:59.164180 19932 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:59.164348 19932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:59.164901 19932 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:59.166042 19932 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:59.166093 19932 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:59.166133 19932 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:59.166148 19932 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:59.181964 19932 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.70:39785
I20260812 08:02:59.181983 20147 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.70:39785 every 8 connection(s)
I20260812 08:02:59.183005 19932 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data/info.pb
I20260812 08:02:59.185815 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 19932
I20260812 08:02:59.185928 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/wal/instance
I20260812 08:02:59.188362 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.71:0
--local_ip_for_outbound_sockets=127.18.42.71
--webserver_interface=127.18.42.71
--webserver_port=0
--tserver_master_addrs=127.18.42.126:35213
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.05
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
I20260812 08:02:59.192389 20148 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:02:59.192709 20148 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:59.193390 20148 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:02:59.194721 18673 ts_manager.cc:194] Registered new tserver with Master: 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785)
I20260812 08:02:59.195341 18673 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.70:35707
W20260812 08:02:59.302397 20151 flags.cc:432] Enabled unsafe flag: --fault_crash_on_leader_request_fraction=0.050000000000000003
W20260812 08:02:59.302614 20151 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:02:59.302634 20151 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:02:59.302649 20151 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:02:59.302711 20151 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:59.302760 20151 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:59.302841 20151 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:59.306636 20151 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:59.306767 20151 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.71
I20260812 08:02:59.311847 20151 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.050000000000000003
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.71:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data/info.pb
--webserver_interface=127.18.42.71
--webserver_port=0
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.20151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.71

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:59.313216 20151 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:59.314575 20151 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:02:59.317766 20151 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:02:59.322155 20156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:59.322199 20159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:59.322168 20157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:59.322590 20151 server_base.cc:1061] running on GCE node
I20260812 08:02:59.322995 20151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:59.323598 20151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:59.324783 20151 hybrid_clock.cc:648] HybridClock initialized: now 1786521779324750 us; error 47 us; skew 500 ppm
I20260812 08:02:59.326737 20151 webserver.cc:533] Webserver started at http://127.18.42.71:37531/ using document root <none> and password file <none>
I20260812 08:02:59.327386 20151 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:59.327484 20151 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:59.327584 19249 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
I20260812 08:02:59.327762 20151 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:59.329699 20151 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data/instance:
uuid: "0e10b84812514a76ab7225dfbf9329bf"
format_stamp: "Formatted at 2026-08-12 08:02:59 on dist-test-slave-zr1t"
I20260812 08:02:59.330338 20151 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/wal/instance:
uuid: "0e10b84812514a76ab7225dfbf9329bf"
format_stamp: "Formatted at 2026-08-12 08:02:59 on dist-test-slave-zr1t"
I20260812 08:02:59.334137 20151 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260812 08:02:59.336561 20168 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:59.337714 20151 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:02:59.337838 20151 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/wal
uuid: "0e10b84812514a76ab7225dfbf9329bf"
format_stamp: "Formatted at 2026-08-12 08:02:59 on dist-test-slave-zr1t"
I20260812 08:02:59.337941 20151 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:59.364773 20151 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:59.365573 20151 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:59.365795 20151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:59.366546 20151 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:59.367767 20151 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:59.367817 20151 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:59.367883 20151 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:59.367923 20151 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:59.383064 20151 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.71:35647
I20260812 08:02:59.383195 20360 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.71:35647 every 8 connection(s)
I20260812 08:02:59.384218 20151 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data/info.pb
I20260812 08:02:59.387679 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 20151
I20260812 08:02:59.387801 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/wal/instance
I20260812 08:02:59.393911 20361 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:02:59.394200 20361 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:59.394824 20361 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:02:59.396049 18673 ts_manager.cc:194] Registered new tserver with Master: 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:02:59.396665 18673 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.71:38997
I20260812 08:02:59.403481 18601 external_mini_cluster.cc:955] 7 TS(s) registered with all masters
I20260812 08:02:59.421674 18673 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51794:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260812 08:02:59.422966 18673 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20260812 08:02:59.443281 18913 tablet_service.cc:1511] Processing CreateTablet for tablet 50b14eb091f7476d9d7682e503745c93 (DEFAULT_TABLE table=test-workload [id=f676baf4d90e47909c86ef51b3260926]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:59.447023 19126 tablet_service.cc:1511] Processing CreateTablet for tablet 50b14eb091f7476d9d7682e503745c93 (DEFAULT_TABLE table=test-workload [id=f676baf4d90e47909c86ef51b3260926]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:59.447712 20040 tablet_service.cc:1511] Processing CreateTablet for tablet 50b14eb091f7476d9d7682e503745c93 (DEFAULT_TABLE table=test-workload [id=f676baf4d90e47909c86ef51b3260926]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:59.448280 19126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50b14eb091f7476d9d7682e503745c93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:59.448688 20040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50b14eb091f7476d9d7682e503745c93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:59.452304 18913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50b14eb091f7476d9d7682e503745c93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:59.452446 19587 tablet_service.cc:1511] Processing CreateTablet for tablet 50b14eb091f7476d9d7682e503745c93 (DEFAULT_TABLE table=test-workload [id=f676baf4d90e47909c86ef51b3260926]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:59.452796 19352 tablet_service.cc:1511] Processing CreateTablet for tablet 50b14eb091f7476d9d7682e503745c93 (DEFAULT_TABLE table=test-workload [id=f676baf4d90e47909c86ef51b3260926]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:59.453383 19587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50b14eb091f7476d9d7682e503745c93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:59.453336 20245 tablet_service.cc:1511] Processing CreateTablet for tablet 50b14eb091f7476d9d7682e503745c93 (DEFAULT_TABLE table=test-workload [id=f676baf4d90e47909c86ef51b3260926]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:59.453733 19352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50b14eb091f7476d9d7682e503745c93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:59.454182 20245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50b14eb091f7476d9d7682e503745c93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:59.459895 19817 tablet_service.cc:1511] Processing CreateTablet for tablet 50b14eb091f7476d9d7682e503745c93 (DEFAULT_TABLE table=test-workload [id=f676baf4d90e47909c86ef51b3260926]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:59.461006 19817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50b14eb091f7476d9d7682e503745c93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:59.465364 20409 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Bootstrap starting.
I20260812 08:02:59.465332 20417 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Bootstrap starting.
I20260812 08:02:59.467064 20418 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Bootstrap starting.
I20260812 08:02:59.468062 20409 tablet_bootstrap.cc:654] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:59.468062 20417 tablet_bootstrap.cc:654] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:59.468997 20409 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:59.468997 20417 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:59.469084 20420 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Bootstrap starting.
I20260812 08:02:59.469350 20418 tablet_bootstrap.cc:654] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:59.469928 20416 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Bootstrap starting.
I20260812 08:02:59.470171 20418 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:59.470440 20411 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Bootstrap starting.
I20260812 08:02:59.470237 20415 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Bootstrap starting.
I20260812 08:02:59.471865 20420 tablet_bootstrap.cc:654] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:59.472018 20416 tablet_bootstrap.cc:654] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:59.472774 20420 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:59.472824 20409 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: No bootstrap required, opened a new log
W20260812 08:02:59.473034 20409 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 36ms of latency in SegmentAllocator::Sync()
I20260812 08:02:59.473223 20417 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: No bootstrap required, opened a new log
W20260812 08:02:59.473461 20417 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Injecting 39ms of latency in SegmentAllocator::Sync()
I20260812 08:02:59.474361 20416 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:59.474606 20420 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: No bootstrap required, opened a new log
I20260812 08:02:59.474606 20418 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: No bootstrap required, opened a new log
I20260812 08:02:59.474838 20418 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Time spent bootstrapping tablet: real 0.008s	user 0.004s	sys 0.000s
I20260812 08:02:59.474848 20420 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Time spent bootstrapping tablet: real 0.006s	user 0.004s	sys 0.000s
I20260812 08:02:59.474911 20415 tablet_bootstrap.cc:654] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:59.476302 20416 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: No bootstrap required, opened a new log
I20260812 08:02:59.476517 20416 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Time spent bootstrapping tablet: real 0.007s	user 0.000s	sys 0.005s
I20260812 08:02:59.476562 20411 tablet_bootstrap.cc:654] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:59.477324 20411 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:59.478348 20415 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:59.478354 20420 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:02:59.478364 20418 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:02:59.478596 20420 raft_consensus.cc:385] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:59.478596 20418 raft_consensus.cc:385] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:59.478653 20418 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7def2bd3a74420d9deec19b32f541ec, State: Initialized, Role: FOLLOWER
I20260812 08:02:59.478653 20420 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 285e078e6cfb46ae9dbe57c20be666d5, State: Initialized, Role: FOLLOWER
I20260812 08:02:59.479156 20418 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:02:59.479188 20420 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:02:59.479919 20416 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:02:59.480123 20416 raft_consensus.cc:385] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:59.480189 20416 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e10b84812514a76ab7225dfbf9329bf, State: Initialized, Role: FOLLOWER
I20260812 08:02:59.480268 20411 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: No bootstrap required, opened a new log
I20260812 08:02:59.480257 20420 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.005s
I20260812 08:02:59.480266 20418 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.003s
I20260812 08:02:59.480484 20411 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Time spent bootstrapping tablet: real 0.011s	user 0.004s	sys 0.000s
I20260812 08:02:59.480646 19702 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
I20260812 08:02:59.480646 19928 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
W20260812 08:02:59.481194 19703 tablet.cc:2406] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:02:59.482769 20416 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:02:59.484721 20416 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260812 08:02:59.485162 20361 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
I20260812 08:02:59.485353 20411 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:02:59.485662 20411 raft_consensus.cc:385] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:59.485730 20411 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 294e0e1970eb4e58bed426de35babb6b, State: Initialized, Role: FOLLOWER
I20260812 08:02:59.486218 20411 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:02:59.487386 20411 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260812 08:02:59.488209 19471 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
I20260812 08:02:59.488312 20415 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: No bootstrap required, opened a new log
I20260812 08:02:59.488575 20415 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Time spent bootstrapping tablet: real 0.018s	user 0.000s	sys 0.005s
I20260812 08:02:59.491254 20415 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:02:59.491465 20415 raft_consensus.cc:385] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:59.491530 20415 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38681904c36540a4876519ec5ffac6b6, State: Initialized, Role: FOLLOWER
I20260812 08:02:59.491967 20415 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:02:59.492677 20148 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
I20260812 08:02:59.492801 20415 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260812 08:02:59.498492 20442 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:02:59.498694 20442 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:02:59.500110 20442 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:02:59.505614 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:02:59.506013 19620 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 0.
I20260812 08:02:59.507308 19396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:02:59.507627 19396 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 0.
I20260812 08:02:59.507323 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
W20260812 08:02:59.509261 19738 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:02:59.509776 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:02:59.510051 20083 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 0.
I20260812 08:02:59.509343 18941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:02:59.510720 19741 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 285e078e6cfb46ae9dbe57c20be666d5, 294e0e1970eb4e58bed426de35babb6b, 38681904c36540a4876519ec5ffac6b6, b7def2bd3a74420d9deec19b32f541ec; no voters: 5b4c3ad48b794617893c88a077be1b8d
W20260812 08:02:59.511063 19742 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:02:59.511067 20442 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:02:59.511184 20442 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:02:59.511231 20442 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:59.511801 20292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:02:59.512064 20292 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 0.
I20260812 08:02:59.512380 20442 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:02:59.513105 20442 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 1 election: Requested vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:02:59.513423 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec"
I20260812 08:02:59.513536 20292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf"
I20260812 08:02:59.513607 19620 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:59.513674 20292 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:59.513824 19396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b"
I20260812 08:02:59.513883 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6"
I20260812 08:02:59.513989 19396 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:59.514034 20083 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:59.514869 19620 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 1.
I20260812 08:02:59.515172 19396 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 1.
I20260812 08:02:59.515182 20417 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Time spent bootstrapping tablet: real 0.050s	user 0.004s	sys 0.002s
I20260812 08:02:59.515669 20292 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 1.
I20260812 08:02:59.515722 18941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7"
I20260812 08:02:59.515911 20083 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 1.
W20260812 08:02:59.516162 19742 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:02:59.516274 19741 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 285e078e6cfb46ae9dbe57c20be666d5, 294e0e1970eb4e58bed426de35babb6b, 38681904c36540a4876519ec5ffac6b6; no voters: a5813459b1d94e089ffbe751f4310de7
I20260812 08:02:59.516395 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d"
I20260812 08:02:59.516489 20442 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:59.516774 20442 raft_consensus.cc:697] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 1 LEADER]: Becoming Leader. State: Replica: 285e078e6cfb46ae9dbe57c20be666d5, State: Running, Role: LEADER
W20260812 08:02:59.516928 19738 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:02:59.517155 20442 consensus_queue.cc:237] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:02:59.518651 20417 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:02:59.518842 20417 raft_consensus.cc:385] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:59.518923 20417 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b4c3ad48b794617893c88a077be1b8d, State: Initialized, Role: FOLLOWER
I20260812 08:02:59.519377 20417 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:02:59.519730 20409 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Time spent bootstrapping tablet: real 0.055s	user 0.004s	sys 0.002s
I20260812 08:02:59.520507 20417 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.002s
I20260812 08:02:59.521741 18673 catalog_manager.cc:5719] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69). New cstate: current_term: 1 leader_uuid: "285e078e6cfb46ae9dbe57c20be666d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } health_report { overall_health: UNKNOWN } } }
W20260812 08:02:59.522122 20458 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 11ms of latency in SegmentAllocator::Sync()
I20260812 08:02:59.523196 20409 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:02:59.523403 20409 raft_consensus.cc:385] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:59.523463 20409 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5813459b1d94e089ffbe751f4310de7, State: Initialized, Role: FOLLOWER
I20260812 08:02:59.523648 20461 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:02:59.523795 20461 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:02:59.523962 20409 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:02:59.524932 20409 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.001s
I20260812 08:02:59.526360 20461 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
W20260812 08:02:59.526728 19472 tablet.cc:2406] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:02:59.536227 20292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:02:59.536523 20292 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b4c3ad48b794617893c88a077be1b8d in current term 1: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:02:59.536792 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:02:59.536986 19620 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b4c3ad48b794617893c88a077be1b8d in current term 1: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:02:59.537199 18941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:02:59.537446 18941 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 0.
I20260812 08:02:59.538519 19848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:02:59.540285 19396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:02:59.540511 19396 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b4c3ad48b794617893c88a077be1b8d in current term 1: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:02:59.540987 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:02:59.541220 20083 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b4c3ad48b794617893c88a077be1b8d in current term 1: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:02:59.541316 19047 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 5b4c3ad48b794617893c88a077be1b8d, a5813459b1d94e089ffbe751f4310de7; no voters: 0e10b84812514a76ab7225dfbf9329bf, 285e078e6cfb46ae9dbe57c20be666d5, 294e0e1970eb4e58bed426de35babb6b, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:02:59.546156 20461 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:59.547222 20461 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260812 08:02:59.563828 20455 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260812 08:02:59.563928 20455 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260812 08:02:59.564365 20501 fault_injection.cc:59] injecting fault for kudu (pid 19711): FLAGS_fault_crash_on_leader_request_fraction (process will exit)
W20260812 08:02:59.564658 19251 tablet.cc:2406] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:02:59.565181 20292 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 1 FOLLOWER]: Refusing update from remote peer 285e078e6cfb46ae9dbe57c20be666d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260812 08:02:59.591363 20173 connection.cc:573] server connection from 127.18.42.69:56533 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 08:02:59.591516 20378 meta_cache.cc:302] tablet 50b14eb091f7476d9d7682e503745c93: replica 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819) has failed: Network error: recv got EOF from 127.18.42.69:34819 (error 108)
I20260812 08:02:59.600472 18601 ts_itest-base.cc:575] TS 4 appears to have crashed. Restarting.
I20260812 08:02:59.601065 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.69:34819
--local_ip_for_outbound_sockets=127.18.42.69
--tserver_master_addrs=127.18.42.126:35213
--webserver_port=40235
--webserver_interface=127.18.42.69
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.05
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
W20260812 08:02:59.601151 19538 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.601509 19547 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.601559 19524 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.601151 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.602258 19523 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.602488 19529 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.602566 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.602680 19550 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.601151 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.603296 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.610822 19015 tablet.cc:2406] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:02:59.614192 18844 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.614192 18854 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.614269 18857 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.614868 18851 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.614920 18852 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.614279 18839 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.615442 18848 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.615896 18843 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.614192 18859 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.616297 18858 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.626276 19087 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:02:59.626277 19094 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:02:59.626410 19092 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:02:59.626626 19090 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:02:59.627379 19088 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:02:59.627741 19094 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:02:59.627820 19089 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:02:59.627913 19090 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:02:59.628172 19085 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:02:59.628441 19083 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:02:59.635455 20363 tablet.cc:2406] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:02:59.641252 19307 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53188: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:02:59.641330 19306 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53188: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:02:59.642795 19310 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53188: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:02:59.642935 19315 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53188: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:02:59.643424 19316 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53188: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:02:59.643694 19311 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53188: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:02:59.643842 19317 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53188: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:02:59.641252 19312 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53188: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:02:59.641256 19318 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53188: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:02:59.644333 19319 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53188: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:02:59.655465 20006 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.655647 20007 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.655822 19991 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.656163 20005 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.655469 19992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.656471 20003 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.656673 19995 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.656701 19989 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.657306 19998 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.657845 19996 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.673091 20206 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:02:59.673183 20212 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:02:59.673550 20198 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:02:59.673959 20201 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:02:59.674502 20200 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:02:59.675026 20199 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:02:59.675508 20197 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:02:59.675990 20196 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:02:59.676393 20208 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:02:59.673094 20210 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:02:59.684252 20149 tablet.cc:2406] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:02:59.695528 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.696230 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.697136 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.697136 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.699751 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.702998 18858 tablet_service.cc:3018] Rejecting scan request for tablet 50b14eb091f7476d9d7682e503745c93: Uninitialized: clean time has not yet been initialized
W20260812 08:02:59.705229 18858 tablet_service.cc:3018] Rejecting scan request for tablet 50b14eb091f7476d9d7682e503745c93: Uninitialized: clean time has not yet been initialized
W20260812 08:02:59.706825 19083 tablet_service.cc:3018] Rejecting scan request for tablet 50b14eb091f7476d9d7682e503745c93: Uninitialized: clean time has not yet been initialized
W20260812 08:02:59.706864 19085 tablet_service.cc:3018] Rejecting scan request for tablet 50b14eb091f7476d9d7682e503745c93: Uninitialized: clean time has not yet been initialized
W20260812 08:02:59.708590 18858 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.709322 18854 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.710019 19318 tablet_service.cc:3018] Rejecting scan request for tablet 50b14eb091f7476d9d7682e503745c93: Uninitialized: clean time has not yet been initialized
W20260812 08:02:59.710587 18854 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.710625 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.710927 18859 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.711491 18843 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.712122 18843 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.712602 18854 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.713217 19319 tablet_service.cc:3018] Rejecting scan request for tablet 50b14eb091f7476d9d7682e503745c93: Uninitialized: clean time has not yet been initialized
W20260812 08:02:59.713843 18859 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:02:59.717080 20495 meta_cache.cc:1510] marking tablet server 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819) as failed
I20260812 08:02:59.717083 20493 meta_cache.cc:1510] marking tablet server 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819) as failed
W20260812 08:02:59.720934 19996 tablet_service.cc:3018] Rejecting scan request for tablet 50b14eb091f7476d9d7682e503745c93: Uninitialized: clean time has not yet been initialized
W20260812 08:02:59.720988 19998 tablet_service.cc:3018] Rejecting scan request for tablet 50b14eb091f7476d9d7682e503745c93: Uninitialized: clean time has not yet been initialized
W20260812 08:02:59.724143 20210 tablet_service.cc:3018] Rejecting scan request for tablet 50b14eb091f7476d9d7682e503745c93: Uninitialized: clean time has not yet been initialized
W20260812 08:02:59.724725 20206 tablet_service.cc:3018] Rejecting scan request for tablet 50b14eb091f7476d9d7682e503745c93: Uninitialized: clean time has not yet been initialized
W20260812 08:02:59.726368 18859 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.727301 19548 tablet_service.cc:3018] Rejecting scan request for tablet 50b14eb091f7476d9d7682e503745c93: Uninitialized: clean time has not yet been initialized
W20260812 08:02:59.727597 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.727826 19083 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:02:59.728874 19083 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:02:59.735071 19535 tablet_service.cc:3018] Rejecting scan request for tablet 50b14eb091f7476d9d7682e503745c93: Uninitialized: clean time has not yet been initialized
W20260812 08:02:59.735180 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.737438 19083 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:02:59.738323 19085 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:02:59.739534 19085 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:02:59.740685 19319 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53188: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:02:59.741118 20508 flags.cc:432] Enabled unsafe flag: --fault_crash_on_leader_request_fraction=0.050000000000000003
W20260812 08:02:59.741400 20508 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:02:59.741472 20508 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:02:59.741505 20508 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:02:59.741588 20508 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:59.741624 20508 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:59.741688 20508 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:59.741811 19319 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53188: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:02:59.742856 19319 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53188: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:02:59.743736 19319 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53188: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:02:59.745991 20508 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:59.746152 20508 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.69
W20260812 08:02:59.747020 19319 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53188: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
I20260812 08:02:59.750730 20508 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.050000000000000003
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.69:34819
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data/info.pb
--webserver_interface=127.18.42.69
--webserver_port=40235
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.20508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.69

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
W20260812 08:02:59.751579 19319 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53188: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:02:59.751621 19318 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53188: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:02:59.752112 19312 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53188: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
I20260812 08:02:59.752210 20508 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:59.753414 20508 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:59.754252 19312 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53188: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
E20260812 08:02:59.756577 20508 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:02:59.757608 19312 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53188: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:02:59.758900 19996 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.758919 19998 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.760213 19996 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.760519 19998 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.761236 20513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:59.761250 20514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:59.761665 19998 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.761271 20516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:59.762080 20508 server_base.cc:1061] running on GCE node
I20260812 08:02:59.762529 20508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:59.763127 20508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:59.764312 20508 hybrid_clock.cc:648] HybridClock initialized: now 1786521779764305 us; error 36 us; skew 500 ppm
I20260812 08:02:59.766383 20508 webserver.cc:533] Webserver started at http://127.18.42.69:40235/ using document root <none> and password file <none>
I20260812 08:02:59.767210 20508 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:59.767314 20508 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
W20260812 08:02:59.767341 19998 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.768616 19998 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.771006 19998 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
I20260812 08:02:59.771713 20508 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.000s
W20260812 08:02:59.772176 19996 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.772253 19998 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
I20260812 08:02:59.774219 20526 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:59.775542 20508 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
W20260812 08:02:59.775943 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:02:59.775663 20508 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
uuid: "285e078e6cfb46ae9dbe57c20be666d5"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:02:59.776166 20508 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260812 08:02:59.776254 20210 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:02:59.776579 20206 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:02:59.777127 20208 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:02:59.779428 20208 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:02:59.786895 20208 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:02:59.786897 20206 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:02:59.788986 20208 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:02:59.790177 20208 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:02:59.790326 20206 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:02:59.792762 19085 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:02:59.792762 19083 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:02:59.794471 20206 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:02:59.795630 19083 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:02:59.804915 19083 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:02:59.809947 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:02:59.811230 20508 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:59.812036 20508 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:59.812276 20508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:59.813177 20508 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:59.815158 20540 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260812 08:02:59.816334 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:02:59.819331 20508 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:02:59.819427 20508 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.005s	user 0.001s	sys 0.000s
I20260812 08:02:59.819535 20508 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:02:59.821151 20508 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:02:59.821228 20508 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:02:59.821732 20540 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Bootstrap starting.
I20260812 08:02:59.828528 20540 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Log is configured to *not* fsync() on all Append() calls
W20260812 08:02:59.831334 18859 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.832072 18854 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:02:59.832324 20540 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 2 replicates
I20260812 08:02:59.832551 20445 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:02:59.832696 20445 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:02:59.832896 20540 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Bootstrap complete.
W20260812 08:02:59.833086 20540 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 41ms of latency in SegmentAllocator::Sync()
I20260812 08:02:59.834802 20445 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
W20260812 08:02:59.836123 19083 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:02:59.836853 19085 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:02:59.837281 19090 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:02:59.837742 19089 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
I20260812 08:02:59.838434 20508 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.69:34819
I20260812 08:02:59.839761 20508 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data/info.pb
W20260812 08:02:59.840145 19089 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:02:59.840651 19090 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:02:59.841074 19085 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
I20260812 08:02:59.841771 20719 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.69:34819 every 8 connection(s)
W20260812 08:02:59.843648 19284 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.42.69:34819: connect: Connection refused (error 111)
W20260812 08:02:59.846462 19085 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
I20260812 08:02:59.846895 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 20508
I20260812 08:02:59.847103 18941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:02:59.847276 18941 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 0.
I20260812 08:02:59.849448 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:02:59.849680 19620 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 1.
I20260812 08:02:59.851461 20292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:02:59.851626 20292 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 1.
I20260812 08:02:59.851781 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:02:59.851970 19165 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 1.
I20260812 08:02:59.852087 19284 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 294e0e1970eb4e58bed426de35babb6b, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec; no voters: 
I20260812 08:02:59.852421 20445 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 08:02:59.852562 20445 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:02:59.852626 20445 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:02:59.854055 20445 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:02:59.854841 20445 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 2 election: Requested vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:02:59.855357 18941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7"
I20260812 08:02:59.855357 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d"
I20260812 08:02:59.855499 19165 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:02:59.855501 18941 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 0 FOLLOWER]: Advancing to term 2
I20260812 08:02:59.856650 18941 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 2.
I20260812 08:02:59.856650 19165 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 2.
I20260812 08:02:59.857381 20292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf"
I20260812 08:02:59.857399 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:02:59.857566 20292 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:02:59.857575 20083 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 1.
I20260812 08:02:59.857697 20078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6"
I20260812 08:02:59.857820 20078 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:02:59.858662 20292 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 2.
I20260812 08:02:59.858983 20078 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 2.
I20260812 08:02:59.859285 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec"
I20260812 08:02:59.859440 19620 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:02:59.859668 19283 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 294e0e1970eb4e58bed426de35babb6b, 38681904c36540a4876519ec5ffac6b6, a5813459b1d94e089ffbe751f4310de7; no voters: 
I20260812 08:02:59.860522 19620 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 2.
I20260812 08:02:59.860726 20445 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:02:59.861011 20445 raft_consensus.cc:697] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 2 LEADER]: Becoming Leader. State: Replica: 294e0e1970eb4e58bed426de35babb6b, State: Running, Role: LEADER
I20260812 08:02:59.861380 20445 consensus_queue.cc:237] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:02:59.865238 20722 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:02:59.865597 20722 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:59.866318 20722 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:02:59.866547 18661 catalog_manager.cc:5719] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b reported cstate change: term changed from 1 to 2, leader changed from 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69) to 294e0e1970eb4e58bed426de35babb6b (127.18.42.67). New cstate: current_term: 2 leader_uuid: "294e0e1970eb4e58bed426de35babb6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } health_report { overall_health: UNKNOWN } } }
I20260812 08:02:59.866828 20631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
W20260812 08:02:59.868610 20731 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Injecting 12ms of latency in SegmentAllocator::Sync()
I20260812 08:02:59.868623 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5"
W20260812 08:02:59.869423 19284 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819): Illegal state: must be running to vote when last-logged opid is not known
W20260812 08:02:59.869828 19284 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:02:59.869818 18661 ts_manager.cc:194] Re-registered known tserver with Master: 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819)
I20260812 08:02:59.870812 18661 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.69:54587
I20260812 08:02:59.874503 20540 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Time spent bootstrapping tablet: real 0.053s	user 0.012s	sys 0.000s
I20260812 08:02:59.876933 20540 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 1 FOLLOWER]: Replica starting. Triggering 2 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:02:59.879053 20540 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 285e078e6cfb46ae9dbe57c20be666d5, State: Initialized, Role: FOLLOWER
I20260812 08:02:59.879554 20540 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:02:59.880260 20722 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
I20260812 08:02:59.880517 20540 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
W20260812 08:02:59.891179 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.891179 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.891160 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.891160 19550 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.892421 19550 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.892421 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.892570 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.893500 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.894636 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:02:59.895031 18854 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.913741 18854 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.914798 18854 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.914884 18843 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.914884 18859 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.916090 18854 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.916143 18859 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.916154 18843 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47726: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:02:59.917649 19085 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58712: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:02:59.921553 20445 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260812 08:02:59.921631 20445 fault_injection.cc:44] THIS SERVER MAY CRASH!
I20260812 08:02:59.922734 20633 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:02:59.922889 19620 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 2 FOLLOWER]: Refusing update from remote peer 294e0e1970eb4e58bed426de35babb6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 08:02:59.923331 19165 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 2 FOLLOWER]: Refusing update from remote peer 294e0e1970eb4e58bed426de35babb6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 08:02:59.923693 20078 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 2 FOLLOWER]: Refusing update from remote peer 294e0e1970eb4e58bed426de35babb6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 08:02:59.924149 20738 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:02:59.924682 20746 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:02:59.925319 20292 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 2 FOLLOWER]: Refusing update from remote peer 294e0e1970eb4e58bed426de35babb6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 08:02:59.925366 20633 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 2 FOLLOWER]: Refusing update from remote peer 294e0e1970eb4e58bed426de35babb6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 2. (term mismatch)
I20260812 08:02:59.925434 20633 pending_rounds.cc:85] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Aborting all ops after (but not including) 1
I20260812 08:02:59.925496 20633 pending_rounds.cc:107] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Aborting uncommitted WRITE_OP operation due to leader change: 1.2
I20260812 08:02:59.926744 18941 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 2 FOLLOWER]: Refusing update from remote peer 294e0e1970eb4e58bed426de35babb6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260812 08:02:59.927946 20738 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:02:59.928113 20445 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:02:59.928329 20737 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:02:59.928534 20746 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:02:59.928653 20746 consensus_queue.cc:1243] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [LEADER]: Peer 285e078e6cfb46ae9dbe57c20be666d5 log is divergent from this leader: its last log entry 1.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20260812 08:02:59.930334 20633 pending_rounds.cc:85] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Aborting all ops after (but not including) 0
I20260812 08:02:59.930414 20633 pending_rounds.cc:107] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260812 08:02:59.930516 20633 raft_consensus.cc:2889] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260812 08:02:59.937183 20764 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 76ms of latency in SegmentAllocator::Sync()
W20260812 08:02:59.938006 20762 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Injecting 101ms of latency in SegmentAllocator::Sync()
W20260812 08:02:59.938935 20761 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 30ms of latency in SegmentAllocator::Sync()
W20260812 08:02:59.939368 20731 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Injecting 47ms of latency in SegmentAllocator::Sync()
W20260812 08:02:59.942792 20755 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 56ms of latency in SegmentAllocator::Sync()
W20260812 08:02:59.944648 20767 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Injecting 165ms of latency in SegmentAllocator::Sync()
W20260812 08:02:59.946909 20752 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Injecting 10ms of latency in SegmentAllocator::Sync()
W20260812 08:02:59.961417 20752 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Injecting 36ms of latency in SegmentAllocator::Sync()
W20260812 08:02:59.978940 20761 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 22ms of latency in SegmentAllocator::Sync()
W20260812 08:03:00.003163 20752 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Injecting 17ms of latency in SegmentAllocator::Sync()
I20260812 08:03:00.003646 20747 mvcc.cc:204] Tried to move back new op lower bound from 7317593210562125824 to 7317593210321301504. Current Snapshot: MvccSnapshot[applied={T|T < 7317593210554376192}]
W20260812 08:03:00.003986 20755 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 63ms of latency in SegmentAllocator::Sync()
W20260812 08:03:00.019531 20764 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 30ms of latency in SegmentAllocator::Sync()
W20260812 08:03:00.023797 20752 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Injecting 100ms of latency in SegmentAllocator::Sync()
E20260812 08:03:00.040820 20445 fault_injection.cc:59] injecting fault for kudu (pid 19255): FLAGS_fault_crash_on_leader_request_fraction (process will exit)
W20260812 08:03:00.067158 18811 connection.cc:573] server connection from 127.18.42.67:45383 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 08:03:00.068060 19501 connection.cc:466] server connection from 127.18.42.67:44343 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.18.42.67:44343 (request call id 5) could send its response
W20260812 08:03:00.070016 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:03:00.072661 18601 ts_itest-base.cc:575] TS 2 appears to have crashed. Restarting.
I20260812 08:03:00.073207 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.67:40205
--local_ip_for_outbound_sockets=127.18.42.67
--tserver_master_addrs=127.18.42.126:35213
--webserver_port=39753
--webserver_interface=127.18.42.67
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.05
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
W20260812 08:03:00.110963 20172 connection.cc:466] server connection from 127.18.42.67:60035 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.18.42.67:60035 (request call id 3) could send its response
W20260812 08:03:00.124673 19956 connection.cc:466] server connection from 127.18.42.67:48089 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.18.42.67:48089 (request call id 7) could send its response
W20260812 08:03:00.231778 20791 flags.cc:432] Enabled unsafe flag: --fault_crash_on_leader_request_fraction=0.050000000000000003
W20260812 08:03:00.232100 20791 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:03:00.232136 20791 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:03:00.232173 20791 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:03:00.232249 20791 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:00.232312 20791 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:00.232417 20791 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:00.236147 20791 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:00.236454 20791 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.67
I20260812 08:03:00.240962 20791 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.050000000000000003
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.67:40205
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data/info.pb
--webserver_interface=127.18.42.67
--webserver_port=39753
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.20791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:00.242439 20791 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:00.243762 20791 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:03:00.246927 20791 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:03:00.251426 20799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:00.251413 20802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:00.251415 20798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:00.252985 20791 server_base.cc:1061] running on GCE node
I20260812 08:03:00.253497 20791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:00.254226 20791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:00.255430 20791 hybrid_clock.cc:648] HybridClock initialized: now 1786521780255412 us; error 90 us; skew 500 ppm
I20260812 08:03:00.258654 20791 webserver.cc:533] Webserver started at http://127.18.42.67:39753/ using document root <none> and password file <none>
I20260812 08:03:00.259409 20791 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:00.259500 20791 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:00.264053 20791 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.000s	sys 0.004s
I20260812 08:03:00.266598 20810 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:00.268079 20791 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.003s
I20260812 08:03:00.268249 20791 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal
uuid: "294e0e1970eb4e58bed426de35babb6b"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:03:00.268726 20791 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:00.291546 20791 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:00.292335 20791 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:00.292560 20791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:00.293201 20791 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:00.294565 20820 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:03:00.298056 20791 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:00.298152 20791 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:03:00.298243 20791 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:00.299816 20791 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:00.299886 20791 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:03:00.300207 20820 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Bootstrap starting.
I20260812 08:03:00.302314 20439 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 294e0e1970eb4e58bed426de35babb6b)
I20260812 08:03:00.302451 20439 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:00.304137 20439 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:00.308421 20820 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:00.311103 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 13 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:00.311277 20733 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 294e0e1970eb4e58bed426de35babb6b)
I20260812 08:03:00.311434 20733 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
W20260812 08:03:00.312142 19501 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.42.67:40205: connect: Connection refused (error 111)
I20260812 08:03:00.313148 20733 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:00.315665 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 13 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:00.315868 20633 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b7def2bd3a74420d9deec19b32f541ec for term 3 because replica has last-logged OpId of term: 2 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 13.
I20260812 08:03:00.317243 20462 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 294e0e1970eb4e58bed426de35babb6b)
I20260812 08:03:00.317386 20462 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:00.317555 20292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 13 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
W20260812 08:03:00.319015 19501 leader_election.cc:336] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205): Network error: Client connection negotiation failed: client connection to 127.18.42.67:40205: connect: Connection refused (error 111)
I20260812 08:03:00.319155 20462 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:00.320616 20791 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.67:40205
W20260812 08:03:00.320981 18816 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.42.67:40205: connect: Connection refused (error 111)
I20260812 08:03:00.321779 20791 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data/info.pb
I20260812 08:03:00.322445 18941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 13 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:00.322623 18941 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b7def2bd3a74420d9deec19b32f541ec for term 3 because replica has last-logged OpId of term: 2 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 13.
I20260812 08:03:00.323143 19502 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: b7def2bd3a74420d9deec19b32f541ec; no voters: 0e10b84812514a76ab7225dfbf9329bf, 294e0e1970eb4e58bed426de35babb6b, 5b4c3ad48b794617893c88a077be1b8d, a5813459b1d94e089ffbe751f4310de7
I20260812 08:03:00.323428 20439 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 08:03:00.325861 20078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 13 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:00.326202 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 20791
I20260812 08:03:00.325008 21025 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.67:40205 every 8 connection(s)
W20260812 08:03:00.328956 20533 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.42.67:40205: connect: Connection refused (error 111)
I20260812 08:03:00.330458 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:00.330624 19620 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 2.
I20260812 08:03:00.333382 20461 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 294e0e1970eb4e58bed426de35babb6b)
I20260812 08:03:00.333583 20461 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:00.334261 20461 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:00.334252 20292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:00.334949 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:00.335098 19620 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 2.
I20260812 08:03:00.335398 20292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:00.336714 20078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:00.337419 20820 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 10 replicates
I20260812 08:03:00.337984 20820 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Bootstrap complete.
W20260812 08:03:00.338166 20820 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Injecting 18ms of latency in SegmentAllocator::Sync()
I20260812 08:03:00.338240 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:00.338416 19620 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b4c3ad48b794617893c88a077be1b8d for term 3 because replica has last-logged OpId of term: 2 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260812 08:03:00.339228 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:00.339282 18941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:00.339432 18941 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b4c3ad48b794617893c88a077be1b8d for term 3 because replica has last-logged OpId of term: 2 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260812 08:03:00.339512 20078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:00.339795 19051 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 5b4c3ad48b794617893c88a077be1b8d; no voters: 0e10b84812514a76ab7225dfbf9329bf, 38681904c36540a4876519ec5ffac6b6, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:00.340083 20461 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 08:03:00.341064 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:00.341195 20633 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 2.
I20260812 08:03:00.341747 20443 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 294e0e1970eb4e58bed426de35babb6b)
I20260812 08:03:00.341881 20443 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:00.344076 20443 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785)
I20260812 08:03:00.348631 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:00.348646 20292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:00.348786 19165 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 2.
I20260812 08:03:00.348785 20292 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 2.
I20260812 08:03:00.349088 18941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:00.349241 18941 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 2.
I20260812 08:03:00.349383 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:00.349510 19165 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 2.
I20260812 08:03:00.349948 20531 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 285e078e6cfb46ae9dbe57c20be666d5, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec; no voters: 38681904c36540a4876519ec5ffac6b6
I20260812 08:03:00.350515 18815 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 285e078e6cfb46ae9dbe57c20be666d5, 5b4c3ad48b794617893c88a077be1b8d, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec; no voters: 0e10b84812514a76ab7225dfbf9329bf, 38681904c36540a4876519ec5ffac6b6
I20260812 08:03:00.350828 20733 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 08:03:00.350891 20733 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 294e0e1970eb4e58bed426de35babb6b)
I20260812 08:03:00.350942 20733 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 2 FOLLOWER]: Advancing to term 3
I20260812 08:03:00.352109 20733 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:00.351778 20937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:00.353088 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec"
I20260812 08:03:00.353209 19620 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 2 FOLLOWER]: Advancing to term 3
W20260812 08:03:00.354169 18816 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:00.354264 19620 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 3.
I20260812 08:03:00.354804 20462 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 08:03:00.354874 20462 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 294e0e1970eb4e58bed426de35babb6b)
I20260812 08:03:00.354918 20462 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 2 FOLLOWER]: Advancing to term 3
I20260812 08:03:00.356069 20462 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:00.356595 20820 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Time spent bootstrapping tablet: real 0.057s	user 0.020s	sys 0.000s
I20260812 08:03:00.357329 19619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec"
I20260812 08:03:00.357507 19619 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a5813459b1d94e089ffbe751f4310de7 in current term 3: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:03:00.358518 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:00.358697 20633 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e10b84812514a76ab7225dfbf9329bf in current term 3: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:03:00.358680 20820 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 2 FOLLOWER]: Replica starting. Triggering 10 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:00.359665 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:00.359820 19620 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e10b84812514a76ab7225dfbf9329bf in current term 3: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:03:00.361054 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d"
I20260812 08:03:00.361209 19165 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 2 FOLLOWER]: Advancing to term 3
I20260812 08:03:00.362489 19165 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 3.
I20260812 08:03:00.363304 20820 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 294e0e1970eb4e58bed426de35babb6b, State: Initialized, Role: FOLLOWER
I20260812 08:03:00.363387 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:00.363552 19163 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e10b84812514a76ab7225dfbf9329bf in current term 3: Already voted for candidate a5813459b1d94e089ffbe751f4310de7 in this term.
I20260812 08:03:00.363931 20820 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:00.364943 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:00.365072 20820 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260812 08:03:00.365115 20633 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b4c3ad48b794617893c88a077be1b8d in current term 3: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:03:00.366019 20937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:00.366238 20937 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e10b84812514a76ab7225dfbf9329bf for term 3 because replica has last-logged OpId of term: 2 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260812 08:03:00.366211 20733 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 3 election: Requested vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:00.366987 20462 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 3 election: Requested vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:00.367204 20292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf"
I20260812 08:03:00.367326 20292 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 2 FOLLOWER]: Advancing to term 3
I20260812 08:03:00.367326 20078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6"
I20260812 08:03:00.367949 18941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:00.368119 18941 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e10b84812514a76ab7225dfbf9329bf in current term 3: Already voted for candidate a5813459b1d94e089ffbe751f4310de7 in this term.
I20260812 08:03:00.368476 20292 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 3.
I20260812 08:03:00.368901 20173 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf; no voters: 285e078e6cfb46ae9dbe57c20be666d5, 5b4c3ad48b794617893c88a077be1b8d, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:00.369302 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5"
I20260812 08:03:00.369490 20633 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a5813459b1d94e089ffbe751f4310de7 in current term 3: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:03:00.370009 18941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7"
I20260812 08:03:00.370152 18941 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 285e078e6cfb46ae9dbe57c20be666d5 in current term 3: Already voted for candidate a5813459b1d94e089ffbe751f4310de7 in this term.
I20260812 08:03:00.370383 20078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6"
I20260812 08:03:00.370863 20078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:00.370939 20937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b"
I20260812 08:03:00.371045 20937 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 2 FOLLOWER]: Advancing to term 3
I20260812 08:03:00.371096 20936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:00.372051 20443 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 08:03:00.372174 20935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b"
I20260812 08:03:00.372049 20292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf"
I20260812 08:03:00.372438 20292 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 285e078e6cfb46ae9dbe57c20be666d5 in current term 3: Already voted for candidate a5813459b1d94e089ffbe751f4310de7 in this term.
I20260812 08:03:00.372439 20936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:00.372831 20536 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 285e078e6cfb46ae9dbe57c20be666d5, b7def2bd3a74420d9deec19b32f541ec; no voters: 0e10b84812514a76ab7225dfbf9329bf, 294e0e1970eb4e58bed426de35babb6b, 38681904c36540a4876519ec5ffac6b6, a5813459b1d94e089ffbe751f4310de7
I20260812 08:03:00.372886 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d"
I20260812 08:03:00.372948 21033 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:03:00.373050 19165 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 285e078e6cfb46ae9dbe57c20be666d5 in current term 3: Already voted for candidate a5813459b1d94e089ffbe751f4310de7 in this term.
I20260812 08:03:00.373101 20733 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260812 08:03:00.373157 21033 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:00.373495 20937 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 3.
I20260812 08:03:00.373847 21033 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:03:00.373978 18816 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 294e0e1970eb4e58bed426de35babb6b, 5b4c3ad48b794617893c88a077be1b8d, a5813459b1d94e089ffbe751f4310de7; no voters: 285e078e6cfb46ae9dbe57c20be666d5, 38681904c36540a4876519ec5ffac6b6, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:00.374223 20462 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 08:03:00.374603 20462 raft_consensus.cc:697] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 3 LEADER]: Becoming Leader. State: Replica: a5813459b1d94e089ffbe751f4310de7, State: Running, Role: LEADER
I20260812 08:03:00.374948 20462 consensus_queue.cc:237] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:00.375846 18674 ts_manager.cc:194] Re-registered known tserver with Master: 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205)
W20260812 08:03:00.376708 20764 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 42ms of latency in SegmentAllocator::Sync()
I20260812 08:03:00.376780 18674 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.67:33185
I20260812 08:03:00.377960 18674 catalog_manager.cc:5719] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 reported cstate change: term changed from 2 to 3, leader changed from 294e0e1970eb4e58bed426de35babb6b (127.18.42.67) to a5813459b1d94e089ffbe751f4310de7 (127.18.42.65). New cstate: current_term: 3 leader_uuid: "a5813459b1d94e089ffbe751f4310de7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } health_report { overall_health: UNKNOWN } } }
I20260812 08:03:00.378840 21033 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
I20260812 08:03:00.423969 20447 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 294e0e1970eb4e58bed426de35babb6b)
I20260812 08:03:00.424122 20447 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:00.425719 20447 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:00.431627 18941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:00.432000 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:00.432178 20633 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38681904c36540a4876519ec5ffac6b6 in current term 3: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:03:00.432960 20937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:00.433148 20937 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38681904c36540a4876519ec5ffac6b6 in current term 3: Already voted for candidate a5813459b1d94e089ffbe751f4310de7 in this term.
I20260812 08:03:00.434226 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:00.434270 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:00.434402 19165 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38681904c36540a4876519ec5ffac6b6 in current term 3: Already voted for candidate a5813459b1d94e089ffbe751f4310de7 in this term.
I20260812 08:03:00.434422 19620 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38681904c36540a4876519ec5ffac6b6 in current term 3: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:03:00.434767 19960 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 38681904c36540a4876519ec5ffac6b6; no voters: 285e078e6cfb46ae9dbe57c20be666d5, 294e0e1970eb4e58bed426de35babb6b, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:00.435021 20292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:00.435075 20447 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 2 FOLLOWER]: Advancing to term 3
I20260812 08:03:00.435190 20292 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38681904c36540a4876519ec5ffac6b6 in current term 3: Already voted for candidate a5813459b1d94e089ffbe751f4310de7 in this term.
I20260812 08:03:00.436268 20447 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260812 08:03:00.456423 20462 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260812 08:03:00.456506 20462 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260812 08:03:00.456525 20462 fault_injection.cc:59] injecting fault for kudu (pid 18784): FLAGS_fault_crash_on_leader_request_fraction (process will exit)
I20260812 08:03:00.474596 18601 ts_itest-base.cc:575] TS 0 appears to have crashed. Restarting.
I20260812 08:03:00.475064 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.65:45661
--local_ip_for_outbound_sockets=127.18.42.65
--tserver_master_addrs=127.18.42.126:35213
--webserver_port=38137
--webserver_interface=127.18.42.65
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.05
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
W20260812 08:03:00.535840 19085 tablet_service.cc:3018] Rejecting scan request for tablet 50b14eb091f7476d9d7682e503745c93: Uninitialized: clean time has not yet been initialized
W20260812 08:03:00.537040 19085 tablet_service.cc:3018] Rejecting scan request for tablet 50b14eb091f7476d9d7682e503745c93: Uninitialized: clean time has not yet been initialized
W20260812 08:03:00.539428 20568 tablet_service.cc:3080] error setting up scanner 8a096c62203049cda619f97b955ae7c4 with request: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: tablet hasn't heard from leader or there hasn't been a stable leader for 0.535s (maximum allowed 0.150s): new_scan_request { tablet_id: "50b14eb091f7476d9d7682e503745c93" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\337\323\360\323\006\"3\n\005slave\022*\n f676baf4d90e47909c86ef51b3260926\020\001\030\001 \001(\001" signature: "<redacted>""3\272\377\3278\0220\236\'\036\225\3266\312\023\204L,\347\252\335\306\257\345\221\275\300\207\352\031\323W\277_ \003\305r\241\377 \344\323\265\031\320\027B*d\272\350xy\316h\030\272\227\026\236s\235\247" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "b7421069db4f4eb795977b787568b5f5"
W20260812 08:03:00.539427 20570 tablet_service.cc:3080] error setting up scanner ef59040dfb6747faa3439b13dd242d25 with request: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: tablet hasn't heard from leader or there hasn't been a stable leader for 0.535s (maximum allowed 0.150s): new_scan_request { tablet_id: "50b14eb091f7476d9d7682e503745c93" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\337\323\360\323\006\"3\n\005slave\022*\n f676baf4d90e47909c86ef51b3260926\020\001\030\001 \001(\001" signature: "<redacted>""3\272\377\3278\0220\236\'\036\225\3266\312\023\204L,\347\252\335\306\257\345\221\275\300\207\352\031\323W\277_ \003\305r\241\377 \344\323\265\031\320\027B*d\272\350xy\316h\030\272\227\026\236s\235\247" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "6d651800a540494bb477f0c7a2cb3c2b"
W20260812 08:03:00.586822 21091 flags.cc:432] Enabled unsafe flag: --fault_crash_on_leader_request_fraction=0.050000000000000003
W20260812 08:03:00.587078 21091 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:03:00.587113 21091 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:03:00.587174 21091 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:03:00.587256 21091 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:00.587308 21091 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:00.587393 21091 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:00.590903 21091 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:00.591090 21091 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.65
I20260812 08:03:00.595148 21091 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.050000000000000003
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.65:45661
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data/info.pb
--webserver_interface=127.18.42.65
--webserver_port=38137
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.21091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:00.596338 21091 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:00.597589 21091 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:03:00.600554 21091 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:03:00.604490 21105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:00.604496 21107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:00.604756 21091 server_base.cc:1061] running on GCE node
W20260812 08:03:00.604907 21111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:00.605270 21091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:00.605903 21091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:00.607095 21091 hybrid_clock.cc:648] HybridClock initialized: now 1786521780607079 us; error 37 us; skew 500 ppm
I20260812 08:03:00.609107 21091 webserver.cc:533] Webserver started at http://127.18.42.65:38137/ using document root <none> and password file <none>
I20260812 08:03:00.609841 21091 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:00.609956 21091 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:00.614100 21091 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:03:00.616314 21120 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:00.617436 21091 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:00.617617 21091 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/wal
uuid: "a5813459b1d94e089ffbe751f4310de7"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:03:00.618047 21091 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:00.658118 21091 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:00.658805 21091 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:00.658994 21091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:00.659647 21091 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:00.661001 21133 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:03:00.664642 21091 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:00.664726 21091 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.001s	sys 0.000s
I20260812 08:03:00.664819 21091 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:00.666370 21091 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:00.666465 21091 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.001s
I20260812 08:03:00.666778 21133 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Bootstrap starting.
I20260812 08:03:00.674516 21133 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:00.678050 21091 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.65:45661
I20260812 08:03:00.678102 21328 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.65:45661 every 8 connection(s)
I20260812 08:03:00.679009 21091 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data/info.pb
I20260812 08:03:00.682868 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 21091
I20260812 08:03:00.684018 21133 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 11 replicates
I20260812 08:03:00.684432 21133 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Bootstrap complete.
W20260812 08:03:00.684600 21133 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 49ms of latency in SegmentAllocator::Sync()
I20260812 08:03:00.687673 21332 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:03:00.687949 21332 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:00.688508 21332 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:03:00.690264 18664 ts_manager.cc:194] Re-registered known tserver with Master: a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661)
I20260812 08:03:00.691061 18664 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.65:48137
I20260812 08:03:00.734170 21133 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Time spent bootstrapping tablet: real 0.068s	user 0.014s	sys 0.004s
I20260812 08:03:00.737181 21133 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 3 FOLLOWER]: Replica starting. Triggering 11 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:00.740257 21133 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5813459b1d94e089ffbe751f4310de7, State: Initialized, Role: FOLLOWER
I20260812 08:03:00.740815 21133 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:00.741741 21332 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
I20260812 08:03:00.741793 21133 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260812 08:03:00.767614 21077 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:00.767772 21077 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:00.769392 21077 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:00.776146 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:00.776146 20078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:00.776198 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:00.776319 20078 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 3.
I20260812 08:03:00.776330 20633 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 3.
I20260812 08:03:00.776348 19620 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 3.
I20260812 08:03:00.776866 20813 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 285e078e6cfb46ae9dbe57c20be666d5, 294e0e1970eb4e58bed426de35babb6b, 38681904c36540a4876519ec5ffac6b6, b7def2bd3a74420d9deec19b32f541ec; no voters: 
I20260812 08:03:00.777158 21077 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 08:03:00.777237 21077 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:03:00.777309 21077 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 3 FOLLOWER]: Advancing to term 4
I20260812 08:03:00.778528 21077 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:00.779244 21077 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 4 election: Requested vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:00.779364 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec"
I20260812 08:03:00.779516 19620 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 3 FOLLOWER]: Advancing to term 4
I20260812 08:03:00.779487 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:00.779839 21239 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 294e0e1970eb4e58bed426de35babb6b for term 4 because replica has last-logged OpId of term: 3 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 14.
I20260812 08:03:00.780267 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:00.780408 19165 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 3.
I20260812 08:03:00.780581 19620 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 4.
I20260812 08:03:00.780628 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d"
I20260812 08:03:00.780745 19163 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 3 FOLLOWER]: Advancing to term 4
I20260812 08:03:00.781112 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7"
I20260812 08:03:00.781209 21239 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 3 FOLLOWER]: Advancing to term 4
I20260812 08:03:00.781211 20292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:00.781292 20291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf"
I20260812 08:03:00.781344 20292 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 3.
I20260812 08:03:00.781921 19163 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 4.
I20260812 08:03:00.782176 20078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6"
I20260812 08:03:00.782176 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5"
I20260812 08:03:00.782325 20078 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 3 FOLLOWER]: Advancing to term 4
I20260812 08:03:00.782325 20633 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 3 FOLLOWER]: Advancing to term 4
I20260812 08:03:00.783350 21239 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 294e0e1970eb4e58bed426de35babb6b for term 4 because replica has last-logged OpId of term: 3 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 14.
I20260812 08:03:00.783329 20633 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 4.
I20260812 08:03:00.783329 20078 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 4.
I20260812 08:03:00.783689 20815 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 285e078e6cfb46ae9dbe57c20be666d5, 294e0e1970eb4e58bed426de35babb6b, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec; no voters: 0e10b84812514a76ab7225dfbf9329bf
I20260812 08:03:00.783910 21077 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 4 FOLLOWER]: Leader election won for term 4
I20260812 08:03:00.784150 21077 raft_consensus.cc:697] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 4 LEADER]: Becoming Leader. State: Replica: 294e0e1970eb4e58bed426de35babb6b, State: Running, Role: LEADER
I20260812 08:03:00.784479 21077 consensus_queue.cc:237] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.14, Last appended by leader: 14, Current term: 4, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:00.787956 18664 catalog_manager.cc:5719] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b reported cstate change: term changed from 3 to 4, leader changed from a5813459b1d94e089ffbe751f4310de7 (127.18.42.65) to 294e0e1970eb4e58bed426de35babb6b (127.18.42.67). New cstate: current_term: 4 leader_uuid: "294e0e1970eb4e58bed426de35babb6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } health_report { overall_health: UNKNOWN } } }
W20260812 08:03:00.861128 21077 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260812 08:03:00.861222 21077 fault_injection.cc:44] THIS SERVER MAY CRASH!
I20260812 08:03:00.862284 19163 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 4 FOLLOWER]: Refusing update from remote peer 294e0e1970eb4e58bed426de35babb6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 15. (index mismatch)
I20260812 08:03:00.862906 21077 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:00.865927 21351 mvcc.cc:204] Tried to move back new op lower bound from 7317593210554376192 to 7317593210321301504. Current Snapshot: MvccSnapshot[applied={T|T < 7317593210554376192}]
E20260812 08:03:00.872237 21077 fault_injection.cc:59] injecting fault for kudu (pid 20791): FLAGS_fault_crash_on_leader_request_fraction (process will exit)
I20260812 08:03:00.888281 18601 ts_itest-base.cc:575] TS 2 appears to have crashed. Restarting.
I20260812 08:03:00.888695 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.67:40205
--local_ip_for_outbound_sockets=127.18.42.67
--tserver_master_addrs=127.18.42.126:35213
--webserver_port=39753
--webserver_interface=127.18.42.67
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.05
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
I20260812 08:03:00.997510 21336 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:00.997776 21336 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:00.999716 21336 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
W20260812 08:03:01.006139 21124 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.42.67:40205: connect: Connection refused (error 111)
I20260812 08:03:01.007269 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 5 candidate_status { last_received { term: 3 index: 15 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:01.007455 19620 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 4.
I20260812 08:03:01.008134 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 5 candidate_status { last_received { term: 3 index: 15 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:01.008292 20633 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 4.
I20260812 08:03:01.010403 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 5 candidate_status { last_received { term: 3 index: 15 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
W20260812 08:03:01.011209 21124 leader_election.cc:336] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205): Network error: Client connection negotiation failed: client connection to 127.18.42.67:40205: connect: Connection refused (error 111)
W20260812 08:03:01.011312 21357 flags.cc:432] Enabled unsafe flag: --fault_crash_on_leader_request_fraction=0.050000000000000003
W20260812 08:03:01.011664 21357 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:03:01.011713 21357 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:03:01.011747 21357 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:03:01.011806 21357 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:01.011835 21357 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
I20260812 08:03:01.011834 20291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 5 candidate_status { last_received { term: 3 index: 15 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
W20260812 08:03:01.011902 21357 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260812 08:03:01.011845 20078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 5 candidate_status { last_received { term: 3 index: 15 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:01.011955 20291 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 3.
I20260812 08:03:01.011989 20078 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 4.
I20260812 08:03:01.012312 21125 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 285e078e6cfb46ae9dbe57c20be666d5, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec; no voters: 294e0e1970eb4e58bed426de35babb6b, 5b4c3ad48b794617893c88a077be1b8d
I20260812 08:03:01.012629 21336 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 08:03:01.012706 21336 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:03:01.012748 21336 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 4 FOLLOWER]: Advancing to term 5
I20260812 08:03:01.014029 21336 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:01.014760 21336 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 5 election: Requested vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:01.014998 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 5 candidate_status { last_received { term: 3 index: 15 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d"
I20260812 08:03:01.015223 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 5 candidate_status { last_received { term: 3 index: 15 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec"
I20260812 08:03:01.015355 19620 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 4 FOLLOWER]: Advancing to term 5
I20260812 08:03:01.015935 20291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 5 candidate_status { last_received { term: 3 index: 15 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf"
I20260812 08:03:01.016067 20291 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 3 FOLLOWER]: Advancing to term 5
I20260812 08:03:01.016466 19620 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 5.
I20260812 08:03:01.016776 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 5 candidate_status { last_received { term: 3 index: 15 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5"
I20260812 08:03:01.016918 20633 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 4 FOLLOWER]: Advancing to term 5
W20260812 08:03:01.017030 21124 leader_election.cc:336] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205): Network error: Client connection negotiation failed: client connection to 127.18.42.67:40205: connect: Connection refused (error 111)
I20260812 08:03:01.017315 20291 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 5.
I20260812 08:03:01.017632 20078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 5 candidate_status { last_received { term: 3 index: 15 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6"
W20260812 08:03:01.017740 21357 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
I20260812 08:03:01.017778 20078 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 4 FOLLOWER]: Advancing to term 5
W20260812 08:03:01.017988 21357 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.67
I20260812 08:03:01.018025 20633 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 5.
I20260812 08:03:01.018497 21125 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 285e078e6cfb46ae9dbe57c20be666d5, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec; no voters: 294e0e1970eb4e58bed426de35babb6b, 5b4c3ad48b794617893c88a077be1b8d
I20260812 08:03:01.018713 21336 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 5 FOLLOWER]: Leader election won for term 5
I20260812 08:03:01.018921 20078 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 5.
I20260812 08:03:01.018983 21336 raft_consensus.cc:697] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 5 LEADER]: Becoming Leader. State: Replica: a5813459b1d94e089ffbe751f4310de7, State: Running, Role: LEADER
I20260812 08:03:01.019377 21336 consensus_queue.cc:237] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.15, Last appended by leader: 15, Current term: 5, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
W20260812 08:03:01.021855 21373 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 86ms of latency in SegmentAllocator::Sync()
I20260812 08:03:01.023468 18664 catalog_manager.cc:5719] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 reported cstate change: term changed from 4 to 5, leader changed from 294e0e1970eb4e58bed426de35babb6b (127.18.42.67) to a5813459b1d94e089ffbe751f4310de7 (127.18.42.65). New cstate: current_term: 5 leader_uuid: "a5813459b1d94e089ffbe751f4310de7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } health_report { overall_health: UNKNOWN } } }
I20260812 08:03:01.024567 21357 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.050000000000000003
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.67:40205
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data/info.pb
--webserver_interface=127.18.42.67
--webserver_port=39753
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.21357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:01.026108 21357 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:01.027405 21357 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:03:01.030493 21357 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:03:01.034335 21380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:01.034337 21379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:01.034483 21386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:01.034874 21357 server_base.cc:1061] running on GCE node
I20260812 08:03:01.035254 21357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:01.035836 21357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:01.037040 21357 hybrid_clock.cc:648] HybridClock initialized: now 1786521781037024 us; error 40 us; skew 500 ppm
I20260812 08:03:01.039258 21357 webserver.cc:533] Webserver started at http://127.18.42.67:39753/ using document root <none> and password file <none>
I20260812 08:03:01.039928 21357 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:01.040030 21357 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:01.044260 21357 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:03:01.047648 21396 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:01.049208 21357 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260812 08:03:01.049396 21357 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal
uuid: "294e0e1970eb4e58bed426de35babb6b"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:03:01.050127 21357 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:01.069764 21357 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:01.070508 21357 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:01.070714 21357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:01.071360 21357 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:01.072629 21404 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:03:01.076254 21357 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:01.076333 21357 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:03:01.076460 21357 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:01.078181 21357 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:01.078233 21357 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:03:01.078667 21404 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Bootstrap starting.
I20260812 08:03:01.085022 21404 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:01.090173 21357 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.67:40205
I20260812 08:03:01.090241 21598 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.67:40205 every 8 connection(s)
I20260812 08:03:01.091177 21357 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data/info.pb
I20260812 08:03:01.095849 21404 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 11 replicates
I20260812 08:03:01.096323 21404 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Bootstrap complete.
W20260812 08:03:01.096503 21404 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Injecting 38ms of latency in SegmentAllocator::Sync()
I20260812 08:03:01.098488 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 21357
I20260812 08:03:01.101052 21601 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:03:01.101362 21601 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:01.102025 21601 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:03:01.103880 18664 ts_manager.cc:194] Re-registered known tserver with Master: 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205)
I20260812 08:03:01.104647 18664 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.67:58421
W20260812 08:03:01.115293 21336 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260812 08:03:01.115376 21336 fault_injection.cc:44] THIS SERVER MAY CRASH!
I20260812 08:03:01.116307 19620 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 5 FOLLOWER]: Refusing update from remote peer a5813459b1d94e089ffbe751f4310de7: Log matching property violated. Preceding OpId in replica: term: 2 index: 13. Preceding OpId from leader: term: 5 index: 16. (index mismatch)
I20260812 08:03:01.117131 21336 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 08:03:01.118186 19163 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 4 FOLLOWER]: Advancing to term 5
I20260812 08:03:01.119611 19163 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 5 FOLLOWER]: Refusing update from remote peer a5813459b1d94e089ffbe751f4310de7: Log matching property violated. Preceding OpId in replica: term: 4 index: 15. Preceding OpId from leader: term: 5 index: 16. (index mismatch)
W20260812 08:03:01.119752 21610 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 102ms of latency in SegmentAllocator::Sync()
I20260812 08:03:01.120191 21336 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 08:03:01.120368 21336 consensus_queue.cc:1243] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [LEADER]: Peer 5b4c3ad48b794617893c88a077be1b8d log is divergent from this leader: its last log entry 4.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
E20260812 08:03:01.121163 21336 fault_injection.cc:59] injecting fault for kudu (pid 21091): FLAGS_fault_crash_on_leader_request_fraction (process will exit)
I20260812 08:03:01.135106 21404 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Time spent bootstrapping tablet: real 0.057s	user 0.018s	sys 0.000s
I20260812 08:03:01.137657 21404 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 4 FOLLOWER]: Replica starting. Triggering 11 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:01.139359 18601 ts_itest-base.cc:575] TS 0 appears to have crashed. Restarting.
I20260812 08:03:01.139858 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.65:45661
--local_ip_for_outbound_sockets=127.18.42.65
--tserver_master_addrs=127.18.42.126:35213
--webserver_port=38137
--webserver_interface=127.18.42.65
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.05
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
I20260812 08:03:01.140312 21404 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 294e0e1970eb4e58bed426de35babb6b, State: Initialized, Role: FOLLOWER
I20260812 08:03:01.140885 21404 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 4.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:01.141739 21601 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
I20260812 08:03:01.141903 21404 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260812 08:03:01.166379 21350 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:01.166558 21350 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:01.167393 21350 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:01.168246 19619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 6 candidate_status { last_received { term: 4 index: 15 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:01.168406 20078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 6 candidate_status { last_received { term: 4 index: 15 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
W20260812 08:03:01.168442 19051 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.42.65:45661: connect: Connection refused (error 111)
I20260812 08:03:01.168558 20078 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 5.
I20260812 08:03:01.168705 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 6 candidate_status { last_received { term: 4 index: 15 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:01.168853 20633 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 5.
I20260812 08:03:01.169505 20291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 6 candidate_status { last_received { term: 4 index: 15 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:01.169708 20291 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 5.
W20260812 08:03:01.169883 19051 leader_election.cc:336] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661): Network error: Client connection negotiation failed: client connection to 127.18.42.65:45661: connect: Connection refused (error 111)
I20260812 08:03:01.170068 19051 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 285e078e6cfb46ae9dbe57c20be666d5, 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d; no voters: a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:01.170262 21350 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 08:03:01.170332 21350 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:03:01.170373 21350 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 5 FOLLOWER]: Advancing to term 6
I20260812 08:03:01.171545 21350 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:01.172209 21350 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 6 election: Requested vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:01.172695 20078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 6 candidate_status { last_received { term: 4 index: 15 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6"
I20260812 08:03:01.172832 20078 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 5 FOLLOWER]: Advancing to term 6
I20260812 08:03:01.173157 20291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 6 candidate_status { last_received { term: 4 index: 15 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf"
I20260812 08:03:01.173282 20291 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 5 FOLLOWER]: Advancing to term 6
I20260812 08:03:01.173954 20078 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 6.
I20260812 08:03:01.174369 20291 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 6.
I20260812 08:03:01.174422 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 6 candidate_status { last_received { term: 4 index: 15 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5"
I20260812 08:03:01.174548 20633 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 5 FOLLOWER]: Advancing to term 6
I20260812 08:03:01.174762 19619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 6 candidate_status { last_received { term: 4 index: 15 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec"
I20260812 08:03:01.175576 20633 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 6.
W20260812 08:03:01.175715 19051 leader_election.cc:336] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661): Network error: Client connection negotiation failed: client connection to 127.18.42.65:45661: connect: Connection refused (error 111)
I20260812 08:03:01.175882 19051 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 285e078e6cfb46ae9dbe57c20be666d5, 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d; no voters: a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:01.176087 21350 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 6 FOLLOWER]: Leader election won for term 6
I20260812 08:03:01.176713 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 6 candidate_status { last_received { term: 4 index: 15 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:01.176302 21350 raft_consensus.cc:697] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 6 LEADER]: Becoming Leader. State: Replica: 5b4c3ad48b794617893c88a077be1b8d, State: Running, Role: LEADER
I20260812 08:03:01.177017 21510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 6 candidate_status { last_received { term: 4 index: 15 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b"
I20260812 08:03:01.177253 21350 consensus_queue.cc:237] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.15, Last appended by leader: 15, Current term: 6, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:01.177044 21512 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 4.
I20260812 08:03:01.179935 18664 catalog_manager.cc:5719] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d reported cstate change: term changed from 5 to 6, leader changed from a5813459b1d94e089ffbe751f4310de7 (127.18.42.65) to 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66). New cstate: current_term: 6 leader_uuid: "5b4c3ad48b794617893c88a077be1b8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } health_report { overall_health: UNKNOWN } } }
W20260812 08:03:01.222384 19501 connection.cc:466] server connection from 127.18.42.65:32959 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.18.42.65:32959 (request call id 3) could send its response
W20260812 08:03:01.263298 21615 flags.cc:432] Enabled unsafe flag: --fault_crash_on_leader_request_fraction=0.050000000000000003
W20260812 08:03:01.263568 21615 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:03:01.263588 21615 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:03:01.263656 21615 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:03:01.263756 21615 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:01.263789 21615 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:01.263859 21615 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:01.267951 21615 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:01.268527 21615 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.65
I20260812 08:03:01.273391 21615 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.050000000000000003
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.65:45661
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data/info.pb
--webserver_interface=127.18.42.65
--webserver_port=38137
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.21615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:01.274786 21615 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:01.276052 21615 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:03:01.279376 21615 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:03:01.279448 21350 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260812 08:03:01.279506 21350 fault_injection.cc:44] THIS SERVER MAY CRASH!
I20260812 08:03:01.280632 21512 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 4 FOLLOWER]: Advancing to term 6
I20260812 08:03:01.281983 20291 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 6 FOLLOWER]: Refusing update from remote peer 5b4c3ad48b794617893c88a077be1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 6 index: 16. (index mismatch)
I20260812 08:03:01.282728 21350 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:01.283106 21512 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 6 FOLLOWER]: Refusing update from remote peer 5b4c3ad48b794617893c88a077be1b8d: Log matching property violated. Preceding OpId in replica: term: 4 index: 15. Preceding OpId from leader: term: 6 index: 16. (index mismatch)
I20260812 08:03:01.283988 21350 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 4, Time since last communication: 0.000s
W20260812 08:03:01.284301 21638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:01.284494 21631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:01.284621 21634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:01.285928 21615 server_base.cc:1061] running on GCE node
I20260812 08:03:01.286379 21615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:01.286372 19620 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 5 FOLLOWER]: Advancing to term 6
I20260812 08:03:01.287091 21615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:01.288108 19620 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 6 FOLLOWER]: Refusing update from remote peer 5b4c3ad48b794617893c88a077be1b8d: Log matching property violated. Preceding OpId in replica: term: 5 index: 16. Preceding OpId from leader: term: 6 index: 16. (term mismatch)
I20260812 08:03:01.288190 19620 pending_rounds.cc:85] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Aborting all ops after (but not including) 15
I20260812 08:03:01.288268 19620 pending_rounds.cc:107] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Aborting uncommitted NO_OP operation due to leader change: 5.16
I20260812 08:03:01.288317 19620 raft_consensus.cc:2889] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260812 08:03:01.289232 21643 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Injecting 47ms of latency in SegmentAllocator::Sync()
I20260812 08:03:01.289351 21624 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 08:03:01.289539 21624 consensus_queue.cc:1243] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [LEADER]: Peer b7def2bd3a74420d9deec19b32f541ec log is divergent from this leader: its last log entry 3.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260812 08:03:01.291720 20078 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 6 FOLLOWER]: Refusing update from remote peer 5b4c3ad48b794617893c88a077be1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 14. Preceding OpId from leader: term: 6 index: 16. (index mismatch)
W20260812 08:03:01.291872 21654 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Injecting 106ms of latency in SegmentAllocator::Sync()
I20260812 08:03:01.291990 19620 raft_consensus.cc:1217] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 6 FOLLOWER]: Deduplicated request from leader. Original: 2.4->[2.5-6.16]   Dedup: 2.14->[4.15-6.16]
W20260812 08:03:01.292119 19051 consensus_peers.cc:597] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d -> Peer a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661): Couldn't send request to peer a5813459b1d94e089ffbe751f4310de7. Status: Network error: Client connection negotiation failed: client connection to 127.18.42.65:45661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 08:03:01.292147 19620 pending_rounds.cc:85] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Aborting all ops after (but not including) 14
I20260812 08:03:01.292186 19620 pending_rounds.cc:107] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Aborting uncommitted NO_OP operation due to leader change: 3.15
I20260812 08:03:01.292215 19620 raft_consensus.cc:2889] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260812 08:03:01.292918 21610 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 3ms of latency in SegmentAllocator::Sync()
I20260812 08:03:01.293648 21658 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 4, Time since last communication: 0.000s
W20260812 08:03:01.295397 21665 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Injecting 24ms of latency in SegmentAllocator::Sync()
I20260812 08:03:01.296126 21615 hybrid_clock.cc:648] HybridClock initialized: now 1786521781295871 us; error 250 us; skew 500 ppm
I20260812 08:03:01.298825 21615 webserver.cc:533] Webserver started at http://127.18.42.65:38137/ using document root <none> and password file <none>
I20260812 08:03:01.299513 21615 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:01.299582 21615 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:01.300992 20633 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 6 FOLLOWER]: Refusing update from remote peer 5b4c3ad48b794617893c88a077be1b8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 14. Preceding OpId from leader: term: 6 index: 16. (index mismatch)
I20260812 08:03:01.301549 21658 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 08:03:01.303934 21615 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:03:01.306411 21674 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:01.307897 21615 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:03:01.308015 21615 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/wal
uuid: "a5813459b1d94e089ffbe751f4310de7"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:03:01.308508 21615 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:01.323135 21691 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48682 (ReqId={client: 8e2317b289a14646951907d5a396d316, seq_no=3, attempt_no=21}) took 1228 ms. Trace:
I20260812 08:03:01.323690 21691 rpcz_store.cc:276] 0812 08:03:00.094565 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:03:00.094736 (+   171us) service_pool.cc:224] Handling call
0812 08:03:01.323116 (+1228380us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:03:01.324306 21690 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48682 (ReqId={client: 8e2317b289a14646951907d5a396d316, seq_no=8, attempt_no=21}) took 1234 ms. Trace:
I20260812 08:03:01.324887 21690 rpcz_store.cc:276] 0812 08:03:00.089809 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:03:00.089891 (+    82us) service_pool.cc:224] Handling call
0812 08:03:01.324274 (+1234383us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:03:01.325726 21690 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48682 (ReqId={client: 8e2317b289a14646951907d5a396d316, seq_no=4, attempt_no=21}) took 1233 ms. Trace:
I20260812 08:03:01.326339 21690 rpcz_store.cc:276] 0812 08:03:00.092332 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:03:00.092467 (+   135us) service_pool.cc:224] Handling call
0812 08:03:01.325713 (+1233246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:03:01.325480 21691 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48682 (ReqId={client: 8e2317b289a14646951907d5a396d316, seq_no=9, attempt_no=21}) took 1233 ms. Trace:
I20260812 08:03:01.326932 21691 rpcz_store.cc:276] 0812 08:03:00.092071 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:03:00.092261 (+   190us) service_pool.cc:224] Handling call
0812 08:03:01.325458 (+1233197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:03:01.327544 21689 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48682 (ReqId={client: 8e2317b289a14646951907d5a396d316, seq_no=2, attempt_no=21}) took 1232 ms. Trace:
I20260812 08:03:01.327615 21608 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59600 (ReqId={client: 8e2317b289a14646951907d5a396d316, seq_no=11, attempt_no=1}) took 1254 ms. Trace:
I20260812 08:03:01.327654 21689 rpcz_store.cc:276] 0812 08:03:00.094784 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:03:00.095225 (+   441us) service_pool.cc:224] Handling call
0812 08:03:01.327530 (+1232305us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:03:01.327692 21608 rpcz_store.cc:276] 0812 08:03:00.073477 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:03:00.073598 (+   121us) service_pool.cc:224] Handling call
0812 08:03:01.327593 (+1253995us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:03:01.328213 21690 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48682 (ReqId={client: 8e2317b289a14646951907d5a396d316, seq_no=7, attempt_no=21}) took 1235 ms. Trace:
I20260812 08:03:01.328274 21690 rpcz_store.cc:276] 0812 08:03:00.092559 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:03:00.092613 (+    54us) service_pool.cc:224] Handling call
0812 08:03:01.328201 (+1235588us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:03:01.328979 21691 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48682 (ReqId={client: 8e2317b289a14646951907d5a396d316, seq_no=5, attempt_no=21}) took 1236 ms. Trace:
I20260812 08:03:01.329039 21691 rpcz_store.cc:276] 0812 08:03:00.092477 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:03:00.093381 (+   904us) service_pool.cc:224] Handling call
0812 08:03:01.328966 (+1235585us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:03:01.334161 21608 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59600 (ReqId={client: 8e2317b289a14646951907d5a396d316, seq_no=10, attempt_no=1}) took 1263 ms. Trace:
I20260812 08:03:01.334290 21608 rpcz_store.cc:276] 0812 08:03:00.070816 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:03:00.070954 (+   138us) service_pool.cc:224] Handling call
0812 08:03:01.334142 (+1263188us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260812 08:03:01.336555 20007 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.336617 19995 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.336787 19996 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.337112 19998 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.337386 19989 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.337903 19996 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.338030 19989 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.338030 20003 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.337112 19992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
E20260812 08:03:01.338788 21695 fault_injection.cc:59] injecting fault for kudu (pid 19017): FLAGS_fault_crash_on_leader_request_fraction (process will exit)
W20260812 08:03:01.339198 19998 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.344719 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.347173 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.348290 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.349306 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.350308 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.351279 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.352210 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.354066 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.354178 19550 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.354574 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:03:01.355966 21615 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:01.356884 21615 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:01.357105 21615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:01.358062 21615 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:01.359964 21730 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260812 08:03:01.361774 20373 connection.cc:573] client connection to 127.18.42.66:32855 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 08:03:01.361937 20373 meta_cache.cc:302] tablet 50b14eb091f7476d9d7682e503745c93: replica 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855) has failed: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 08:03:01.363412 20568 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
I20260812 08:03:01.364967 21615 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:01.365041 21615 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.005s	user 0.001s	sys 0.000s
I20260812 08:03:01.365142 21615 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:01.366993 21615 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:01.367080 21615 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.001s	sys 0.000s
W20260812 08:03:01.367470 20568 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
I20260812 08:03:01.367532 21730 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Bootstrap starting.
W20260812 08:03:01.367548 20570 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.368103 20576 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.368357 20565 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.368736 20576 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.369028 20570 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.369985 20576 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.370046 20565 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.370507 20570 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.370801 20206 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.373199 20206 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.374826 20208 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.374822 20206 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.377144 20206 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.378415 20210 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.378808 20208 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.379158 20206 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
I20260812 08:03:01.380421 21730 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:01.380959 21615 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.65:45661
I20260812 08:03:01.381996 21915 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.65:45661 every 8 connection(s)
I20260812 08:03:01.382236 21615 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data/info.pb
I20260812 08:03:01.387104 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 21615
I20260812 08:03:01.387255 18601 ts_itest-base.cc:575] TS 1 appears to have crashed. Restarting.
I20260812 08:03:01.387714 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.66:32855
--local_ip_for_outbound_sockets=127.18.42.66
--tserver_master_addrs=127.18.42.126:35213
--webserver_port=42027
--webserver_interface=127.18.42.66
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.05
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
I20260812 08:03:01.394838 21730 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 12 replicates
I20260812 08:03:01.395551 21730 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Bootstrap complete.
I20260812 08:03:01.395954 21730 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Time spent bootstrapping tablet: real 0.029s	user 0.017s	sys 0.005s
W20260812 08:03:01.398649 21400 connection.cc:466] server connection from 127.18.42.66:55301 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.18.42.66:55301 (request call id 3) could send its response
I20260812 08:03:01.398195 21730 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 5 FOLLOWER]: Replica starting. Triggering 12 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:01.401397 21730 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5813459b1d94e089ffbe751f4310de7, State: Initialized, Role: FOLLOWER
I20260812 08:03:01.403196 21730 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 5.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:01.404342 21730 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
W20260812 08:03:01.404488 21765 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.405229 21761 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.405844 21746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.406374 21762 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.406507 21764 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.406945 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.406911 21751 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.407267 21744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:03:01.413043 21918 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:03:01.413364 21918 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:01.414167 21918 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:03:01.416782 18674 ts_manager.cc:194] Re-registered known tserver with Master: a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661)
I20260812 08:03:01.418028 18674 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.65:44149
I20260812 08:03:01.420492 21918 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
W20260812 08:03:01.422586 21444 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.422991 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.423497 21436 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.424082 21436 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.424742 21435 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.425318 21431 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.425957 21430 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.426465 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.422586 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.427189 21444 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.436684 20206 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.436793 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.446125 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.446128 20206 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.448884 20570 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.450711 20570 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.450984 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.452078 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.458544 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.461186 21744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.462097 21744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.463207 19998 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.463270 20003 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.463477 19989 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.463207 19992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.464335 19989 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.466670 19989 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.470202 19989 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.473372 19989 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.473372 19992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.476058 20206 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.476046 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.476210 20208 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.477087 20208 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.477097 20206 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.479604 20206 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.479723 20208 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.487228 20208 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.487547 20210 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.487228 20206 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.490281 21744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.492851 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.496062 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.501485 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.501554 21744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.504041 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.504041 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.504071 21444 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.505003 21444 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.505964 21444 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.507042 21444 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.507167 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.512650 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.519495 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.520761 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.520853 20570 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.520927 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.521981 20570 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.522292 21744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.522389 21751 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.524595 21751 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.526849 20570 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.531101 20570 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.537835 20576 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.537835 20570 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.539251 20570 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.539203 21922 flags.cc:432] Enabled unsafe flag: --fault_crash_on_leader_request_fraction=0.050000000000000003
W20260812 08:03:01.539386 20206 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.539511 21922 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:03:01.539563 21922 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:03:01.539593 21922 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:03:01.539657 21922 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:01.539685 21922 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:01.539753 21922 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:01.540470 19992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.540472 20570 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.542317 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.544087 21922 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:01.544239 21922 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.66
I20260812 08:03:01.548839 21922 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.050000000000000003
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.66:32855
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/data/info.pb
--webserver_interface=127.18.42.66
--webserver_port=42027
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.21922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:01.550118 21922 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:01.551380 21922 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:01.551412 21751 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
E20260812 08:03:01.554548 21922 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:03:01.554855 21751 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.558785 21751 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.558856 21933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:01.559234 21934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:01.559234 21938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:01.560257 20576 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.560366 20570 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
I20260812 08:03:01.560461 21922 server_base.cc:1061] running on GCE node
I20260812 08:03:01.560952 21922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:01.561604 21922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:01.562786 21922 hybrid_clock.cc:648] HybridClock initialized: now 1786521781562759 us; error 42 us; skew 500 ppm
I20260812 08:03:01.564719 21922 webserver.cc:533] Webserver started at http://127.18.42.66:42027/ using document root <none> and password file <none>
I20260812 08:03:01.565389 21922 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:01.565487 21922 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
W20260812 08:03:01.565712 20570 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
I20260812 08:03:01.569166 21922 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.000s	sys 0.003s
I20260812 08:03:01.571271 21952 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:01.572531 21922 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.001s
I20260812 08:03:01.572674 21922 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/wal
uuid: "5b4c3ad48b794617893c88a077be1b8d"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:03:01.573201 21922 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260812 08:03:01.579607 19992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.579607 19989 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.579867 19998 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.579869 20003 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
I20260812 08:03:01.584789 21647 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b4c3ad48b794617893c88a077be1b8d)
I20260812 08:03:01.584935 21647 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
W20260812 08:03:01.585361 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:03:01.585618 21647 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785)
I20260812 08:03:01.586107 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:01.586755 20078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:01.587224 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
W20260812 08:03:01.588821 20170 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.42.66:32855: connect: Connection refused (error 111)
I20260812 08:03:01.589804 21922 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:01.590619 21922 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:01.590834 21922 kserver.cc:163] Server-wide thread pool size limit: 3276
W20260812 08:03:01.591250 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:03:01.591535 21922 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:01.591550 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:01.591989 20172 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf; no voters: 285e078e6cfb46ae9dbe57c20be666d5, 294e0e1970eb4e58bed426de35babb6b, 38681904c36540a4876519ec5ffac6b6, b7def2bd3a74420d9deec19b32f541ec
W20260812 08:03:01.592250 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:03:01.592556 21647 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260812 08:03:01.592844 20170 leader_election.cc:336] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855): Network error: Client connection negotiation failed: client connection to 127.18.42.66:32855: connect: Connection refused (error 111)
I20260812 08:03:01.593122 21968 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:03:01.593596 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:01.593909 21840 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e10b84812514a76ab7225dfbf9329bf in term 5.
I20260812 08:03:01.597319 21922 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:01.597402 21922 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.005s	user 0.000s	sys 0.000s
I20260812 08:03:01.597481 21922 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:01.599035 21922 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:01.599117 21922 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:03:01.599476 21968 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Bootstrap starting.
W20260812 08:03:01.599905 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.600032 20206 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.601372 20206 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.601472 20208 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.601893 20210 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.602900 20210 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
I20260812 08:03:01.608626 21968 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Log is configured to *not* fsync() on all Append() calls
W20260812 08:03:01.611605 20210 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
I20260812 08:03:01.612646 21922 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.66:32855
W20260812 08:03:01.612890 20210 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.613381 20208 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
I20260812 08:03:01.613806 22163 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.66:32855 every 8 connection(s)
I20260812 08:03:01.613888 21922 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/data/info.pb
I20260812 08:03:01.614797 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 21922
W20260812 08:03:01.620029 20208 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.620029 20210 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
I20260812 08:03:01.622740 21701 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b4c3ad48b794617893c88a077be1b8d)
I20260812 08:03:01.622866 21701 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:01.623860 21701 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:01.623989 22165 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:03:01.624228 22165 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:01.624770 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:01.624842 22165 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:03:01.625154 20291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:01.625281 20291 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 6.
I20260812 08:03:01.625782 21968 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=0 applied=16 ignored=0} inserts{seen=13 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:03:01.626231 21968 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Bootstrap complete.
I20260812 08:03:01.626554 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:01.626705 19620 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 6.
I20260812 08:03:01.627027 21712 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b4c3ad48b794617893c88a077be1b8d)
I20260812 08:03:01.627156 21712 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
W20260812 08:03:01.626394 21968 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Injecting 106ms of latency in SegmentAllocator::Sync()
I20260812 08:03:01.627308 18674 ts_manager.cc:194] Re-registered known tserver with Master: 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855)
I20260812 08:03:01.627907 21712 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:01.628408 18674 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.66:38835
I20260812 08:03:01.631055 20078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:01.631219 20078 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7def2bd3a74420d9deec19b32f541ec in term 6.
I20260812 08:03:01.631311 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:01.631496 20633 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7def2bd3a74420d9deec19b32f541ec in term 6.
I20260812 08:03:01.631686 21722 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b4c3ad48b794617893c88a077be1b8d)
I20260812 08:03:01.631795 20291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:01.631795 21722 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:01.631951 20291 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7def2bd3a74420d9deec19b32f541ec in term 6.
I20260812 08:03:01.632532 21722 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:01.633092 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:01.633255 21840 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 5.
W20260812 08:03:01.634927 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.634927 19550 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.635629 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.636133 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:03:01.636217 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:01.636353 19620 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 6.
I20260812 08:03:01.636756 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:01.637017 20078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:01.637145 20078 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 6.
I20260812 08:03:01.638140 20291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:01.638298 20291 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 6.
I20260812 08:03:01.640327 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:01.643028 19961 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 38681904c36540a4876519ec5ffac6b6, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec; no voters: 285e078e6cfb46ae9dbe57c20be666d5, 294e0e1970eb4e58bed426de35babb6b
W20260812 08:03:01.643199 19959 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 7 pre-election: Tablet error from VoteRequest() call to peer 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:01.643317 20533 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 285e078e6cfb46ae9dbe57c20be666d5, 38681904c36540a4876519ec5ffac6b6, b7def2bd3a74420d9deec19b32f541ec; no voters: 
I20260812 08:03:01.643541 21722 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 08:03:01.643628 21722 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5b4c3ad48b794617893c88a077be1b8d)
I20260812 08:03:01.643675 21722 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 6 FOLLOWER]: Advancing to term 7
I20260812 08:03:01.644681 22059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:01.645622 19501 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 285e078e6cfb46ae9dbe57c20be666d5, 38681904c36540a4876519ec5ffac6b6, b7def2bd3a74420d9deec19b32f541ec; no voters: 
W20260812 08:03:01.645659 20531 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 7 pre-election: Tablet error from VoteRequest() call to peer 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:01.646118 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:01.646507 21712 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 08:03:01.646575 21712 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5b4c3ad48b794617893c88a077be1b8d)
I20260812 08:03:01.646621 21712 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 6 FOLLOWER]: Advancing to term 7
I20260812 08:03:01.647559 21712 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:01.648376 21712 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 7 election: Requested vote from peers a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:01.648569 22059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d"
I20260812 08:03:01.648653 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
W20260812 08:03:01.648994 19500 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 7 election: Tablet error from VoteRequest() call to peer 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:01.649370 20291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf"
I20260812 08:03:01.649376 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5"
I20260812 08:03:01.649502 20291 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 6 FOLLOWER]: Advancing to term 7
I20260812 08:03:01.650887 20291 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7def2bd3a74420d9deec19b32f541ec in term 7.
I20260812 08:03:01.651911 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:01.652038 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7"
I20260812 08:03:01.652078 21840 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7def2bd3a74420d9deec19b32f541ec in term 5.
I20260812 08:03:01.652454 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:01.652585 21840 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 5.
I20260812 08:03:01.653017 21701 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 08:03:01.653119 21701 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5b4c3ad48b794617893c88a077be1b8d)
I20260812 08:03:01.653191 21701 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 6 FOLLOWER]: Advancing to term 7
I20260812 08:03:01.653360 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b"
I20260812 08:03:01.653698 20078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6"
I20260812 08:03:01.654243 21701 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:01.654438 21722 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:01.654541 20078 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b7def2bd3a74420d9deec19b32f541ec in current term 7: Already voted for candidate 38681904c36540a4876519ec5ffac6b6 in this term.
I20260812 08:03:01.654959 19501 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, b7def2bd3a74420d9deec19b32f541ec; no voters: 294e0e1970eb4e58bed426de35babb6b, 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d, a5813459b1d94e089ffbe751f4310de7
I20260812 08:03:01.655128 21701 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 7 election: Requested vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:01.655272 21722 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 7 election: Requested vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:01.655346 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec"
I20260812 08:03:01.655516 19620 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 38681904c36540a4876519ec5ffac6b6 in current term 7: Already voted for candidate b7def2bd3a74420d9deec19b32f541ec in this term.
I20260812 08:03:01.655781 20633 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b7def2bd3a74420d9deec19b32f541ec in current term 7: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:03:01.655762 19619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec"
I20260812 08:03:01.655784 22059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d"
I20260812 08:03:01.655822 21510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b"
I20260812 08:03:01.655925 19619 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 285e078e6cfb46ae9dbe57c20be666d5 in current term 7: Already voted for candidate b7def2bd3a74420d9deec19b32f541ec in this term.
I20260812 08:03:01.656250 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
W20260812 08:03:01.656536 20531 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 7 election: Tablet error from VoteRequest() call to peer 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855): Illegal state: must be running to vote when last-logged opid is not known
W20260812 08:03:01.656635 19500 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 7 pre-election: Tablet error from VoteRequest() call to peer 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:01.656939 20291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf"
I20260812 08:03:01.657107 20291 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 285e078e6cfb46ae9dbe57c20be666d5 in current term 7: Already voted for candidate b7def2bd3a74420d9deec19b32f541ec in this term.
I20260812 08:03:01.657076 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b"
I20260812 08:03:01.657656 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7"
I20260812 08:03:01.657702 20291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf"
I20260812 08:03:01.657799 21840 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 5 FOLLOWER]: Advancing to term 7
I20260812 08:03:01.657846 20291 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 38681904c36540a4876519ec5ffac6b6 in current term 7: Already voted for candidate b7def2bd3a74420d9deec19b32f541ec in this term.
I20260812 08:03:01.657956 20536 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 285e078e6cfb46ae9dbe57c20be666d5; no voters: 0e10b84812514a76ab7225dfbf9329bf, 294e0e1970eb4e58bed426de35babb6b, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:01.657949 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7"
I20260812 08:03:01.658257 21722 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260812 08:03:01.658295 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5"
I20260812 08:03:01.658447 20633 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 38681904c36540a4876519ec5ffac6b6 in current term 7: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:03:01.658452 20078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6"
I20260812 08:03:01.658591 20078 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 285e078e6cfb46ae9dbe57c20be666d5 in current term 7: Already voted for candidate 38681904c36540a4876519ec5ffac6b6 in this term.
I20260812 08:03:01.659016 19961 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 38681904c36540a4876519ec5ffac6b6; no voters: 0e10b84812514a76ab7225dfbf9329bf, 285e078e6cfb46ae9dbe57c20be666d5, 294e0e1970eb4e58bed426de35babb6b, a5813459b1d94e089ffbe751f4310de7
I20260812 08:03:01.659216 21701 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260812 08:03:01.659235 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d"
I20260812 08:03:01.659528 21712 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260812 08:03:01.659596 19959 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 7 election: Tablet error from VoteRequest() call to peer 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:01.660140 21840 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 7.
W20260812 08:03:01.675760 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.675765 21751 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.676705 21751 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.677987 21751 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.679342 21751 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.679457 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.679457 21744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.682632 21744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.687042 21744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.687084 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:03:01.690481 21613 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b4c3ad48b794617893c88a077be1b8d)
I20260812 08:03:01.690615 21613 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:01.692085 21613 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:01.697628 19619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:01.697695 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:01.697729 20078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:01.697829 19619 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 294e0e1970eb4e58bed426de35babb6b in current term 7: Already voted for candidate b7def2bd3a74420d9deec19b32f541ec in this term.
I20260812 08:03:01.697882 20078 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 294e0e1970eb4e58bed426de35babb6b in current term 7: Already voted for candidate 38681904c36540a4876519ec5ffac6b6 in this term.
W20260812 08:03:01.698189 21398 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 7 pre-election: Tablet error from VoteRequest() call to peer 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855): Illegal state: must be running to vote when last-logged opid is not known
W20260812 08:03:01.698589 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
I20260812 08:03:01.700625 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:01.700829 21840 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 294e0e1970eb4e58bed426de35babb6b in current term 7: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:03:01.701277 21400 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 294e0e1970eb4e58bed426de35babb6b; no voters: 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:01.701311 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
W20260812 08:03:01.701356 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
I20260812 08:03:01.701471 20633 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 294e0e1970eb4e58bed426de35babb6b in current term 7: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:03:01.701563 21613 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 6 FOLLOWER]: Advancing to term 7
W20260812 08:03:01.702757 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
I20260812 08:03:01.702801 20291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:01.702755 21613 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 08:03:01.702958 20291 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 294e0e1970eb4e58bed426de35babb6b in current term 7: Already voted for candidate b7def2bd3a74420d9deec19b32f541ec in this term.
W20260812 08:03:01.704102 21444 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.704071 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.704077 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.705045 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.705235 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.711953 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.712013 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.723889 20570 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.728154 20570 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.729504 20570 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.729504 20576 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.731032 20576 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.731304 20003 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.731344 19989 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.731887 19998 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
I20260812 08:03:01.733749 21968 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Time spent bootstrapping tablet: real 0.135s	user 0.022s	sys 0.004s
I20260812 08:03:01.735682 21968 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 6 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:01.736166 21968 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b4c3ad48b794617893c88a077be1b8d, State: Initialized, Role: FOLLOWER
W20260812 08:03:01.736490 20576 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
I20260812 08:03:01.736851 21968 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 6.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:01.737779 22165 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
I20260812 08:03:01.737974 21968 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
W20260812 08:03:01.739522 21996 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:01.748678 19998 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.754832 19998 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.755920 19998 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.755992 19989 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.756506 21996 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:01.757189 20576 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.758469 20576 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.759905 21996 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:01.765621 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.768781 21996 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:01.776957 21996 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:01.783783 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.785081 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.785118 21996 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:01.785313 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.787437 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.787506 21996 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:01.788507 21996 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:01.795061 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.796631 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.807293 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.812417 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.812716 20210 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.813336 20208 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.815644 21996 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:01.815717 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.815876 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.816805 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.823580 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.826948 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.834050 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.843750 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.844404 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.844926 21996 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:01.844942 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.845696 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.846319 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.848704 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.850975 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.856727 20576 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.862843 20576 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.875267 20570 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.875258 20565 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.875258 20576 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.876221 20576 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.876454 20565 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.876603 20570 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.879565 20570 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.879565 20565 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.888167 19989 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.896446 19989 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.905897 19998 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.905931 19989 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.905931 20003 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.907935 20003 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.907940 19989 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.910329 20003 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.911612 20003 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.914019 20003 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48682: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.921758 20208 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.929119 20208 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.937912 20208 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.938702 20210 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.940856 20210 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.941119 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.942065 20210 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.942253 20208 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.943284 20208 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.945811 20208 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:01.957093 21996 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:01.960481 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.970875 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.974511 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.974726 21996 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:01.974956 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:01.976243 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:01.977293 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.977495 21996 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:01.977821 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:01.992676 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:01.997001 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
I20260812 08:03:02.000578 22212 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:02.000765 22212 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:02.002326 22212 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
W20260812 08:03:02.007167 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.007824 21744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:03:02.007889 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 8 candidate_status { last_received { term: 5 index: 16 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:02.008173 22069 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5813459b1d94e089ffbe751f4310de7 for term 8 because replica has last-logged OpId of term: 6 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 16.
I20260812 08:03:02.009421 20291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 8 candidate_status { last_received { term: 5 index: 16 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:02.009639 20291 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5813459b1d94e089ffbe751f4310de7 for term 8 because replica has last-logged OpId of term: 6 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 16.
I20260812 08:03:02.011551 19619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 8 candidate_status { last_received { term: 5 index: 16 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:02.011641 20078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 8 candidate_status { last_received { term: 5 index: 16 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
W20260812 08:03:02.011710 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:03:02.011868 20078 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5813459b1d94e089ffbe751f4310de7 for term 8 because replica has last-logged OpId of term: 6 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 16.
I20260812 08:03:02.011724 19619 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5813459b1d94e089ffbe751f4310de7 for term 8 because replica has last-logged OpId of term: 6 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 16.
W20260812 08:03:02.013258 21744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.013607 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
I20260812 08:03:02.013774 21679 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: a5813459b1d94e089ffbe751f4310de7; no voters: 0e10b84812514a76ab7225dfbf9329bf, 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:02.013837 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 8 candidate_status { last_received { term: 5 index: 16 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:02.014024 21512 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5813459b1d94e089ffbe751f4310de7 for term 8 because replica has last-logged OpId of term: 6 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 16.
I20260812 08:03:02.014024 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 8 candidate_status { last_received { term: 5 index: 16 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:02.014083 22212 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 08:03:02.014201 20633 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5813459b1d94e089ffbe751f4310de7 for term 8 because replica has last-logged OpId of term: 6 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 16.
W20260812 08:03:02.015082 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:02.015084 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:02.015106 21996 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:02.029997 21744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.031514 21744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.042382 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:02.043540 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
I20260812 08:03:02.044032 21701 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:02.044149 21701 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:02.044838 21701 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:02.045176 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 8 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:02.045236 20291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 8 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:02.045277 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 8 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:02.045385 20291 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 7.
I20260812 08:03:02.045404 21840 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 7.
I20260812 08:03:02.045431 22069 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 6.
I20260812 08:03:02.045679 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 8 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:02.045821 21512 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 7.
I20260812 08:03:02.045819 19619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 8 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:02.045975 19619 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 7.
I20260812 08:03:02.046108 19961 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d, a5813459b1d94e089ffbe751f4310de7; no voters: 
I20260812 08:03:02.046334 21701 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 08:03:02.046422 21701 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:03:02.046376 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 8 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:02.046483 21701 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 7 FOLLOWER]: Advancing to term 8
I20260812 08:03:02.046561 20633 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 7.
I20260812 08:03:02.047964 21701 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:02.048748 21701 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 8 election: Requested vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:02.048992 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 8 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7"
I20260812 08:03:02.049026 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 8 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d"
I20260812 08:03:02.049142 21840 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 7 FOLLOWER]: Advancing to term 8
I20260812 08:03:02.049180 22069 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 6 FOLLOWER]: Advancing to term 8
I20260812 08:03:02.049458 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 8 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b"
I20260812 08:03:02.049614 21512 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 7 FOLLOWER]: Advancing to term 8
I20260812 08:03:02.049701 19619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 8 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec"
I20260812 08:03:02.049819 19619 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 7 FOLLOWER]: Advancing to term 8
I20260812 08:03:02.050345 21840 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 8.
I20260812 08:03:02.050748 21512 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 8.
I20260812 08:03:02.050961 19619 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 8.
I20260812 08:03:02.051326 20291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 8 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf"
I20260812 08:03:02.051468 20291 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 7 FOLLOWER]: Advancing to term 8
I20260812 08:03:02.051492 19960 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 294e0e1970eb4e58bed426de35babb6b, 38681904c36540a4876519ec5ffac6b6, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec; no voters: 
I20260812 08:03:02.051621 22069 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 8.
I20260812 08:03:02.051690 21701 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 8 FOLLOWER]: Leader election won for term 8
I20260812 08:03:02.051891 21701 raft_consensus.cc:697] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 8 LEADER]: Becoming Leader. State: Replica: 38681904c36540a4876519ec5ffac6b6, State: Running, Role: LEADER
W20260812 08:03:02.051929 21744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:03:02.052240 21701 consensus_queue.cc:237] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 6.16, Last appended by leader: 16, Current term: 8, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
W20260812 08:03:02.052444 21996 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
I20260812 08:03:02.052521 20291 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 8.
I20260812 08:03:02.053094 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 8 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5"
I20260812 08:03:02.053243 20633 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 7 FOLLOWER]: Advancing to term 8
W20260812 08:03:02.054127 21744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:03:02.054313 20633 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 8.
W20260812 08:03:02.054340 21751 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.054795 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.054853 21764 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.055243 21665 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Injecting 20ms of latency in SegmentAllocator::Sync()
I20260812 08:03:02.056246 18661 catalog_manager.cc:5719] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 reported cstate change: term changed from 6 to 8, leader changed from 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66) to 38681904c36540a4876519ec5ffac6b6 (127.18.42.70). New cstate: current_term: 8 leader_uuid: "38681904c36540a4876519ec5ffac6b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } health_report { overall_health: UNKNOWN } } }
W20260812 08:03:02.064000 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:02.069087 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:02.077988 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:02.083045 21665 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Injecting 112ms of latency in SegmentAllocator::Sync()
W20260812 08:03:02.083082 21701 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260812 08:03:02.083192 21701 fault_injection.cc:44] THIS SERVER MAY CRASH!
I20260812 08:03:02.084056 21512 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 8 FOLLOWER]: Refusing update from remote peer 38681904c36540a4876519ec5ffac6b6: Log matching property violated. Preceding OpId in replica: term: 6 index: 16. Preceding OpId from leader: term: 8 index: 18. (index mismatch)
I20260812 08:03:02.084398 20633 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 8 FOLLOWER]: Refusing update from remote peer 38681904c36540a4876519ec5ffac6b6: Log matching property violated. Preceding OpId in replica: term: 6 index: 16. Preceding OpId from leader: term: 8 index: 18. (index mismatch)
I20260812 08:03:02.084502 22069 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 8 FOLLOWER]: Refusing update from remote peer 38681904c36540a4876519ec5ffac6b6: Log matching property violated. Preceding OpId in replica: term: 6 index: 16. Preceding OpId from leader: term: 8 index: 18. (index mismatch)
I20260812 08:03:02.084563 20291 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 8 FOLLOWER]: Refusing update from remote peer 38681904c36540a4876519ec5ffac6b6: Log matching property violated. Preceding OpId in replica: term: 6 index: 16. Preceding OpId from leader: term: 8 index: 18. (index mismatch)
I20260812 08:03:02.084910 22239 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 08:03:02.085158 22223 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260812 08:03:02.085203 21840 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 8 FOLLOWER]: Refusing update from remote peer 38681904c36540a4876519ec5ffac6b6: Log matching property violated. Preceding OpId in replica: term: 5 index: 16. Preceding OpId from leader: term: 8 index: 18. (index mismatch)
E20260812 08:03:02.085345 22223 fault_injection.cc:59] injecting fault for kudu (pid 19932): FLAGS_fault_crash_on_leader_request_fraction (process will exit)
I20260812 08:03:02.085505 19619 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 8 FOLLOWER]: Refusing update from remote peer 38681904c36540a4876519ec5ffac6b6: Log matching property violated. Preceding OpId in replica: term: 6 index: 16. Preceding OpId from leader: term: 8 index: 18. (index mismatch)
W20260812 08:03:02.109838 19501 connection.cc:573] server connection from 127.18.42.70:58515 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 08:03:02.110535 20377 connection.cc:573] client connection to 127.18.42.70:39785 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 08:03:02.110548 21680 connection.cc:573] server connection from 127.18.42.70:44991 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 08:03:02.112835 18601 ts_itest-base.cc:575] TS 5 appears to have crashed. Restarting.
I20260812 08:03:02.113341 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.70:39785
--local_ip_for_outbound_sockets=127.18.42.70
--tserver_master_addrs=127.18.42.126:35213
--webserver_port=40739
--webserver_interface=127.18.42.70
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.05
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
W20260812 08:03:02.113930 20570 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.146664 20576 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.146664 20565 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.146667 20570 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.147709 20570 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.147765 20565 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.148947 20565 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.150239 20565 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.150341 20576 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.150782 20570 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48926: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.153108 20208 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:02.184827 20208 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:02.185925 20208 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:02.188244 20208 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:02.188613 20206 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:02.189182 20196 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:02.189467 20210 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:02.189687 20206 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:02.190289 20206 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:02.190547 20210 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48214: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:02.223160 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:02.225226 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:02.228420 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:02.228574 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:02.230613 22241 flags.cc:432] Enabled unsafe flag: --fault_crash_on_leader_request_fraction=0.050000000000000003
W20260812 08:03:02.230822 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:02.230958 22241 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:03:02.231019 22241 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:03:02.231062 22241 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:03:02.231132 22241 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:02.231182 22241 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:02.231204 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:02.231204 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:02.231268 22241 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:02.232252 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:02.235246 22241 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:02.235405 22241 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.70
W20260812 08:03:02.236445 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:03:02.240185 22241 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.050000000000000003
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.70:39785
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data/info.pb
--webserver_interface=127.18.42.70
--webserver_port=40739
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.22241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.70

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:02.241358 22241 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:02.242678 22241 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:03:02.245566 22241 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:03:02.249492 22250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:02.249507 22249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:02.249486 22254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:02.249923 22241 server_base.cc:1061] running on GCE node
I20260812 08:03:02.250304 22241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:02.250939 22241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:02.252151 22241 hybrid_clock.cc:648] HybridClock initialized: now 1786521782252123 us; error 44 us; skew 500 ppm
I20260812 08:03:02.254361 22241 webserver.cc:533] Webserver started at http://127.18.42.70:40739/ using document root <none> and password file <none>
I20260812 08:03:02.254984 22241 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:02.255100 22241 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:02.258762 22241 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.001s	sys 0.003s
I20260812 08:03:02.261145 22264 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:02.262447 22241 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260812 08:03:02.262637 22241 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/wal
uuid: "38681904c36540a4876519ec5ffac6b6"
format_stamp: "Formatted at 2026-08-12 08:02:59 on dist-test-slave-zr1t"
I20260812 08:03:02.263118 22241 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260812 08:03:02.264235 21764 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.267400 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:02.271937 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:02.275139 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:02.280508 21764 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:03:02.291497 22241 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:02.292294 22241 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:02.292532 22241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:02.293301 22241 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:02.294734 22274 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:03:02.298375 22241 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:02.298439 22241 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.001s	sys 0.000s
I20260812 08:03:02.298537 22241 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:02.300076 22241 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:02.300122 22241 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:03:02.300495 22274 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Bootstrap starting.
W20260812 08:03:02.303328 21996 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
I20260812 08:03:02.307652 22274 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Log is configured to *not* fsync() on all Append() calls
W20260812 08:03:02.310491 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:02.310492 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:03:02.314170 22241 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.70:39785
I20260812 08:03:02.314231 22460 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.70:39785 every 8 connection(s)
W20260812 08:03:02.314524 21764 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.314713 21751 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.315171 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:03:02.315542 22241 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data/info.pb
W20260812 08:03:02.316072 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:03:02.322801 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 22241
I20260812 08:03:02.325575 22274 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Bootstrap replayed 1/1 log segments. Stats: ops{read=18 overwritten=0 applied=16 ignored=0} inserts{seen=13 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 2 replicates
I20260812 08:03:02.325726 22461 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:03:02.325969 22461 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:02.326102 22274 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Bootstrap complete.
I20260812 08:03:02.326471 22274 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Time spent bootstrapping tablet: real 0.026s	user 0.020s	sys 0.003s
I20260812 08:03:02.326587 22461 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:03:02.328895 18661 ts_manager.cc:194] Re-registered known tserver with Master: 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785)
I20260812 08:03:02.328845 22274 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 8 FOLLOWER]: Replica starting. Triggering 2 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:02.329790 18661 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.70:54847
I20260812 08:03:02.330533 22274 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 8 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38681904c36540a4876519ec5ffac6b6, State: Initialized, Role: FOLLOWER
I20260812 08:03:02.331048 22274 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 8.18, Last appended by leader: 18, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:02.331822 22461 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
I20260812 08:03:02.331959 22274 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
W20260812 08:03:02.353820 22310 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.353820 22307 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.354110 22309 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.356925 22309 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.357013 22307 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.357013 22310 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.358042 22307 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.371733 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:02.371726 21996 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:02.397400 22307 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.398833 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.398903 21751 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.398941 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:03:02.399281 22470 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:02.399477 22470 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
W20260812 08:03:02.400288 21996 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
I20260812 08:03:02.400537 19619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:02.400687 19619 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e10b84812514a76ab7225dfbf9329bf in term 8.
I20260812 08:03:02.400928 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:02.401057 21840 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e10b84812514a76ab7225dfbf9329bf in term 8.
I20260812 08:03:02.401206 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:02.401206 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:02.401346 21512 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e10b84812514a76ab7225dfbf9329bf in term 8.
I20260812 08:03:02.401346 20633 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e10b84812514a76ab7225dfbf9329bf in term 8.
W20260812 08:03:02.401770 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
I20260812 08:03:02.401772 20172 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 285e078e6cfb46ae9dbe57c20be666d5, 294e0e1970eb4e58bed426de35babb6b, a5813459b1d94e089ffbe751f4310de7; no voters: 
I20260812 08:03:02.402005 22470 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785)
I20260812 08:03:02.402107 22470 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260812 08:03:02.402158 22470 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:03:02.402204 22470 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 8 FOLLOWER]: Advancing to term 9
W20260812 08:03:02.402971 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
I20260812 08:03:02.403668 22470 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:02.404528 19619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec"
I20260812 08:03:02.404572 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7"
I20260812 08:03:02.404662 19619 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 8 FOLLOWER]: Advancing to term 9
I20260812 08:03:02.404683 21840 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 8 FOLLOWER]: Advancing to term 9
I20260812 08:03:02.405774 19619 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e10b84812514a76ab7225dfbf9329bf in term 9.
I20260812 08:03:02.405774 21840 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e10b84812514a76ab7225dfbf9329bf in term 9.
I20260812 08:03:02.405962 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:02.406107 22069 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e10b84812514a76ab7225dfbf9329bf in term 8.
I20260812 08:03:02.406550 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d"
I20260812 08:03:02.406654 22069 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 8 FOLLOWER]: Advancing to term 9
I20260812 08:03:02.406731 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5"
I20260812 08:03:02.406728 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b"
I20260812 08:03:02.406855 20633 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 8 FOLLOWER]: Advancing to term 9
I20260812 08:03:02.406854 21512 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 8 FOLLOWER]: Advancing to term 9
I20260812 08:03:02.407438 22470 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 9 election: Requested vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785)
I20260812 08:03:02.407791 22069 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e10b84812514a76ab7225dfbf9329bf in term 9.
I20260812 08:03:02.407840 20633 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e10b84812514a76ab7225dfbf9329bf in term 9.
I20260812 08:03:02.407837 21512 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e10b84812514a76ab7225dfbf9329bf in term 9.
I20260812 08:03:02.408130 20170 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 5b4c3ad48b794617893c88a077be1b8d, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec; no voters: 
I20260812 08:03:02.408363 22470 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 9 FOLLOWER]: Leader election won for term 9
I20260812 08:03:02.408476 22470 raft_consensus.cc:697] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 9 LEADER]: Becoming Leader. State: Replica: 0e10b84812514a76ab7225dfbf9329bf, State: Running, Role: LEADER
I20260812 08:03:02.408509 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6"
I20260812 08:03:02.408486 22385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:02.408735 22384 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 8 FOLLOWER]: Advancing to term 9
I20260812 08:03:02.408818 22470 consensus_queue.cc:237] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.16, Last appended by leader: 16, Current term: 9, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:02.410739 22384 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e10b84812514a76ab7225dfbf9329bf for term 9 because replica has last-logged OpId of term: 8 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 16.
W20260812 08:03:02.411221 22478 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Injecting 99ms of latency in SegmentAllocator::Sync()
I20260812 08:03:02.412050 18661 catalog_manager.cc:5719] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf reported cstate change: term changed from 8 to 9, leader changed from 38681904c36540a4876519ec5ffac6b6 (127.18.42.70) to 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71). New cstate: current_term: 9 leader_uuid: "0e10b84812514a76ab7225dfbf9329bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } health_report { overall_health: HEALTHY } } }
W20260812 08:03:02.415458 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:02.415491 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:02.441929 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:02.442930 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:02.443776 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:02.444011 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:02.446949 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:02.447368 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:02.451084 22476 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260812 08:03:02.451187 22476 fault_injection.cc:44] THIS SERVER MAY CRASH!
I20260812 08:03:02.452077 20633 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 9 FOLLOWER]: Refusing update from remote peer 0e10b84812514a76ab7225dfbf9329bf: Log matching property violated. Preceding OpId in replica: term: 6 index: 16. Preceding OpId from leader: term: 9 index: 18. (index mismatch)
I20260812 08:03:02.452176 22069 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 9 FOLLOWER]: Refusing update from remote peer 0e10b84812514a76ab7225dfbf9329bf: Log matching property violated. Preceding OpId in replica: term: 6 index: 16. Preceding OpId from leader: term: 9 index: 18. (index mismatch)
I20260812 08:03:02.452167 21840 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 9 FOLLOWER]: Refusing update from remote peer 0e10b84812514a76ab7225dfbf9329bf: Log matching property violated. Preceding OpId in replica: term: 5 index: 16. Preceding OpId from leader: term: 9 index: 18. (index mismatch)
I20260812 08:03:02.452446 21512 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 9 FOLLOWER]: Refusing update from remote peer 0e10b84812514a76ab7225dfbf9329bf: Log matching property violated. Preceding OpId in replica: term: 6 index: 16. Preceding OpId from leader: term: 9 index: 18. (index mismatch)
I20260812 08:03:02.452854 19619 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 9 FOLLOWER]: Refusing update from remote peer 0e10b84812514a76ab7225dfbf9329bf: Log matching property violated. Preceding OpId in replica: term: 6 index: 16. Preceding OpId from leader: term: 9 index: 18. (index mismatch)
I20260812 08:03:02.452801 22485 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260812 08:03:02.452957 22482 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 08:03:02.453032 22482 consensus_queue.cc:1243] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [LEADER]: Peer a5813459b1d94e089ffbe751f4310de7 log is divergent from this leader: its last log entry 5.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260812 08:03:02.453158 22384 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 9 FOLLOWER]: Refusing update from remote peer 0e10b84812514a76ab7225dfbf9329bf: Log matching property violated. Preceding OpId in replica: term: 8 index: 18. Preceding OpId from leader: term: 9 index: 18. (term mismatch)
I20260812 08:03:02.453225 22384 pending_rounds.cc:85] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Aborting all ops after (but not including) 17
I20260812 08:03:02.453161 22470 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 08:03:02.453298 22384 pending_rounds.cc:107] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Aborting uncommitted WRITE_OP operation due to leader change: 8.18
I20260812 08:03:02.453328 22486 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260812 08:03:02.454022 22470 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260812 08:03:02.454319 21840 raft_consensus.cc:1217] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 9 FOLLOWER]: Deduplicated request from leader. Original: 2.4->[2.5-9.18]   Dedup: 2.14->[4.15-9.18]
I20260812 08:03:02.454487 21840 pending_rounds.cc:85] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Aborting all ops after (but not including) 14
I20260812 08:03:02.454547 21840 pending_rounds.cc:107] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Aborting uncommitted NO_OP operation due to leader change: 3.15
I20260812 08:03:02.454604 21840 raft_consensus.cc:2889] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260812 08:03:02.454680 21840 pending_rounds.cc:107] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Aborting uncommitted NO_OP operation due to leader change: 5.16
I20260812 08:03:02.454706 21840 raft_consensus.cc:2889] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260812 08:03:02.455909 22486 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260812 08:03:02.456005 22486 consensus_queue.cc:1243] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [LEADER]: Peer 38681904c36540a4876519ec5ffac6b6 log is divergent from this leader: its last log entry 8.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
W20260812 08:03:02.456228 22503 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Injecting 25ms of latency in SegmentAllocator::Sync()
I20260812 08:03:02.456833 22384 pending_rounds.cc:85] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Aborting all ops after (but not including) 16
I20260812 08:03:02.456887 22384 pending_rounds.cc:107] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Aborting uncommitted NO_OP operation due to leader change: 8.17
I20260812 08:03:02.456923 22384 raft_consensus.cc:2889] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260812 08:03:02.458254 22499 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 154ms of latency in SegmentAllocator::Sync()
W20260812 08:03:02.461833 22504 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 21ms of latency in SegmentAllocator::Sync()
W20260812 08:03:02.466221 22498 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 129ms of latency in SegmentAllocator::Sync()
W20260812 08:03:02.466701 22508 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Injecting 62ms of latency in SegmentAllocator::Sync()
W20260812 08:03:02.466955 22511 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Injecting 124ms of latency in SegmentAllocator::Sync()
I20260812 08:03:02.484259 22481 mvcc.cc:204] Tried to move back new op lower bound from 7317593220916731904 to 7317593214098055168. Current Snapshot: MvccSnapshot[applied={T|T < 7317593210645311488}]
W20260812 08:03:02.484946 22503 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Injecting 80ms of latency in SegmentAllocator::Sync()
I20260812 08:03:02.488838 22497 mvcc.cc:204] Tried to move back new op lower bound from 7317593220916731904 to 7317593214098055168. Current Snapshot: MvccSnapshot[applied={T|T < 7317593210638352384}]
W20260812 08:03:02.489982 22504 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 37ms of latency in SegmentAllocator::Sync()
W20260812 08:03:02.499586 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:02.511494 22478 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Injecting 121ms of latency in SegmentAllocator::Sync()
W20260812 08:03:02.530644 22504 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 85ms of latency in SegmentAllocator::Sync()
W20260812 08:03:02.545636 22307 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.568687 22503 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Injecting 130ms of latency in SegmentAllocator::Sync()
W20260812 08:03:02.568956 22307 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.571276 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:02.574646 21751 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.581037 21996 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:02.586879 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:02.595105 22508 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Injecting 95ms of latency in SegmentAllocator::Sync()
W20260812 08:03:02.596302 22511 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Injecting 112ms of latency in SegmentAllocator::Sync()
I20260812 08:03:02.596398 22510 mvcc.cc:204] Tried to move back new op lower bound from 7317593220956483584 to 7317593214098055168. Current Snapshot: MvccSnapshot[applied={T|T < 7317593220956483584 or (T in {7317593220956483584})}]
I20260812 08:03:02.598650 22495 mvcc.cc:204] Tried to move back new op lower bound from 7317593220916731904 to 7317593220751626240. Current Snapshot: MvccSnapshot[applied={T|T < 7317593220916731904}]
W20260812 08:03:02.600581 22498 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 104ms of latency in SegmentAllocator::Sync()
W20260812 08:03:02.635448 22478 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Injecting 74ms of latency in SegmentAllocator::Sync()
W20260812 08:03:02.642838 22504 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 43ms of latency in SegmentAllocator::Sync()
W20260812 08:03:02.644429 22499 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 58ms of latency in SegmentAllocator::Sync()
W20260812 08:03:02.688377 22504 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 67ms of latency in SegmentAllocator::Sync()
W20260812 08:03:02.694336 22508 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Injecting 55ms of latency in SegmentAllocator::Sync()
W20260812 08:03:02.703328 22503 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Injecting 1ms of latency in SegmentAllocator::Sync()
W20260812 08:03:02.710075 22498 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 17ms of latency in SegmentAllocator::Sync()
W20260812 08:03:02.710610 22478 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Injecting 39ms of latency in SegmentAllocator::Sync()
W20260812 08:03:02.713567 22503 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Injecting 17ms of latency in SegmentAllocator::Sync()
W20260812 08:03:02.714661 22499 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 52ms of latency in SegmentAllocator::Sync()
W20260812 08:03:02.722090 22511 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Injecting 74ms of latency in SegmentAllocator::Sync()
W20260812 08:03:02.734649 22498 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 33ms of latency in SegmentAllocator::Sync()
W20260812 08:03:02.736326 22503 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Injecting 64ms of latency in SegmentAllocator::Sync()
W20260812 08:03:02.751271 22478 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Injecting 35ms of latency in SegmentAllocator::Sync()
W20260812 08:03:02.754130 22508 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Injecting 60ms of latency in SegmentAllocator::Sync()
E20260812 08:03:02.756542 22569 fault_injection.cc:59] injecting fault for kudu (pid 20151): FLAGS_fault_crash_on_leader_request_fraction (process will exit)
W20260812 08:03:02.775564 20531 connection.cc:573] server connection from 127.18.42.71:50323 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 08:03:02.776324 20378 meta_cache.cc:302] tablet 50b14eb091f7476d9d7682e503745c93: replica 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647) has failed: Network error: recv got EOF from 127.18.42.71:35647 (error 108)
W20260812 08:03:02.776351 19502 connection.cc:573] server connection from 127.18.42.71:52063 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 08:03:02.776419 18601 ts_itest-base.cc:575] TS 6 appears to have crashed. Restarting.
I20260812 08:03:02.776842 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.71:35647
--local_ip_for_outbound_sockets=127.18.42.71
--tserver_master_addrs=127.18.42.126:35213
--webserver_port=37531
--webserver_interface=127.18.42.71
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.05
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
W20260812 08:03:02.780869 19548 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:02.780915 19527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:02.781270 19529 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:02.781553 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:02.780946 19550 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:02.782141 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:02.783368 19550 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:02.783384 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:02.784926 21751 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.786017 21751 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.787227 21751 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.788347 21751 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.788388 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:02.796697 21397 connection.cc:466] server connection from 127.18.42.71:58061 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.18.42.71:58061 (request call id 9) could send its response
W20260812 08:03:02.800957 21953 connection.cc:466] server connection from 127.18.42.71:49221 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.18.42.71:49221 (request call id 10) could send its response
W20260812 08:03:02.814903 22266 connection.cc:466] server connection from 127.18.42.71:52617 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.18.42.71:52617 (request call id 9) could send its response
W20260812 08:03:02.891822 22571 flags.cc:432] Enabled unsafe flag: --fault_crash_on_leader_request_fraction=0.050000000000000003
W20260812 08:03:02.892052 22571 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:03:02.892086 22571 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:03:02.892141 22571 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:03:02.892215 22571 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:02.892263 22571 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:02.892344 22571 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:02.895807 22571 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:02.895963 22571 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.71
I20260812 08:03:02.899942 22571 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.050000000000000003
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.71:35647
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data/info.pb
--webserver_interface=127.18.42.71
--webserver_port=37531
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.22571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.71

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:02.901060 22571 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:02.902328 22571 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:03:02.905014 22571 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:03:02.908937 22587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:02.908921 22585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:02.909024 22571 server_base.cc:1061] running on GCE node
W20260812 08:03:02.908912 22592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:02.909644 22571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:02.910247 22571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:02.911440 22571 hybrid_clock.cc:648] HybridClock initialized: now 1786521782911428 us; error 30 us; skew 500 ppm
I20260812 08:03:02.913285 22571 webserver.cc:533] Webserver started at http://127.18.42.71:37531/ using document root <none> and password file <none>
I20260812 08:03:02.913930 22571 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:02.914067 22571 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:02.917483 22571 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:03:02.919567 22601 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:02.920997 22571 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260812 08:03:02.921120 22571 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/wal
uuid: "0e10b84812514a76ab7225dfbf9329bf"
format_stamp: "Formatted at 2026-08-12 08:02:59 on dist-test-slave-zr1t"
I20260812 08:03:02.921571 22571 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:02.934092 22571 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:02.934765 22571 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:02.934979 22571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:02.935572 22571 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:02.936853 22611 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:03:02.940434 22571 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:02.940496 22571 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:03:02.940568 22571 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:02.942099 22571 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:02.942144 22571 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:03:02.942601 22611 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Bootstrap starting.
I20260812 08:03:02.953611 22611 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:02.953831 22571 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.71:35647
I20260812 08:03:02.953855 22787 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.71:35647 every 8 connection(s)
I20260812 08:03:02.954875 22571 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data/info.pb
I20260812 08:03:02.955124 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 22571
I20260812 08:03:02.967121 22791 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:03:02.967386 22791 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:02.967926 22791 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:03:02.969748 18674 ts_manager.cc:194] Re-registered known tserver with Master: 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:02.970536 18674 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.71:44583
I20260812 08:03:02.980870 22611 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Bootstrap replayed 1/1 log segments. Stats: ops{read=47 overwritten=0 applied=37 ignored=0} inserts{seen=33 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 10 replicates
I20260812 08:03:02.981509 22611 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Bootstrap complete.
W20260812 08:03:02.981760 22611 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Injecting 133ms of latency in SegmentAllocator::Sync()
I20260812 08:03:02.988462 22534 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e10b84812514a76ab7225dfbf9329bf)
I20260812 08:03:02.988595 22534 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:02.989374 22534 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:02.989959 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 10 candidate_status { last_received { term: 9 index: 37 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:02.990176 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 10 candidate_status { last_received { term: 9 index: 37 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:02.990171 19619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 10 candidate_status { last_received { term: 9 index: 37 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:02.990224 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 10 candidate_status { last_received { term: 9 index: 37 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:02.990909 21680 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: a5813459b1d94e089ffbe751f4310de7; no voters: 285e078e6cfb46ae9dbe57c20be666d5, 294e0e1970eb4e58bed426de35babb6b, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:02.991109 22534 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260812 08:03:02.995394 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 10 candidate_status { last_received { term: 9 index: 37 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:02.996320 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 10 candidate_status { last_received { term: 9 index: 37 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
W20260812 08:03:02.998162 21680 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 10 pre-election: Tablet error from VoteRequest() call to peer 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:03.015033 22492 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e10b84812514a76ab7225dfbf9329bf)
I20260812 08:03:03.015254 22492 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:03.016150 22492 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:03.016794 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 39 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:03.016980 21840 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7def2bd3a74420d9deec19b32f541ec in term 9.
I20260812 08:03:03.017000 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 39 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:03.017194 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 39 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:03.017438 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 39 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:03.021013 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 39 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:03.021445 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 39 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
W20260812 08:03:03.021468 19502 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 10 pre-election: Tablet error from VoteRequest() call to peer 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:03.021606 19502 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec; no voters: 0e10b84812514a76ab7225dfbf9329bf, 285e078e6cfb46ae9dbe57c20be666d5, 294e0e1970eb4e58bed426de35babb6b, 5b4c3ad48b794617893c88a077be1b8d
I20260812 08:03:03.021775 22492 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260812 08:03:03.022010 22547 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e10b84812514a76ab7225dfbf9329bf)
I20260812 08:03:03.022116 22547 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:03.022857 22547 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:03.023025 19619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 42 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:03.023108 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 42 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:03.023170 19619 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 9.
I20260812 08:03:03.023551 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 42 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:03.023690 21840 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 9.
I20260812 08:03:03.024029 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 42 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:03.027395 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 42 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
W20260812 08:03:03.027787 21400 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 10 pre-election: Tablet error from VoteRequest() call to peer 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:03.028111 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 42 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:03.028488 21399 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 294e0e1970eb4e58bed426de35babb6b, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec; no voters: 0e10b84812514a76ab7225dfbf9329bf, 285e078e6cfb46ae9dbe57c20be666d5, 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d
I20260812 08:03:03.028705 22547 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260812 08:03:03.034816 22810 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e10b84812514a76ab7225dfbf9329bf)
I20260812 08:03:03.034840 22489 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e10b84812514a76ab7225dfbf9329bf)
I20260812 08:03:03.034938 22810 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:03.034983 22489 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:03.035720 22489 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:03.035898 19619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 10 candidate_status { last_received { term: 9 index: 47 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:03.036041 19619 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 9.
I20260812 08:03:03.036445 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 10 candidate_status { last_received { term: 9 index: 47 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:03.036585 21840 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 9.
I20260812 08:03:03.036646 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 10 candidate_status { last_received { term: 9 index: 47 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:03.036648 22810 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:03.036790 22069 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 9.
I20260812 08:03:03.037148 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 10 candidate_status { last_received { term: 9 index: 47 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:03.037289 21512 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 9.
I20260812 08:03:03.039057 20533 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 285e078e6cfb46ae9dbe57c20be666d5, 294e0e1970eb4e58bed426de35babb6b, 5b4c3ad48b794617893c88a077be1b8d, a5813459b1d94e089ffbe751f4310de7; no voters: 
I20260812 08:03:03.039271 22489 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260812 08:03:03.039335 22489 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0e10b84812514a76ab7225dfbf9329bf)
I20260812 08:03:03.039392 22489 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 9 FOLLOWER]: Advancing to term 10
I20260812 08:03:03.040632 22489 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:03.041236 22489 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 10 election: Requested vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:03.042912 19619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 10 candidate_status { last_received { term: 9 index: 47 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec"
I20260812 08:03:03.043046 19619 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 9 FOLLOWER]: Advancing to term 10
I20260812 08:03:03.044310 19619 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 10.
I20260812 08:03:03.044600 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 10 candidate_status { last_received { term: 9 index: 47 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b"
I20260812 08:03:03.044735 21512 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 9 FOLLOWER]: Advancing to term 10
I20260812 08:03:03.045979 21512 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 10.
I20260812 08:03:03.046288 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 10 candidate_status { last_received { term: 9 index: 47 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7"
I20260812 08:03:03.046427 21840 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 9 FOLLOWER]: Advancing to term 10
I20260812 08:03:03.047798 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 10 candidate_status { last_received { term: 9 index: 47 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:03.047900 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 10 candidate_status { last_received { term: 9 index: 47 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:03.048161 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 10 candidate_status { last_received { term: 9 index: 47 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf"
I20260812 08:03:03.048483 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 10 candidate_status { last_received { term: 9 index: 47 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d"
I20260812 08:03:03.048623 22069 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 9 FOLLOWER]: Advancing to term 10
I20260812 08:03:03.048718 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 10 candidate_status { last_received { term: 9 index: 47 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6"
I20260812 08:03:03.050036 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 10 candidate_status { last_received { term: 9 index: 47 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:03.050117 22069 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 10.
W20260812 08:03:03.050508 21956 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 10 pre-election: Tablet error from VoteRequest() call to peer 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:03.051258 20531 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 285e078e6cfb46ae9dbe57c20be666d5, 294e0e1970eb4e58bed426de35babb6b, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec; no voters: 38681904c36540a4876519ec5ffac6b6
W20260812 08:03:03.051569 20536 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 10 pre-election: Tablet error from VoteRequest() call to peer 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647): Illegal state: must be running to vote when last-logged opid is not known
W20260812 08:03:03.052117 20536 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 10 election: Tablet error from VoteRequest() call to peer 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:03.052356 21840 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 10.
I20260812 08:03:03.053014 19619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 10 candidate_status { last_received { term: 9 index: 47 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:03.053179 19619 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b4c3ad48b794617893c88a077be1b8d in current term 10: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:03:03.053730 22489 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 10 FOLLOWER]: Leader election won for term 10
I20260812 08:03:03.053949 22489 raft_consensus.cc:697] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 10 LEADER]: Becoming Leader. State: Replica: 285e078e6cfb46ae9dbe57c20be666d5, State: Running, Role: LEADER
I20260812 08:03:03.054292 22489 consensus_queue.cc:237] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 37, Committed index: 37, Last appended: 9.47, Last appended by leader: 47, Current term: 10, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:03.055481 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 10 candidate_status { last_received { term: 9 index: 47 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:03.055685 21512 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b4c3ad48b794617893c88a077be1b8d in current term 10: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:03:03.056937 22832 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e10b84812514a76ab7225dfbf9329bf)
I20260812 08:03:03.057134 22832 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:03.060353 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 10 candidate_status { last_received { term: 9 index: 47 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:03.060509 22384 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 9.
I20260812 08:03:03.061084 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 10 candidate_status { last_received { term: 9 index: 47 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:03.061285 21840 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b4c3ad48b794617893c88a077be1b8d in current term 10: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:03:03.061813 21956 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d; no voters: 0e10b84812514a76ab7225dfbf9329bf, 294e0e1970eb4e58bed426de35babb6b, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:03.062809 22810 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260812 08:03:03.063859 22832 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
W20260812 08:03:03.064841 22498 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 8ms of latency in SegmentAllocator::Sync()
I20260812 08:03:03.066847 18674 catalog_manager.cc:5719] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 reported cstate change: term changed from 9 to 10, leader changed from 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71) to 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69). New cstate: current_term: 10 leader_uuid: "285e078e6cfb46ae9dbe57c20be666d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } health_report { overall_health: UNKNOWN } } }
I20260812 08:03:03.068544 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 10 candidate_status { last_received { term: 9 index: 47 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:03.068709 21840 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38681904c36540a4876519ec5ffac6b6 in current term 10: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:03:03.070905 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 10 candidate_status { last_received { term: 9 index: 47 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:03.076304 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 10 candidate_status { last_received { term: 9 index: 47 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:03.076519 21512 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38681904c36540a4876519ec5ffac6b6 in current term 10: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:03:03.077648 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 10 candidate_status { last_received { term: 9 index: 47 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:03.077847 22069 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38681904c36540a4876519ec5ffac6b6 in current term 10: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:03:03.078563 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 10 candidate_status { last_received { term: 9 index: 47 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
W20260812 08:03:03.079078 22269 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 10 pre-election: Tablet error from VoteRequest() call to peer 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:03.079190 22269 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 38681904c36540a4876519ec5ffac6b6; no voters: 0e10b84812514a76ab7225dfbf9329bf, 294e0e1970eb4e58bed426de35babb6b, 5b4c3ad48b794617893c88a077be1b8d, a5813459b1d94e089ffbe751f4310de7
I20260812 08:03:03.079702 22832 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 9 FOLLOWER]: Advancing to term 10
I20260812 08:03:03.079968 19619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 10 candidate_status { last_received { term: 9 index: 47 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:03.080159 19619 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38681904c36540a4876519ec5ffac6b6 in current term 10: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:03:03.081069 22832 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260812 08:03:03.082298 20633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 10 candidate_status { last_received { term: 9 index: 47 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:03.115231 22611 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Time spent bootstrapping tablet: real 0.173s	user 0.031s	sys 0.008s
I20260812 08:03:03.116981 22611 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 9 FOLLOWER]: Replica starting. Triggering 10 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:03.119453 22611 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 9 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e10b84812514a76ab7225dfbf9329bf, State: Initialized, Role: FOLLOWER
I20260812 08:03:03.119948 22611 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 37, Last appended: 9.47, Last appended by leader: 47, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:03.120700 22791 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
I20260812 08:03:03.120949 22611 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
W20260812 08:03:03.139182 22836 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260812 08:03:03.139272 22836 fault_injection.cc:44] THIS SERVER MAY CRASH!
I20260812 08:03:03.140182 21840 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 10 FOLLOWER]: Refusing update from remote peer 285e078e6cfb46ae9dbe57c20be666d5: Log matching property violated. Preceding OpId in replica: term: 9 index: 37. Preceding OpId from leader: term: 10 index: 48. (index mismatch)
I20260812 08:03:03.140512 22069 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 10 FOLLOWER]: Refusing update from remote peer 285e078e6cfb46ae9dbe57c20be666d5: Log matching property violated. Preceding OpId in replica: term: 9 index: 47. Preceding OpId from leader: term: 10 index: 48. (index mismatch)
I20260812 08:03:03.140823 22836 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 48, Last known committed idx: 27, Time since last communication: 0.000s
I20260812 08:03:03.141014 22489 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 48, Last known committed idx: 37, Time since last communication: 0.000s
I20260812 08:03:03.145032 22384 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 10 FOLLOWER]: Refusing update from remote peer 285e078e6cfb46ae9dbe57c20be666d5: Log matching property violated. Preceding OpId in replica: term: 9 index: 47. Preceding OpId from leader: term: 10 index: 48. (index mismatch)
I20260812 08:03:03.145550 22836 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 48, Last known committed idx: 37, Time since last communication: 0.000s
W20260812 08:03:03.146811 22508 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Injecting 116ms of latency in SegmentAllocator::Sync()
I20260812 08:03:03.148132 19619 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 10 FOLLOWER]: Refusing update from remote peer 285e078e6cfb46ae9dbe57c20be666d5: Log matching property violated. Preceding OpId in replica: term: 9 index: 39. Preceding OpId from leader: term: 10 index: 48. (index mismatch)
I20260812 08:03:03.148607 22489 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 48, Last known committed idx: 30, Time since last communication: 0.000s
W20260812 08:03:03.151432 22499 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 67ms of latency in SegmentAllocator::Sync()
I20260812 08:03:03.177403 22707 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 9 FOLLOWER]: Advancing to term 10
I20260812 08:03:03.179596 22707 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 10 FOLLOWER]: Refusing update from remote peer 285e078e6cfb46ae9dbe57c20be666d5: Log matching property violated. Preceding OpId in replica: term: 9 index: 47. Preceding OpId from leader: term: 10 index: 48. (index mismatch)
I20260812 08:03:03.180318 22836 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 48, Last known committed idx: 37, Time since last communication: 0.000s
I20260812 08:03:03.181013 21512 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 10 FOLLOWER]: Refusing update from remote peer 285e078e6cfb46ae9dbe57c20be666d5: Log matching property violated. Preceding OpId in replica: term: 9 index: 42. Preceding OpId from leader: term: 10 index: 48. (index mismatch)
I20260812 08:03:03.181543 22836 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 48, Last known committed idx: 36, Time since last communication: 0.000s
W20260812 08:03:03.182838 22869 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Injecting 25ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.184417 22511 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Injecting 4ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.197149 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:03.198379 21996 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:03.198603 21999 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:03.198690 21995 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:03.198751 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:03.199124 21994 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:03.199223 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:03.199589 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:03.201917 19550 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:03.202843 19550 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:03.205004 19550 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:03.206210 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.206676 21751 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.206728 21764 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.206977 21744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.207139 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.207139 21762 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.209307 21762 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.209440 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.210585 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:03.211728 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:03.211814 19550 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:03.211895 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:03.213101 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:03.215363 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:03.216404 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:03.216594 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:03.217446 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:03.217460 19550 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:03.218555 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:03.218626 19550 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:03.219591 22498 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 76ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.221437 22511 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Injecting 69ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.223706 22503 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Injecting 34ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.224234 22499 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 122ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.230633 22869 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Injecting 55ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.231515 22504 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 78ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.261039 22503 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Injecting 34ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.267495 22508 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Injecting 81ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.288091 22869 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Injecting 77ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.294015 22511 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Injecting 17ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.302569 22498 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 86ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.303704 22503 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Injecting 84ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.313396 22504 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 37ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.351653 22499 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 3ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.353663 22508 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Injecting 50ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.359252 22499 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 16ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.361774 22504 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 41ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.361976 22511 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Injecting 49ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.370512 22869 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Injecting 85ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.389781 22498 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 157ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.394073 22499 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 44ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.398048 22503 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Injecting 36ms of latency in SegmentAllocator::Sync()
E20260812 08:03:03.403720 22836 fault_injection.cc:59] injecting fault for kudu (pid 20508): FLAGS_fault_crash_on_leader_request_fraction (process will exit)
W20260812 08:03:03.422238 21400 connection.cc:573] server connection from 127.18.42.69:44981 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 08:03:03.422297 22268 connection.cc:573] server connection from 127.18.42.69:34191 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 08:03:03.424796 19550 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:03:03.426169 18601 ts_itest-base.cc:575] TS 4 appears to have crashed. Restarting.
I20260812 08:03:03.426661 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.69:34819
--local_ip_for_outbound_sockets=127.18.42.69
--tserver_master_addrs=127.18.42.126:35213
--webserver_port=40235
--webserver_interface=127.18.42.69
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.05
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
W20260812 08:03:03.427237 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:03.430166 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:03.430290 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:03.431382 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.434607 21955 connection.cc:466] server connection from 127.18.42.69:39165 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.18.42.69:39165 (request call id 13) could send its response
W20260812 08:03:03.435897 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:03.435997 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.436882 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:03.438210 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:03.438699 19500 connection.cc:466] server connection from 127.18.42.69:60241 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.18.42.69:60241 (request call id 15) could send its response
W20260812 08:03:03.441294 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:03.441690 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.444810 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:03.444875 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:03.447902 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:03.448881 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:03.454440 22307 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.454430 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.455586 22307 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.456130 22603 connection.cc:466] server connection from 127.18.42.69:50477 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.18.42.69:50477 (request call id 8) could send its response
W20260812 08:03:03.459128 22307 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.462669 22307 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.470870 22642 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43128: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:03.470870 22641 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43128: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:03.472584 22642 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43128: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:03.476848 22642 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43128: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:03.486425 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:03.490453 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:03.496968 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.497062 21762 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.505187 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.506319 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.509835 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:03.512056 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:03.518364 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:03.522598 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:03.525331 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:03.528936 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:03.530979 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:03.537251 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:03.539386 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:03.543794 22307 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.546226 22307 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.551726 22888 flags.cc:432] Enabled unsafe flag: --fault_crash_on_leader_request_fraction=0.050000000000000003
W20260812 08:03:03.552043 22888 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:03:03.552078 22888 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:03:03.552109 22888 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:03:03.552181 22888 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:03.552217 22888 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:03.552294 22888 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:03.552753 22307 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.553865 22307 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.556430 22888 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:03.556576 22888 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.69
I20260812 08:03:03.560842 22888 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.050000000000000003
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.69:34819
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data/info.pb
--webserver_interface=127.18.42.69
--webserver_port=40235
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.22888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.69

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:03.562105 22888 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:03.563349 22888 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:03.563815 22642 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43128: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
E20260812 08:03:03.566390 22888 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:03:03.570222 22902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:03.570222 22900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:03.570219 22898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:03.570744 22888 server_base.cc:1061] running on GCE node
I20260812 08:03:03.571163 22888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:03.571852 22888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
W20260812 08:03:03.571974 22642 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43128: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:03.572018 22641 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43128: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
I20260812 08:03:03.573051 22888 hybrid_clock.cc:648] HybridClock initialized: now 1786521783572987 us; error 78 us; skew 500 ppm
I20260812 08:03:03.575158 22888 webserver.cc:533] Webserver started at http://127.18.42.69:40235/ using document root <none> and password file <none>
I20260812 08:03:03.575919 22888 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:03.575989 22888 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
W20260812 08:03:03.578722 22641 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43128: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
I20260812 08:03:03.580253 22888 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.003s	sys 0.002s
I20260812 08:03:03.582551 22912 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:03.584066 22888 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:03:03.584199 22888 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
uuid: "285e078e6cfb46ae9dbe57c20be666d5"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:03:03.584728 22888 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260812 08:03:03.596269 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:03.602764 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:03:03.604693 22888 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:03.605353 22888 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:03.605612 22888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:03.606299 22888 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
W20260812 08:03:03.606667 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:03.606776 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:03:03.607909 22923 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:03:03.611361 22888 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:03.611438 22888 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:03:03.611532 22888 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:03.613313 22888 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:03.613391 22888 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:03:03.613760 22923 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Bootstrap starting.
W20260812 08:03:03.618121 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.621595 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:03:03.624825 22923 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Log is configured to *not* fsync() on all Append() calls
W20260812 08:03:03.626060 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:03:03.627844 22888 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.69:34819
I20260812 08:03:03.629141 22888 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data/info.pb
I20260812 08:03:03.630365 23099 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.69:34819 every 8 connection(s)
I20260812 08:03:03.636090 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 22888
W20260812 08:03:03.639225 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
I20260812 08:03:03.639353 23100 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:03:03.639597 23100 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:03.640233 23100 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:03:03.642259 18674 ts_manager.cc:194] Re-registered known tserver with Master: 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819)
I20260812 08:03:03.643231 18674 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.69:46155
W20260812 08:03:03.643455 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:03.645503 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:03.645696 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
I20260812 08:03:03.652079 22832 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 285e078e6cfb46ae9dbe57c20be666d5)
I20260812 08:03:03.652210 22832 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:03.653023 22832 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:03.653542 19619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 11 candidate_status { last_received { term: 10 index: 68 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:03.653942 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 11 candidate_status { last_received { term: 10 index: 68 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:03.653966 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 11 candidate_status { last_received { term: 10 index: 68 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:03.654255 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 11 candidate_status { last_received { term: 10 index: 68 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:03.654584 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 11 candidate_status { last_received { term: 10 index: 68 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:03.654784 22268 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 38681904c36540a4876519ec5ffac6b6; no voters: 0e10b84812514a76ab7225dfbf9329bf, 5b4c3ad48b794617893c88a077be1b8d, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:03.655033 22832 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260812 08:03:03.660041 23016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 11 candidate_status { last_received { term: 10 index: 68 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:03.661350 22534 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 285e078e6cfb46ae9dbe57c20be666d5)
I20260812 08:03:03.661546 22534 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:03.661983 22547 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 285e078e6cfb46ae9dbe57c20be666d5)
I20260812 08:03:03.662101 22547 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:03.662350 22534 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:03.662837 22547 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:03.663206 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 11 candidate_status { last_received { term: 10 index: 69 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:03.663522 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 11 candidate_status { last_received { term: 10 index: 69 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:03.663640 21512 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 10.
I20260812 08:03:03.663789 19619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 11 candidate_status { last_received { term: 10 index: 69 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:03.663846 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 69 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:03.663944 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 69 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:03.664069 21840 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 10.
W20260812 08:03:03.664487 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:03.664487 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
I20260812 08:03:03.664767 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 69 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:03.665580 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 69 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:03.665733 22384 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 10.
I20260812 08:03:03.666067 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 11 candidate_status { last_received { term: 10 index: 69 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:03.666280 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 69 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
W20260812 08:03:03.666821 22269 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 11 pre-election: Tablet error from VoteRequest() call to peer 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819): Illegal state: must be running to vote when last-logged opid is not known
W20260812 08:03:03.667516 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:03.667544 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
I20260812 08:03:03.667956 22852 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 285e078e6cfb46ae9dbe57c20be666d5)
I20260812 08:03:03.667999 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 11 candidate_status { last_received { term: 10 index: 69 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:03.668107 22384 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 10.
I20260812 08:03:03.668128 22852 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:03.669965 22923 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Bootstrap replayed 1/1 log segments. Stats: ops{read=75 overwritten=2 applied=63 ignored=0} inserts{seen=58 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 10 replicates
I20260812 08:03:03.670430 22852 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785)
I20260812 08:03:03.670902 22923 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Bootstrap complete.
W20260812 08:03:03.671083 22923 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 53ms of latency in SegmentAllocator::Sync()
I20260812 08:03:03.671604 23016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 69 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
W20260812 08:03:03.672027 21400 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 11 pre-election: Tablet error from VoteRequest() call to peer 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:03.672111 21400 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 294e0e1970eb4e58bed426de35babb6b, 38681904c36540a4876519ec5ffac6b6, a5813459b1d94e089ffbe751f4310de7; no voters: 0e10b84812514a76ab7225dfbf9329bf, 285e078e6cfb46ae9dbe57c20be666d5, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:03.672386 22547 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260812 08:03:03.673887 23016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 11 candidate_status { last_received { term: 10 index: 69 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
W20260812 08:03:03.674435 21680 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 11 pre-election: Tablet error from VoteRequest() call to peer 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:03.674516 21680 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 294e0e1970eb4e58bed426de35babb6b, 38681904c36540a4876519ec5ffac6b6, a5813459b1d94e089ffbe751f4310de7; no voters: 0e10b84812514a76ab7225dfbf9329bf, 285e078e6cfb46ae9dbe57c20be666d5, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:03.674676 22534 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260812 08:03:03.677223 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 73 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:03.677636 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 73 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:03.677773 21840 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e10b84812514a76ab7225dfbf9329bf in term 10.
I20260812 08:03:03.678755 23016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 73 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
W20260812 08:03:03.679205 22607 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 11 pre-election: Tablet error from VoteRequest() call to peer 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:03.680521 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 73 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:03.680540 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 73 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:03.680675 22384 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e10b84812514a76ab7225dfbf9329bf in term 10.
I20260812 08:03:03.681021 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 73 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:03.681167 21512 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e10b84812514a76ab7225dfbf9329bf in term 10.
I20260812 08:03:03.681459 22606 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 294e0e1970eb4e58bed426de35babb6b, 38681904c36540a4876519ec5ffac6b6, a5813459b1d94e089ffbe751f4310de7; no voters: 285e078e6cfb46ae9dbe57c20be666d5, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:03.681767 22852 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260812 08:03:03.681826 22852 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 10 FOLLOWER]: Starting leader election (detected failure of leader 285e078e6cfb46ae9dbe57c20be666d5)
I20260812 08:03:03.681855 22852 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 10 FOLLOWER]: Advancing to term 11
I20260812 08:03:03.682874 22852 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:03.683523 22852 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 11 election: Requested vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785)
I20260812 08:03:03.683805 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 73 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d"
I20260812 08:03:03.684000 23016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 73 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5"
I20260812 08:03:03.684157 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 73 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7"
I20260812 08:03:03.684288 21840 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 10 FOLLOWER]: Advancing to term 11
W20260812 08:03:03.684355 22607 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 11 election: Tablet error from VoteRequest() call to peer 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:03.684476 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 73 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6"
I20260812 08:03:03.684582 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 73 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec"
I20260812 08:03:03.684649 22384 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 10 FOLLOWER]: Advancing to term 11
I20260812 08:03:03.685307 21840 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e10b84812514a76ab7225dfbf9329bf in term 11.
I20260812 08:03:03.685626 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 73 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b"
I20260812 08:03:03.685767 21512 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 10 FOLLOWER]: Advancing to term 11
I20260812 08:03:03.685940 22384 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e10b84812514a76ab7225dfbf9329bf in term 11.
I20260812 08:03:03.686725 21512 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e10b84812514a76ab7225dfbf9329bf in term 11.
I20260812 08:03:03.687047 22606 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 294e0e1970eb4e58bed426de35babb6b, 38681904c36540a4876519ec5ffac6b6, a5813459b1d94e089ffbe751f4310de7; no voters: 285e078e6cfb46ae9dbe57c20be666d5, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:03.687297 22852 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 11 FOLLOWER]: Leader election won for term 11
I20260812 08:03:03.687577 22852 raft_consensus.cc:697] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 11 LEADER]: Becoming Leader. State: Replica: 0e10b84812514a76ab7225dfbf9329bf, State: Running, Role: LEADER
I20260812 08:03:03.687907 22852 consensus_queue.cc:237] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 63, Committed index: 63, Last appended: 10.73, Last appended by leader: 73, Current term: 11, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
W20260812 08:03:03.690231 22307 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.690459 22309 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.690878 22310 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.691048 22306 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
I20260812 08:03:03.691802 18674 catalog_manager.cc:5719] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf reported cstate change: term changed from 10 to 11, leader changed from 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69) to 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71). New cstate: current_term: 11 leader_uuid: "0e10b84812514a76ab7225dfbf9329bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:03.694306 22492 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 285e078e6cfb46ae9dbe57c20be666d5)
I20260812 08:03:03.694442 22492 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:03.695305 22492 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:03.695511 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 74 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:03.695974 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 74 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:03.695989 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 74 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:03.696167 21512 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b7def2bd3a74420d9deec19b32f541ec in current term 11: Already voted for candidate 0e10b84812514a76ab7225dfbf9329bf in this term.
I20260812 08:03:03.696192 22384 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b7def2bd3a74420d9deec19b32f541ec in current term 11: Already voted for candidate 0e10b84812514a76ab7225dfbf9329bf in this term.
I20260812 08:03:03.696611 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 74 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:03.696841 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 74 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:03.697041 21840 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b7def2bd3a74420d9deec19b32f541ec in current term 11: Already voted for candidate 0e10b84812514a76ab7225dfbf9329bf in this term.
I20260812 08:03:03.697070 19502 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: b7def2bd3a74420d9deec19b32f541ec; no voters: 0e10b84812514a76ab7225dfbf9329bf, 294e0e1970eb4e58bed426de35babb6b, 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d
I20260812 08:03:03.697201 23147 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 285e078e6cfb46ae9dbe57c20be666d5)
I20260812 08:03:03.697360 23147 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:03.697484 22492 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 10 FOLLOWER]: Advancing to term 11
I20260812 08:03:03.698122 23147 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:03.698642 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 11 candidate_status { last_received { term: 10 index: 74 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:03.698689 22492 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260812 08:03:03.698797 19620 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 11.
I20260812 08:03:03.699163 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 11 candidate_status { last_received { term: 10 index: 74 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:03.699324 21512 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b4c3ad48b794617893c88a077be1b8d in current term 11: Already voted for candidate 0e10b84812514a76ab7225dfbf9329bf in this term.
I20260812 08:03:03.699574 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 11 candidate_status { last_received { term: 10 index: 74 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:03.700040 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 11 candidate_status { last_received { term: 10 index: 74 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:03.700208 22384 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b4c3ad48b794617893c88a077be1b8d in current term 11: Already voted for candidate 0e10b84812514a76ab7225dfbf9329bf in this term.
I20260812 08:03:03.700452 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 11 candidate_status { last_received { term: 10 index: 74 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:03.700623 21840 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b4c3ad48b794617893c88a077be1b8d in current term 11: Already voted for candidate 0e10b84812514a76ab7225dfbf9329bf in this term.
I20260812 08:03:03.701005 23016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 74 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:03.701032 21956 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec; no voters: 0e10b84812514a76ab7225dfbf9329bf, 294e0e1970eb4e58bed426de35babb6b, 38681904c36540a4876519ec5ffac6b6, a5813459b1d94e089ffbe751f4310de7
I20260812 08:03:03.701226 23147 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 10 FOLLOWER]: Advancing to term 11
W20260812 08:03:03.701372 19502 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 11 pre-election: Tablet error from VoteRequest() call to peer 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:03.702446 23147 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260812 08:03:03.702739 23016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 11 candidate_status { last_received { term: 10 index: 74 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
W20260812 08:03:03.703166 21956 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 11 pre-election: Tablet error from VoteRequest() call to peer 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819): Illegal state: must be running to vote when last-logged opid is not known
W20260812 08:03:03.713124 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:03.715974 22852 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260812 08:03:03.716071 22852 fault_injection.cc:44] THIS SERVER MAY CRASH!
I20260812 08:03:03.716843 19620 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 11 FOLLOWER]: Refusing update from remote peer 0e10b84812514a76ab7225dfbf9329bf: Log matching property violated. Preceding OpId in replica: term: 10 index: 74. Preceding OpId from leader: term: 11 index: 75. (index mismatch)
I20260812 08:03:03.717018 22384 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 11 FOLLOWER]: Refusing update from remote peer 0e10b84812514a76ab7225dfbf9329bf: Log matching property violated. Preceding OpId in replica: term: 10 index: 68. Preceding OpId from leader: term: 11 index: 75. (index mismatch)
I20260812 08:03:03.717180 22069 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 11 FOLLOWER]: Refusing update from remote peer 0e10b84812514a76ab7225dfbf9329bf: Log matching property violated. Preceding OpId in replica: term: 10 index: 74. Preceding OpId from leader: term: 11 index: 75. (index mismatch)
I20260812 08:03:03.717885 23156 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 64, Time since last communication: 0.000s
I20260812 08:03:03.718086 23156 consensus_queue.cc:1243] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [LEADER]: Peer b7def2bd3a74420d9deec19b32f541ec log is divergent from this leader: its last log entry 10.74 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 64
I20260812 08:03:03.718194 23138 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 58, Time since last communication: 0.000s
I20260812 08:03:03.718386 21840 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 11 FOLLOWER]: Refusing update from remote peer 0e10b84812514a76ab7225dfbf9329bf: Log matching property violated. Preceding OpId in replica: term: 10 index: 69. Preceding OpId from leader: term: 11 index: 75. (index mismatch)
I20260812 08:03:03.718386 21512 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 11 FOLLOWER]: Refusing update from remote peer 0e10b84812514a76ab7225dfbf9329bf: Log matching property violated. Preceding OpId in replica: term: 10 index: 69. Preceding OpId from leader: term: 11 index: 75. (index mismatch)
I20260812 08:03:03.718806 22852 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 68, Time since last communication: 0.000s
I20260812 08:03:03.718928 22852 consensus_queue.cc:1243] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [LEADER]: Peer 5b4c3ad48b794617893c88a077be1b8d log is divergent from this leader: its last log entry 10.74 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 68
I20260812 08:03:03.718999 23152 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 63, Time since last communication: 0.000s
I20260812 08:03:03.720039 22069 raft_consensus.cc:1217] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 11 FOLLOWER]: Deduplicated request from leader. Original: 10.68->[10.69-11.77]   Dedup: 10.73->[11.74-11.77]
I20260812 08:03:03.720132 19620 raft_consensus.cc:1217] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 11 FOLLOWER]: Deduplicated request from leader. Original: 10.64->[10.65-11.77]   Dedup: 10.73->[11.74-11.77]
I20260812 08:03:03.720203 22069 pending_rounds.cc:85] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Aborting all ops after (but not including) 73
I20260812 08:03:03.720224 19620 pending_rounds.cc:85] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Aborting all ops after (but not including) 73
I20260812 08:03:03.720252 19620 pending_rounds.cc:107] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Aborting uncommitted WRITE_OP operation due to leader change: 10.74
I20260812 08:03:03.720263 22069 pending_rounds.cc:107] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Aborting uncommitted WRITE_OP operation due to leader change: 10.74
W20260812 08:03:03.722198 22503 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Injecting 66ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.723877 22499 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 61ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.725476 22607 consensus_peers.cc:597] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf -> Peer 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819): Couldn't send request to peer 285e078e6cfb46ae9dbe57c20be666d5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:03:03.725557 22923 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Time spent bootstrapping tablet: real 0.112s	user 0.050s	sys 0.006s
I20260812 08:03:03.725734 23156 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 63, Time since last communication: 0.000s
I20260812 08:03:03.727638 22923 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 10 FOLLOWER]: Replica starting. Triggering 10 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
W20260812 08:03:03.729920 22504 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 16ms of latency in SegmentAllocator::Sync()
I20260812 08:03:03.731186 22923 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 10 FOLLOWER]: Becoming Follower/Learner. State: Replica: 285e078e6cfb46ae9dbe57c20be666d5, State: Initialized, Role: FOLLOWER
I20260812 08:03:03.731940 22923 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 63, Last appended: 10.73, Last appended by leader: 73, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:03.732728 23100 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
W20260812 08:03:03.732758 22511 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Injecting 64ms of latency in SegmentAllocator::Sync()
I20260812 08:03:03.733320 22923 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
W20260812 08:03:03.737879 22508 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Injecting 116ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.750428 22504 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 10ms of latency in SegmentAllocator::Sync()
W20260812 08:03:03.754215 22869 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Injecting 99ms of latency in SegmentAllocator::Sync()
E20260812 08:03:03.761468 23149 fault_injection.cc:59] injecting fault for kudu (pid 22571): FLAGS_fault_crash_on_leader_request_fraction (process will exit)
W20260812 08:03:03.784126 20378 meta_cache.cc:302] tablet 50b14eb091f7476d9d7682e503745c93: replica 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647) has failed: Network error: recv got EOF from 127.18.42.71:35647 (error 108)
I20260812 08:03:03.785079 18601 ts_itest-base.cc:575] TS 6 appears to have crashed. Restarting.
I20260812 08:03:03.785570 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.71:35647
--local_ip_for_outbound_sockets=127.18.42.71
--tserver_master_addrs=127.18.42.126:35213
--webserver_port=37531
--webserver_interface=127.18.42.71
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.05
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
W20260812 08:03:03.785590 19502 connection.cc:466] server connection from 127.18.42.71:42471 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.18.42.71:42471 (request call id 3) could send its response
W20260812 08:03:03.788787 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:03.788841 21955 connection.cc:466] server connection from 127.18.42.71:48541 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.18.42.71:48541 (request call id 3) could send its response
W20260812 08:03:03.788787 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:03.789588 19550 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:03.790488 19550 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:03.794185 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.795259 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.795385 21762 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.797338 21398 connection.cc:466] server connection from 127.18.42.71:33553 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.18.42.71:33553 (request call id 4) could send its response
W20260812 08:03:03.799916 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:03.800614 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:03.800923 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:03.801975 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:03.810581 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:03.811530 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:03.811575 21437 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:03.823393 22928 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44224: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.823393 22936 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44224: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.823393 22952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44224: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.831709 22310 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.831719 22306 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.833078 22306 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.854527 22269 connection.cc:466] server connection from 127.18.42.71:44557 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.18.42.71:44557 (request call id 4) could send its response
W20260812 08:03:03.858470 19550 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:03.858601 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:03.858470 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:03.869571 21762 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.873831 21764 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.873838 21744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.881891 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:03.887887 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:03.888909 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:03.895406 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:03.901633 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:03.904695 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:03.911376 22952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44224: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.912738 23162 flags.cc:432] Enabled unsafe flag: --fault_crash_on_leader_request_fraction=0.050000000000000003
W20260812 08:03:03.912949 23162 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:03:03.912967 23162 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:03:03.912983 23162 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:03:03.913022 23162 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:03.913048 23162 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:03.913093 23162 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:03.916599 23162 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:03.916712 23162 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.71
W20260812 08:03:03.918603 22952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44224: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:03.919587 22952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44224: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
I20260812 08:03:03.921116 23162 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.050000000000000003
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.71:35647
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data/info.pb
--webserver_interface=127.18.42.71
--webserver_port=37531
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.23162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.71

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:03.922258 23162 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:03.923441 23162 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:03.926074 22306 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
E20260812 08:03:03.926499 23162 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:03:03.930198 23167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:03.930194 23168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:03.930194 23174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:03.930562 23162 server_base.cc:1061] running on GCE node
I20260812 08:03:03.930967 23162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:03.931524 23162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:03.932708 23162 hybrid_clock.cc:648] HybridClock initialized: now 1786521783932684 us; error 37 us; skew 500 ppm
W20260812 08:03:03.934410 22306 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
I20260812 08:03:03.934803 23162 webserver.cc:533] Webserver started at http://127.18.42.71:37531/ using document root <none> and password file <none>
I20260812 08:03:03.935388 23162 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:03.935452 23162 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
W20260812 08:03:03.936450 22306 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
I20260812 08:03:03.939136 23162 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.004s	sys 0.000s
I20260812 08:03:03.941043 23183 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:03.942102 23162 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:03.942229 23162 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/wal
uuid: "0e10b84812514a76ab7225dfbf9329bf"
format_stamp: "Formatted at 2026-08-12 08:02:59 on dist-test-slave-zr1t"
I20260812 08:03:03.942731 23162 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260812 08:03:03.965080 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:03:03.967450 23162 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:03.968214 23162 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:03.968447 23162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:03.969117 23162 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:03.970484 23194 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260812 08:03:03.972218 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:03.974272 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:03:03.974411 23162 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:03.974485 23162 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.001s	sys 0.000s
I20260812 08:03:03.974570 23162 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:03.976127 23162 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:03.976189 23162 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:03:03.976512 23194 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Bootstrap starting.
W20260812 08:03:03.983165 21764 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:03:03.988660 23162 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.71:35647
I20260812 08:03:03.988677 23357 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.71:35647 every 8 connection(s)
I20260812 08:03:03.990033 23162 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data/info.pb
W20260812 08:03:03.990175 21764 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:03:03.991531 23194 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Log is configured to *not* fsync() on all Append() calls
W20260812 08:03:03.994217 21764 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:03:03.995064 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 23162
I20260812 08:03:03.999473 23358 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:03:03.999709 23358 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:04.000276 23358 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:03:04.002161 18661 ts_manager.cc:194] Re-registered known tserver with Master: 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:04.003067 18661 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.71:40457
W20260812 08:03:04.003829 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:04.013211 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:04.014238 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
I20260812 08:03:04.021467 23147 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e10b84812514a76ab7225dfbf9329bf)
I20260812 08:03:04.021648 23147 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:04.021793 22492 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e10b84812514a76ab7225dfbf9329bf)
I20260812 08:03:04.021925 22492 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:04.022445 23147 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:04.022727 22492 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:04.023065 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 12 candidate_status { last_received { term: 11 index: 77 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:04.023149 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 12 candidate_status { last_received { term: 11 index: 77 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:04.023198 19620 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 11.
I20260812 08:03:04.023839 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 12 candidate_status { last_received { term: 11 index: 77 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:04.023919 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 12 candidate_status { last_received { term: 11 index: 77 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:04.024252 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 77 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:04.024413 23016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 12 candidate_status { last_received { term: 11 index: 77 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:04.024682 23016 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 10.
I20260812 08:03:04.024917 23017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 77 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:04.025058 23017 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7def2bd3a74420d9deec19b32f541ec in term 10.
I20260812 08:03:04.025946 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 77 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:04.026539 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 77 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:04.026674 22069 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7def2bd3a74420d9deec19b32f541ec in term 11.
I20260812 08:03:04.026919 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 77 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
W20260812 08:03:04.027849 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
I20260812 08:03:04.032390 22547 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e10b84812514a76ab7225dfbf9329bf)
I20260812 08:03:04.032513 22547 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:04.031420 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 12 candidate_status { last_received { term: 11 index: 77 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:04.033164 22547 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:04.034065 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 78 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:04.031821 23283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 77 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
W20260812 08:03:04.035095 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:04.035104 19502 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 12 pre-election: Tablet error from VoteRequest() call to peer 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:04.035180 19502 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 285e078e6cfb46ae9dbe57c20be666d5, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec; no voters: 0e10b84812514a76ab7225dfbf9329bf, 294e0e1970eb4e58bed426de35babb6b, 38681904c36540a4876519ec5ffac6b6, a5813459b1d94e089ffbe751f4310de7
I20260812 08:03:04.035514 22492 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260812 08:03:04.035516 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 78 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
W20260812 08:03:04.036543 21442 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53094: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:04.036579 21956 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 12 pre-election: Tablet error from VoteRequest() call to peer 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:04.036655 21956 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 285e078e6cfb46ae9dbe57c20be666d5, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec; no voters: 0e10b84812514a76ab7225dfbf9329bf, 294e0e1970eb4e58bed426de35babb6b, 38681904c36540a4876519ec5ffac6b6, a5813459b1d94e089ffbe751f4310de7
I20260812 08:03:04.036994 23147 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260812 08:03:04.037981 23017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 78 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:04.038139 23017 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 10.
I20260812 08:03:04.038417 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 78 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:04.038543 19620 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 11.
I20260812 08:03:04.038791 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 78 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:04.038913 22069 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 11.
I20260812 08:03:04.039789 21400 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 285e078e6cfb46ae9dbe57c20be666d5, 294e0e1970eb4e58bed426de35babb6b, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec; no voters: 38681904c36540a4876519ec5ffac6b6, a5813459b1d94e089ffbe751f4310de7
I20260812 08:03:04.040062 22547 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260812 08:03:04.040135 22547 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0e10b84812514a76ab7225dfbf9329bf)
I20260812 08:03:04.040179 22547 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 11 FOLLOWER]: Advancing to term 12
I20260812 08:03:04.041337 23194 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Bootstrap replayed 1/1 log segments. Stats: ops{read=81 overwritten=0 applied=77 ignored=0} inserts{seen=71 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 4 replicates
I20260812 08:03:04.041389 22547 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:04.041906 23283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 78 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:04.042142 22547 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 12 election: Requested vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:04.041911 23194 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Bootstrap complete.
W20260812 08:03:04.042490 23194 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Injecting 109ms of latency in SegmentAllocator::Sync()
I20260812 08:03:04.042735 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 78 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d"
I20260812 08:03:04.042745 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 78 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6"
I20260812 08:03:04.042871 22069 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 11 FOLLOWER]: Advancing to term 12
I20260812 08:03:04.042848 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 78 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec"
I20260812 08:03:04.042984 19620 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 11 FOLLOWER]: Advancing to term 12
I20260812 08:03:04.044024 19620 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 12.
I20260812 08:03:04.044108 22069 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 12.
W20260812 08:03:04.044756 21400 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 12 pre-election: Tablet error from VoteRequest() call to peer 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:04.044871 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 78 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7"
I20260812 08:03:04.044915 23017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 78 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5"
I20260812 08:03:04.045069 23017 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 10 FOLLOWER]: Advancing to term 12
I20260812 08:03:04.045218 23283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 78 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf"
W20260812 08:03:04.045608 21400 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 12 election: Tablet error from VoteRequest() call to peer 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:04.047202 23017 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 12.
I20260812 08:03:04.047592 21400 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 285e078e6cfb46ae9dbe57c20be666d5, 294e0e1970eb4e58bed426de35babb6b, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec; no voters: 0e10b84812514a76ab7225dfbf9329bf, 38681904c36540a4876519ec5ffac6b6, a5813459b1d94e089ffbe751f4310de7
I20260812 08:03:04.047823 22547 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 12 FOLLOWER]: Leader election won for term 12
I20260812 08:03:04.048004 22547 raft_consensus.cc:697] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 12 LEADER]: Becoming Leader. State: Replica: 294e0e1970eb4e58bed426de35babb6b, State: Running, Role: LEADER
I20260812 08:03:04.048355 22547 consensus_queue.cc:237] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 63, Committed index: 63, Last appended: 11.78, Last appended by leader: 78, Current term: 12, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:04.048738 22534 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e10b84812514a76ab7225dfbf9329bf)
I20260812 08:03:04.048852 22534 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
W20260812 08:03:04.049415 22952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44224: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
I20260812 08:03:04.049814 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 12 candidate_status { last_received { term: 11 index: 79 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:04.049988 22069 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5813459b1d94e089ffbe751f4310de7 in current term 12: Already voted for candidate 294e0e1970eb4e58bed426de35babb6b in this term.
I20260812 08:03:04.050117 22534 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:04.050117 23017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 12 candidate_status { last_received { term: 11 index: 79 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:04.050318 23017 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5813459b1d94e089ffbe751f4310de7 in current term 12: Already voted for candidate 294e0e1970eb4e58bed426de35babb6b in this term.
I20260812 08:03:04.050545 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 12 candidate_status { last_received { term: 11 index: 79 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:04.050745 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 12 candidate_status { last_received { term: 11 index: 79 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:04.050904 19620 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5813459b1d94e089ffbe751f4310de7 in current term 12: Already voted for candidate 294e0e1970eb4e58bed426de35babb6b in this term.
I20260812 08:03:04.051265 21679 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: a5813459b1d94e089ffbe751f4310de7; no voters: 285e078e6cfb46ae9dbe57c20be666d5, 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:04.051366 21512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 12 candidate_status { last_received { term: 11 index: 79 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:04.051465 22534 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 11 FOLLOWER]: Advancing to term 12
I20260812 08:03:04.052563 22534 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260812 08:03:04.053090 18661 catalog_manager.cc:5719] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b reported cstate change: term changed from 11 to 12, leader changed from 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71) to 294e0e1970eb4e58bed426de35babb6b (127.18.42.67). New cstate: current_term: 12 leader_uuid: "294e0e1970eb4e58bed426de35babb6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } health_report { overall_health: UNKNOWN } } }
I20260812 08:03:04.054942 23283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 12 candidate_status { last_received { term: 11 index: 79 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
W20260812 08:03:04.055317 21680 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 12 pre-election: Tablet error from VoteRequest() call to peer 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647): Illegal state: must be running to vote when last-logged opid is not known
W20260812 08:03:04.059762 22952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44224: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.060746 22952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44224: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.075342 22306 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.082554 22306 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.084601 22306 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.100679 23366 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260812 08:03:04.100854 23366 fault_injection.cc:44] THIS SERVER MAY CRASH!
I20260812 08:03:04.101711 22384 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 11 FOLLOWER]: Advancing to term 12
I20260812 08:03:04.101810 19620 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 12 FOLLOWER]: Refusing update from remote peer 294e0e1970eb4e58bed426de35babb6b: Log matching property violated. Preceding OpId in replica: term: 11 index: 77. Preceding OpId from leader: term: 12 index: 80. (index mismatch)
I20260812 08:03:04.102252 21840 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 12 FOLLOWER]: Refusing update from remote peer 294e0e1970eb4e58bed426de35babb6b: Log matching property violated. Preceding OpId in replica: term: 11 index: 79. Preceding OpId from leader: term: 12 index: 80. (index mismatch)
I20260812 08:03:04.102866 23017 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 12 FOLLOWER]: Refusing update from remote peer 294e0e1970eb4e58bed426de35babb6b: Log matching property violated. Preceding OpId in replica: term: 10 index: 73. Preceding OpId from leader: term: 12 index: 80. (index mismatch)
I20260812 08:03:04.103048 22384 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 12 FOLLOWER]: Refusing update from remote peer 294e0e1970eb4e58bed426de35babb6b: Log matching property violated. Preceding OpId in replica: term: 11 index: 78. Preceding OpId from leader: term: 12 index: 80. (index mismatch)
W20260812 08:03:04.103139 21400 consensus_peers.cc:597] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b -> Peer 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647): Couldn't send request to peer 0e10b84812514a76ab7225dfbf9329bf. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:03:04.103338 23366 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 79, Last known committed idx: 77, Time since last communication: 0.000s
I20260812 08:03:04.103492 23366 consensus_queue.cc:1243] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [LEADER]: Peer a5813459b1d94e089ffbe751f4310de7 log is divergent from this leader: its last log entry 11.79 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 77
I20260812 08:03:04.103503 22069 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 12 FOLLOWER]: Refusing update from remote peer 294e0e1970eb4e58bed426de35babb6b: Log matching property violated. Preceding OpId in replica: term: 11 index: 77. Preceding OpId from leader: term: 12 index: 80. (index mismatch)
I20260812 08:03:04.103566 23368 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 79, Last known committed idx: 63, Time since last communication: 0.000s
I20260812 08:03:04.103721 22547 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 79, Last known committed idx: 63, Time since last communication: 0.000s
I20260812 08:03:04.104274 22547 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 79, Last known committed idx: 63, Time since last communication: 0.000s
I20260812 08:03:04.104770 21840 raft_consensus.cc:1217] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 11.77->[11.78-12.80]   Dedup: 11.78->[12.79-12.80]
I20260812 08:03:04.104974 21840 pending_rounds.cc:85] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Aborting all ops after (but not including) 78
I20260812 08:03:04.105010 21840 pending_rounds.cc:107] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Aborting uncommitted WRITE_OP operation due to leader change: 11.79
I20260812 08:03:04.108045 23368 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 79, Last known committed idx: 63, Time since last communication: 0.000s
W20260812 08:03:04.109130 22499 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 67ms of latency in SegmentAllocator::Sync()
I20260812 08:03:04.116106 22493 mvcc.cc:204] Tried to move back new op lower bound from 7317593227673690112 to 7317593225990066176. Current Snapshot: MvccSnapshot[applied={T|T < 7317593224654495744}]
W20260812 08:03:04.131038 20495 scanner-internal.cc:458] Time spent opening tablet: real 4.427s	user 0.004s	sys 0.000s
W20260812 08:03:04.132123 20493 scanner-internal.cc:458] Time spent opening tablet: real 4.432s	user 0.006s	sys 0.000s
W20260812 08:03:04.137725 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
E20260812 08:03:04.139560 23372 fault_injection.cc:59] injecting fault for kudu (pid 21357): FLAGS_fault_crash_on_leader_request_fraction (process will exit)
W20260812 08:03:04.150707 21762 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.150727 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:03:04.152070 23194 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Time spent bootstrapping tablet: real 0.176s	user 0.060s	sys 0.004s
I20260812 08:03:04.154279 23194 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 11 FOLLOWER]: Replica starting. Triggering 4 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
W20260812 08:03:04.155506 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:04.158528 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:03:04.158603 18601 ts_itest-base.cc:575] TS 2 appears to have crashed. Restarting.
I20260812 08:03:04.159025 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.67:40205
--local_ip_for_outbound_sockets=127.18.42.67
--tserver_master_addrs=127.18.42.126:35213
--webserver_port=39753
--webserver_interface=127.18.42.67
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.05
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
W20260812 08:03:04.159698 21762 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:03:04.161551 23194 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 11 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e10b84812514a76ab7225dfbf9329bf, State: Initialized, Role: FOLLOWER
W20260812 08:03:04.162163 22952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44224: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
I20260812 08:03:04.162209 23194 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 77, Last appended: 11.81, Last appended by leader: 81, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:04.163228 23358 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
I20260812 08:03:04.163229 23194 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
W20260812 08:03:04.164970 22952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44224: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.170429 22306 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.171456 22306 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.176594 19500 connection.cc:466] server connection from 127.18.42.67:59229 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.18.42.67:59229 (request call id 6) could send its response
W20260812 08:03:04.178009 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:04.186350 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:04.187501 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:04.188704 22952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44224: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.191581 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:04.198877 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:04.204308 21762 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.208511 21762 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.215332 21762 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.215332 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.219606 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:04.220852 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:04.235828 22952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44224: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.236815 22952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44224: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.243402 22928 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44224: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.243402 22952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44224: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.250399 22306 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.252315 22306 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.253379 22306 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.267892 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:04.270385 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:04.274982 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:04.283532 21763 tablet_service.cc:3080] error setting up scanner e181a2acd7a94c308ff8d2afbdb6fdc3 with request: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: tablet hasn't heard from leader or there hasn't been a stable leader for 0.161s (maximum allowed 0.150s): new_scan_request { tablet_id: "50b14eb091f7476d9d7682e503745c93" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7317593227800997888 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\337\323\360\323\006\"3\n\005slave\022*\n f676baf4d90e47909c86ef51b3260926\020\001\030\001 \001(\001" signature: "<redacted>""3\272\377\3278\0220\236\'\036\225\3266\312\023\204L,\347\252\335\306\257\345\221\275\300\207\352\031\323W\277_ \003\305r\241\377 \344\323\265\031\320\027B*d\272\350xy\316h\030\272\227\026\236s\235\247" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "d4716c3919d347228becbc6c15e47625"
W20260812 08:03:04.284528 21763 tablet_service.cc:3080] error setting up scanner cac9cd6950144971919fc2dd611b2a65 with request: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: tablet hasn't heard from leader or there hasn't been a stable leader for 0.162s (maximum allowed 0.150s): new_scan_request { tablet_id: "50b14eb091f7476d9d7682e503745c93" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7317593227800997888 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\337\323\360\323\006\"3\n\005slave\022*\n f676baf4d90e47909c86ef51b3260926\020\001\030\001 \001(\001" signature: "<redacted>""3\272\377\3278\0220\236\'\036\225\3266\312\023\204L,\347\252\335\306\257\345\221\275\300\207\352\031\323W\277_ \003\305r\241\377 \344\323\265\031\320\027B*d\272\350xy\316h\030\272\227\026\236s\235\247" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "5ff8540deebd45bda469c5dfbf71d1a2"
W20260812 08:03:04.301510 23407 flags.cc:432] Enabled unsafe flag: --fault_crash_on_leader_request_fraction=0.050000000000000003
W20260812 08:03:04.301985 23407 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:03:04.302045 23407 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:03:04.302093 23407 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:03:04.302176 23407 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:04.302235 23407 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:04.302357 23407 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:04.302991 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:04.305051 23220 tablet_service.cc:3080] error setting up scanner c479e0455ad84b48b87b19ceb0de5f03 with request: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: tablet hasn't heard from leader or there hasn't been a stable leader for 0.152s (maximum allowed 0.150s): new_scan_request { tablet_id: "50b14eb091f7476d9d7682e503745c93" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7317593227800997888 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\337\323\360\323\006\"3\n\005slave\022*\n f676baf4d90e47909c86ef51b3260926\020\001\030\001 \001(\001" signature: "<redacted>""3\272\377\3278\0220\236\'\036\225\3266\312\023\204L,\347\252\335\306\257\345\221\275\300\207\352\031\323W\277_ \003\305r\241\377 \344\323\265\031\320\027B*d\272\350xy\316h\030\272\227\026\236s\235\247" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "d4716c3919d347228becbc6c15e47625"
W20260812 08:03:04.305418 23222 tablet_service.cc:3080] error setting up scanner bf37889900eb4c8a8159ff56dfadf331 with request: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: tablet hasn't heard from leader or there hasn't been a stable leader for 0.153s (maximum allowed 0.150s): new_scan_request { tablet_id: "50b14eb091f7476d9d7682e503745c93" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7317593227800997888 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\337\323\360\323\006\"3\n\005slave\022*\n f676baf4d90e47909c86ef51b3260926\020\001\030\001 \001(\001" signature: "<redacted>""3\272\377\3278\0220\236\'\036\225\3266\312\023\204L,\347\252\335\306\257\345\221\275\300\207\352\031\323W\277_ \003\305r\241\377 \344\323\265\031\320\027B*d\272\350xy\316h\030\272\227\026\236s\235\247" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "5ff8540deebd45bda469c5dfbf71d1a2"
W20260812 08:03:04.306831 23407 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:04.307009 23407 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.67
I20260812 08:03:04.311102 23407 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.050000000000000003
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.67:40205
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data/info.pb
--webserver_interface=127.18.42.67
--webserver_port=39753
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.23407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:04.312250 23407 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:04.313380 23407 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:03:04.316854 23407 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:03:04.321040 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.322074 23415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:04.322119 23417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:04.322659 23407 server_base.cc:1061] running on GCE node
W20260812 08:03:04.322685 23413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:04.323112 23407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:04.323829 23407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:04.325004 23407 hybrid_clock.cc:648] HybridClock initialized: now 1786521784324977 us; error 43 us; skew 500 ppm
I20260812 08:03:04.327293 23407 webserver.cc:533] Webserver started at http://127.18.42.67:39753/ using document root <none> and password file <none>
I20260812 08:03:04.328118 23407 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:04.328186 23407 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
W20260812 08:03:04.328161 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:03:04.332412 23407 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.001s	sys 0.001s
I20260812 08:03:04.334611 23429 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:04.335719 23407 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:03:04.335850 23407 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal
uuid: "294e0e1970eb4e58bed426de35babb6b"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:03:04.336256 23407 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260812 08:03:04.338723 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:04.341722 21763 tablet_service.cc:3080] error setting up scanner a579fd239929478388b89e5cfd6e4347 with request: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: tablet hasn't heard from leader or there hasn't been a stable leader for 0.219s (maximum allowed 0.150s): new_scan_request { tablet_id: "50b14eb091f7476d9d7682e503745c93" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7317593227800997888 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\337\323\360\323\006\"3\n\005slave\022*\n f676baf4d90e47909c86ef51b3260926\020\001\030\001 \001(\001" signature: "<redacted>""3\272\377\3278\0220\236\'\036\225\3266\312\023\204L,\347\252\335\306\257\345\221\275\300\207\352\031\323W\277_ \003\305r\241\377 \344\323\265\031\320\027B*d\272\350xy\316h\030\272\227\026\236s\235\247" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "d4716c3919d347228becbc6c15e47625"
W20260812 08:03:04.343128 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:04.343271 21763 tablet_service.cc:3080] error setting up scanner f2c30ea4c0e44af393e230271daafef6 with request: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: tablet hasn't heard from leader or there hasn't been a stable leader for 0.221s (maximum allowed 0.150s): new_scan_request { tablet_id: "50b14eb091f7476d9d7682e503745c93" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7317593227800997888 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\337\323\360\323\006\"3\n\005slave\022*\n f676baf4d90e47909c86ef51b3260926\020\001\030\001 \001(\001" signature: "<redacted>""3\272\377\3278\0220\236\'\036\225\3266\312\023\204L,\347\252\335\306\257\345\221\275\300\207\352\031\323W\277_ \003\305r\241\377 \344\323\265\031\320\027B*d\272\350xy\316h\030\272\227\026\236s\235\247" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "5ff8540deebd45bda469c5dfbf71d1a2"
W20260812 08:03:04.350859 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:04.363859 22952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44224: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.372108 22310 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.372143 22306 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
I20260812 08:03:04.374838 23407 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:04.375480 23407 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:04.375630 23407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:04.376235 23407 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:04.377571 23443 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:03:04.381606 23407 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:04.381676 23407 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:03:04.381757 23407 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:04.383287 23407 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:04.383360 23407 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:03:04.383674 23443 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Bootstrap starting.
W20260812 08:03:04.385649 22306 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.389423 21763 tablet_service.cc:3080] error setting up scanner db407486de234a43a1673ef86658be95 with request: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: tablet hasn't heard from leader or there hasn't been a stable leader for 0.267s (maximum allowed 0.150s): new_scan_request { tablet_id: "50b14eb091f7476d9d7682e503745c93" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7317593227800997888 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\337\323\360\323\006\"3\n\005slave\022*\n f676baf4d90e47909c86ef51b3260926\020\001\030\001 \001(\001" signature: "<redacted>""3\272\377\3278\0220\236\'\036\225\3266\312\023\204L,\347\252\335\306\257\345\221\275\300\207\352\031\323W\277_ \003\305r\241\377 \344\323\265\031\320\027B*d\272\350xy\316h\030\272\227\026\236s\235\247" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "5ff8540deebd45bda469c5dfbf71d1a2"
W20260812 08:03:04.394620 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:03:04.396528 23443 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:04.396615 23407 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.67:40205
I20260812 08:03:04.396711 23613 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.67:40205 every 8 connection(s)
I20260812 08:03:04.397825 23407 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data/info.pb
I20260812 08:03:04.399991 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 23407
I20260812 08:03:04.406956 23615 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:03:04.407228 23615 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:04.407881 23615 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:03:04.409497 22492 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 294e0e1970eb4e58bed426de35babb6b)
I20260812 08:03:04.409655 22492 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
W20260812 08:03:04.410054 23222 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
I20260812 08:03:04.410384 22492 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:04.410735 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:04.410809 18661 ts_manager.cc:194] Re-registered known tserver with Master: 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205)
I20260812 08:03:04.411739 18661 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.67:42017
I20260812 08:03:04.411885 23017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:04.412014 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:04.412076 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:04.412361 23283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:04.412582 23283 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7def2bd3a74420d9deec19b32f541ec in term 11.
I20260812 08:03:04.413251 19502 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: b7def2bd3a74420d9deec19b32f541ec; no voters: 285e078e6cfb46ae9dbe57c20be666d5, 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d, a5813459b1d94e089ffbe751f4310de7
I20260812 08:03:04.413509 22492 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260812 08:03:04.413679 23161 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 294e0e1970eb4e58bed426de35babb6b)
I20260812 08:03:04.413800 23161 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:04.415058 22832 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 294e0e1970eb4e58bed426de35babb6b)
I20260812 08:03:04.415158 22832 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:04.415567 23161 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:04.415808 22832 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:04.417279 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:04.417410 19620 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 12.
I20260812 08:03:04.417691 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:04.417891 23404 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 294e0e1970eb4e58bed426de35babb6b)
I20260812 08:03:04.418011 23404 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:04.418588 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:04.418725 22069 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 12.
I20260812 08:03:04.418766 23404 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
W20260812 08:03:04.420477 23222 tablet_service.cc:3080] error setting up scanner 52a4a5c6bd9244ad9d85fa4f5119fe72 with request: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: tablet hasn't heard from leader or there hasn't been a stable leader for 0.268s (maximum allowed 0.150s): new_scan_request { tablet_id: "50b14eb091f7476d9d7682e503745c93" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7317593227800997888 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\337\323\360\323\006\"3\n\005slave\022*\n f676baf4d90e47909c86ef51b3260926\020\001\030\001 \001(\001" signature: "<redacted>""3\272\377\3278\0220\236\'\036\225\3266\312\023\204L,\347\252\335\306\257\345\221\275\300\207\352\031\323W\277_ \003\305r\241\377 \344\323\265\031\320\027B*d\272\350xy\316h\030\272\227\026\236s\235\247" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "d4716c3919d347228becbc6c15e47625"
I20260812 08:03:04.421535 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:04.421881 23016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:04.422020 23016 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 12.
I20260812 08:03:04.422461 23283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:04.422592 23283 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 11.
I20260812 08:03:04.423282 22534 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 294e0e1970eb4e58bed426de35babb6b)
I20260812 08:03:04.423384 22534 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:04.424013 22534 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:04.425227 19619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:04.425364 19619 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 12.
I20260812 08:03:04.425625 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:04.425781 22384 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 12.
I20260812 08:03:04.426555 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:04.426678 19620 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 12.
I20260812 08:03:04.427026 23017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:04.427160 23017 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 12.
I20260812 08:03:04.426141 23538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:04.427423 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:04.427548 22384 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 12.
I20260812 08:03:04.427675 23283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:04.427789 23283 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 11.
I20260812 08:03:04.428179 22269 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 285e078e6cfb46ae9dbe57c20be666d5, 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec; no voters: a5813459b1d94e089ffbe751f4310de7
I20260812 08:03:04.428220 21680 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 38681904c36540a4876519ec5ffac6b6, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec; no voters: 
W20260812 08:03:04.428364 22268 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 13 pre-election: Tablet error from VoteRequest() call to peer 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:04.428406 22534 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260812 08:03:04.428403 22832 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260812 08:03:04.428453 22534 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 294e0e1970eb4e58bed426de35babb6b)
I20260812 08:03:04.428465 22832 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 294e0e1970eb4e58bed426de35babb6b)
I20260812 08:03:04.428498 22534 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 12 FOLLOWER]: Advancing to term 13
I20260812 08:03:04.428511 22832 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 12 FOLLOWER]: Advancing to term 13
I20260812 08:03:04.429654 22534 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:04.429674 22832 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:04.430254 22534 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 13 election: Requested vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:04.430385 22832 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 13 election: Requested vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:04.431336 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7"
I20260812 08:03:04.431485 21840 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 38681904c36540a4876519ec5ffac6b6 in current term 13: Already voted for candidate a5813459b1d94e089ffbe751f4310de7 in this term.
I20260812 08:03:04.431579 19619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec"
I20260812 08:03:04.431694 19619 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 12 FOLLOWER]: Advancing to term 13
I20260812 08:03:04.431807 23283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf"
I20260812 08:03:04.431934 23283 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 11 FOLLOWER]: Advancing to term 13
I20260812 08:03:04.432842 19619 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 13.
I20260812 08:03:04.433218 21956 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec; no voters: 
I20260812 08:03:04.433662 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:04.433800 22069 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 12.
I20260812 08:03:04.433907 22059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d"
I20260812 08:03:04.434032 22059 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 12 FOLLOWER]: Advancing to term 13
I20260812 08:03:04.434469 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec"
I20260812 08:03:04.434599 19620 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a5813459b1d94e089ffbe751f4310de7 in current term 13: Already voted for candidate 38681904c36540a4876519ec5ffac6b6 in this term.
I20260812 08:03:04.435134 22059 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 13.
I20260812 08:03:04.435485 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:04.435973 23019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5"
I20260812 08:03:04.436044 23017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:04.436121 23019 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 12 FOLLOWER]: Advancing to term 13
I20260812 08:03:04.437367 23533 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b"
I20260812 08:03:04.437494 23019 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 13.
I20260812 08:03:04.435986 23016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5"
I20260812 08:03:04.437799 23016 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a5813459b1d94e089ffbe751f4310de7 in current term 13: Already voted for candidate 38681904c36540a4876519ec5ffac6b6 in this term.
W20260812 08:03:04.437842 21679 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 13 election: Tablet error from VoteRequest() call to peer 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:04.437973 23283 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 13.
I20260812 08:03:04.438084 23534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:04.438360 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6"
I20260812 08:03:04.438526 22384 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a5813459b1d94e089ffbe751f4310de7 in current term 13: Already voted for candidate 38681904c36540a4876519ec5ffac6b6 in this term.
W20260812 08:03:04.438619 21679 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 13 pre-election: Tablet error from VoteRequest() call to peer 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:04.439730 23404 raft_consensus.cc:2764] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260812 08:03:04.440164 21680 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 5b4c3ad48b794617893c88a077be1b8d, a5813459b1d94e089ffbe751f4310de7; no voters: 285e078e6cfb46ae9dbe57c20be666d5, 294e0e1970eb4e58bed426de35babb6b, 38681904c36540a4876519ec5ffac6b6, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:04.440578 22059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d"
I20260812 08:03:04.440743 22059 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 38681904c36540a4876519ec5ffac6b6 in current term 13: Already voted for candidate a5813459b1d94e089ffbe751f4310de7 in this term.
I20260812 08:03:04.441256 23535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
W20260812 08:03:04.441689 21955 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 13 pre-election: Tablet error from VoteRequest() call to peer 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:04.442893 23542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
W20260812 08:03:04.443338 19501 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 13 pre-election: Tablet error from VoteRequest() call to peer 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:04.443567 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf"
I20260812 08:03:04.443698 23535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:04.443768 23284 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 38681904c36540a4876519ec5ffac6b6 in current term 13: Already voted for candidate a5813459b1d94e089ffbe751f4310de7 in this term.
I20260812 08:03:04.443567 23283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:04.444078 23283 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38681904c36540a4876519ec5ffac6b6 in current term 13: Already voted for candidate a5813459b1d94e089ffbe751f4310de7 in this term.
W20260812 08:03:04.444075 22918 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 13 pre-election: Tablet error from VoteRequest() call to peer 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:04.444281 22534 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260812 08:03:04.444581 23538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b"
W20260812 08:03:04.444931 22268 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 13 election: Tablet error from VoteRequest() call to peer 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:04.445019 22268 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 285e078e6cfb46ae9dbe57c20be666d5, 38681904c36540a4876519ec5ffac6b6, b7def2bd3a74420d9deec19b32f541ec; no voters: 0e10b84812514a76ab7225dfbf9329bf, 294e0e1970eb4e58bed426de35babb6b, 5b4c3ad48b794617893c88a077be1b8d, a5813459b1d94e089ffbe751f4310de7
I20260812 08:03:04.445355 22832 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260812 08:03:04.448251 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:04.448400 19620 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 285e078e6cfb46ae9dbe57c20be666d5 in current term 13: Already voted for candidate 38681904c36540a4876519ec5ffac6b6 in this term.
I20260812 08:03:04.449388 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:04.449579 22384 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 285e078e6cfb46ae9dbe57c20be666d5 in current term 13: Already voted for candidate 38681904c36540a4876519ec5ffac6b6 in this term.
I20260812 08:03:04.449980 22918 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 285e078e6cfb46ae9dbe57c20be666d5; no voters: 0e10b84812514a76ab7225dfbf9329bf, 294e0e1970eb4e58bed426de35babb6b, 38681904c36540a4876519ec5ffac6b6, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:04.450284 23161 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260812 08:03:04.453070 22059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:04.453084 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 13 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:04.453248 21840 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 285e078e6cfb46ae9dbe57c20be666d5 in current term 13: Already voted for candidate a5813459b1d94e089ffbe751f4310de7 in this term.
I20260812 08:03:04.453261 22059 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 285e078e6cfb46ae9dbe57c20be666d5 in current term 13: Already voted for candidate a5813459b1d94e089ffbe751f4310de7 in this term.
I20260812 08:03:04.456270 23443 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Bootstrap replayed 1/1 log segments. Stats: ops{read=81 overwritten=0 applied=80 ignored=0} inserts{seen=73 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260812 08:03:04.456723 23443 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Bootstrap complete.
W20260812 08:03:04.456918 23443 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Injecting 53ms of latency in SegmentAllocator::Sync()
W20260812 08:03:04.467092 22952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44224: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.482875 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:04.494086 23222 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:04.495365 23222 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:04.495639 23220 tablet_service.cc:3080] error setting up scanner e08a8980849344ca87bdaa58c07bd3b2 with request: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: tablet hasn't heard from leader or there hasn't been a stable leader for 0.343s (maximum allowed 0.150s): new_scan_request { tablet_id: "50b14eb091f7476d9d7682e503745c93" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7317593227800997888 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\337\323\360\323\006\"3\n\005slave\022*\n f676baf4d90e47909c86ef51b3260926\020\001\030\001 \001(\001" signature: "<redacted>""3\272\377\3278\0220\236\'\036\225\3266\312\023\204L,\347\252\335\306\257\345\221\275\300\207\352\031\323W\277_ \003\305r\241\377 \344\323\265\031\320\027B*d\272\350xy\316h\030\272\227\026\236s\235\247" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "5ff8540deebd45bda469c5dfbf71d1a2"
I20260812 08:03:04.510432 23443 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Time spent bootstrapping tablet: real 0.127s	user 0.063s	sys 0.006s
I20260812 08:03:04.512331 23443 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 12 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:04.514093 23443 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 12 FOLLOWER]: Becoming Follower/Learner. State: Replica: 294e0e1970eb4e58bed426de35babb6b, State: Initialized, Role: FOLLOWER
I20260812 08:03:04.514678 23443 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 80, Last appended: 12.81, Last appended by leader: 81, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:04.515377 23615 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
I20260812 08:03:04.515650 23443 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
W20260812 08:03:04.516776 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:04.520561 23449 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53158: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:04.521588 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:04.547279 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:04.550026 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.552543 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.561697 23220 tablet_service.cc:3080] error setting up scanner 170fef628a5c445eb7d682d430ade717 with request: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: tablet hasn't heard from leader or there hasn't been a stable leader for 0.409s (maximum allowed 0.150s): new_scan_request { tablet_id: "50b14eb091f7476d9d7682e503745c93" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7317593227800997888 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\337\323\360\323\006\"3\n\005slave\022*\n f676baf4d90e47909c86ef51b3260926\020\001\030\001 \001(\001" signature: "<redacted>""3\272\377\3278\0220\236\'\036\225\3266\312\023\204L,\347\252\335\306\257\345\221\275\300\207\352\031\323W\277_ \003\305r\241\377 \344\323\265\031\320\027B*d\272\350xy\316h\030\272\227\026\236s\235\247" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "d4716c3919d347228becbc6c15e47625"
W20260812 08:03:04.577471 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.581266 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:04.588357 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:04.604774 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:04.610472 22952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44224: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.628881 22952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44224: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.637209 22952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44224: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.641947 22306 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.668311 22306 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.670272 22306 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.677313 23449 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53158: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:04.703320 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:04.708830 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:04.739171 23449 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53158: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:04.742811 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.753732 23220 tablet_service.cc:3080] error setting up scanner 982c467a4b7e483aac67b5a7b933b71f with request: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: tablet hasn't heard from leader or there hasn't been a stable leader for 0.601s (maximum allowed 0.150s): new_scan_request { tablet_id: "50b14eb091f7476d9d7682e503745c93" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7317593227800997888 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\337\323\360\323\006\"3\n\005slave\022*\n f676baf4d90e47909c86ef51b3260926\020\001\030\001 \001(\001" signature: "<redacted>""3\272\377\3278\0220\236\'\036\225\3266\312\023\204L,\347\252\335\306\257\345\221\275\300\207\352\031\323W\277_ \003\305r\241\377 \344\323\265\031\320\027B*d\272\350xy\316h\030\272\227\026\236s\235\247" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "5ff8540deebd45bda469c5dfbf71d1a2"
I20260812 08:03:04.753834 23652 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:04.754082 23652 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:04.755757 23652 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:04.762115 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 81 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:04.762223 22059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 81 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:04.762360 21840 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 294e0e1970eb4e58bed426de35babb6b in current term 13: Already voted for candidate a5813459b1d94e089ffbe751f4310de7 in this term.
I20260812 08:03:04.762440 22059 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 294e0e1970eb4e58bed426de35babb6b in current term 13: Already voted for candidate a5813459b1d94e089ffbe751f4310de7 in this term.
I20260812 08:03:04.763422 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 81 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:04.763607 22384 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 294e0e1970eb4e58bed426de35babb6b in current term 13: Already voted for candidate 38681904c36540a4876519ec5ffac6b6 in this term.
I20260812 08:03:04.764402 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 81 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:04.764575 19620 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 294e0e1970eb4e58bed426de35babb6b in current term 13: Already voted for candidate 38681904c36540a4876519ec5ffac6b6 in this term.
I20260812 08:03:04.764950 23434 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 294e0e1970eb4e58bed426de35babb6b; no voters: 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:04.765264 23652 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 12 FOLLOWER]: Advancing to term 13
I20260812 08:03:04.765384 23161 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:04.765496 23161 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:04.766346 23161 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:04.766528 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 14 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:04.766664 19620 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 13.
I20260812 08:03:04.766901 23283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 81 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:04.767089 23283 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 294e0e1970eb4e58bed426de35babb6b in current term 13: Already voted for candidate a5813459b1d94e089ffbe751f4310de7 in this term.
I20260812 08:03:04.767062 23016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 81 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:04.767201 23016 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 294e0e1970eb4e58bed426de35babb6b in current term 13: Already voted for candidate 38681904c36540a4876519ec5ffac6b6 in this term.
I20260812 08:03:04.767441 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 14 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:04.767575 23284 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 13.
I20260812 08:03:04.767688 23652 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260812 08:03:04.767820 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 14 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:04.767973 21840 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 13.
I20260812 08:03:04.768029 23538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 14 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:04.767990 22059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 14 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:04.768149 22059 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 13.
I20260812 08:03:04.768205 23538 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 285e078e6cfb46ae9dbe57c20be666d5 for term 14 because replica has last-logged OpId of term: 12 index: 81, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 80.
I20260812 08:03:04.768417 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 14 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:04.768499 22920 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 285e078e6cfb46ae9dbe57c20be666d5, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec; no voters: 
I20260812 08:03:04.768596 22384 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 13.
I20260812 08:03:04.768754 23161 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260812 08:03:04.768846 23161 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:03:04.768935 23161 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 13 FOLLOWER]: Advancing to term 14
I20260812 08:03:04.769987 23161 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:04.770738 23161 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 14 election: Requested vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:04.770933 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 14 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7"
W20260812 08:03:04.771062 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:03:04.771068 21840 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 13 FOLLOWER]: Advancing to term 14
I20260812 08:03:04.771210 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 14 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf"
I20260812 08:03:04.771332 23284 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 13 FOLLOWER]: Advancing to term 14
I20260812 08:03:04.771739 23538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 14 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b"
I20260812 08:03:04.771797 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 14 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6"
I20260812 08:03:04.771865 23538 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 13 FOLLOWER]: Advancing to term 14
I20260812 08:03:04.771935 22384 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 13 FOLLOWER]: Advancing to term 14
I20260812 08:03:04.772322 21840 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 14.
I20260812 08:03:04.772523 23284 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 14.
I20260812 08:03:04.772986 23538 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 285e078e6cfb46ae9dbe57c20be666d5 for term 14 because replica has last-logged OpId of term: 12 index: 81, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 80.
I20260812 08:03:04.773138 22384 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 14.
I20260812 08:03:04.773378 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 14 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec"
I20260812 08:03:04.773444 22918 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 285e078e6cfb46ae9dbe57c20be666d5, 38681904c36540a4876519ec5ffac6b6, a5813459b1d94e089ffbe751f4310de7; no voters: 294e0e1970eb4e58bed426de35babb6b
I20260812 08:03:04.773479 22059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 14 candidate_status { last_received { term: 12 index: 80 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d"
I20260812 08:03:04.773546 19620 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 13 FOLLOWER]: Advancing to term 14
I20260812 08:03:04.773641 22059 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 13 FOLLOWER]: Advancing to term 14
I20260812 08:03:04.773648 23161 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 14 FOLLOWER]: Leader election won for term 14
I20260812 08:03:04.773885 23161 raft_consensus.cc:697] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 14 LEADER]: Becoming Leader. State: Replica: 285e078e6cfb46ae9dbe57c20be666d5, State: Running, Role: LEADER
I20260812 08:03:04.774201 23161 consensus_queue.cc:237] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 80, Committed index: 80, Last appended: 12.80, Last appended by leader: 80, Current term: 14, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:04.774652 22059 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 14.
I20260812 08:03:04.774649 19620 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 14.
W20260812 08:03:04.775851 23377 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 57ms of latency in SegmentAllocator::Sync()
I20260812 08:03:04.777145 18661 catalog_manager.cc:5719] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 reported cstate change: term changed from 12 to 14, leader changed from 294e0e1970eb4e58bed426de35babb6b (127.18.42.67) to 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69). New cstate: current_term: 14 leader_uuid: "285e078e6cfb46ae9dbe57c20be666d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } health_report { overall_health: UNKNOWN } } }
W20260812 08:03:04.778939 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:04.803781 23220 tablet_service.cc:3080] error setting up scanner 714d4439e1384d399b93d5150e873032 with request: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: tablet hasn't heard from leader or there hasn't been a stable leader for 0.651s (maximum allowed 0.150s): new_scan_request { tablet_id: "50b14eb091f7476d9d7682e503745c93" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7317593227800997888 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\337\323\360\323\006\"3\n\005slave\022*\n f676baf4d90e47909c86ef51b3260926\020\001\030\001 \001(\001" signature: "<redacted>""3\272\377\3278\0220\236\'\036\225\3266\312\023\204L,\347\252\335\306\257\345\221\275\300\207\352\031\323W\277_ \003\305r\241\377 \344\323\265\031\320\027B*d\272\350xy\316h\030\272\227\026\236s\235\247" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "d4716c3919d347228becbc6c15e47625"
W20260812 08:03:04.807366 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:04.816035 23161 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260812 08:03:04.816116 23161 fault_injection.cc:44] THIS SERVER MAY CRASH!
I20260812 08:03:04.816962 22059 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 14 FOLLOWER]: Refusing update from remote peer 285e078e6cfb46ae9dbe57c20be666d5: Log matching property violated. Preceding OpId in replica: term: 12 index: 80. Preceding OpId from leader: term: 14 index: 82. (index mismatch)
I20260812 08:03:04.817186 23284 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 14 FOLLOWER]: Refusing update from remote peer 285e078e6cfb46ae9dbe57c20be666d5: Log matching property violated. Preceding OpId in replica: term: 11 index: 81. Preceding OpId from leader: term: 14 index: 82. (index mismatch)
I20260812 08:03:04.817404 23538 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 14 FOLLOWER]: Refusing update from remote peer 285e078e6cfb46ae9dbe57c20be666d5: Log matching property violated. Preceding OpId in replica: term: 12 index: 81. Preceding OpId from leader: term: 14 index: 82. (index mismatch)
I20260812 08:03:04.817431 22384 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 14 FOLLOWER]: Refusing update from remote peer 285e078e6cfb46ae9dbe57c20be666d5: Log matching property violated. Preceding OpId in replica: term: 12 index: 80. Preceding OpId from leader: term: 14 index: 82. (index mismatch)
I20260812 08:03:04.817499 19620 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 14 FOLLOWER]: Refusing update from remote peer 285e078e6cfb46ae9dbe57c20be666d5: Log matching property violated. Preceding OpId in replica: term: 12 index: 80. Preceding OpId from leader: term: 14 index: 82. (index mismatch)
I20260812 08:03:04.817971 21840 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 14 FOLLOWER]: Refusing update from remote peer 285e078e6cfb46ae9dbe57c20be666d5: Log matching property violated. Preceding OpId in replica: term: 12 index: 80. Preceding OpId from leader: term: 14 index: 82. (index mismatch)
I20260812 08:03:04.817950 23669 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 81, Last known committed idx: 77, Time since last communication: 0.000s
I20260812 08:03:04.818120 23669 consensus_queue.cc:1243] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [LEADER]: Peer 0e10b84812514a76ab7225dfbf9329bf log is divergent from this leader: its last log entry 11.81 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 77
I20260812 08:03:04.818229 23668 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 81, Last known committed idx: 80, Time since last communication: 0.000s
I20260812 08:03:04.818410 23671 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 81, Last known committed idx: 80, Time since last communication: 0.000s
I20260812 08:03:04.818542 23161 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 81, Last known committed idx: 80, Time since last communication: 0.000s
I20260812 08:03:04.818800 23661 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 81, Last known committed idx: 80, Time since last communication: 0.000s
I20260812 08:03:04.818885 23661 consensus_queue.cc:1243] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [LEADER]: Peer 294e0e1970eb4e58bed426de35babb6b log is divergent from this leader: its last log entry 12.81 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 80
I20260812 08:03:04.819399 23670 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 81, Last known committed idx: 63, Time since last communication: 0.000s
E20260812 08:03:04.819730 23670 fault_injection.cc:59] injecting fault for kudu (pid 22888): FLAGS_fault_crash_on_leader_request_fraction (process will exit)
I20260812 08:03:04.819932 23284 raft_consensus.cc:1217] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 14 FOLLOWER]: Deduplicated request from leader. Original: 11.77->[11.78-14.82]   Dedup: 11.78->[12.79-14.82]
I20260812 08:03:04.820094 23284 pending_rounds.cc:85] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Aborting all ops after (but not including) 78
I20260812 08:03:04.820155 23284 pending_rounds.cc:107] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Aborting uncommitted WRITE_OP operation due to leader change: 11.79
I20260812 08:03:04.820706 23284 pending_rounds.cc:107] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Aborting uncommitted WRITE_OP operation due to leader change: 11.80
I20260812 08:03:04.820969 23284 pending_rounds.cc:107] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Aborting uncommitted WRITE_OP operation due to leader change: 11.81
I20260812 08:03:04.838461 23538 pending_rounds.cc:85] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Aborting all ops after (but not including) 80
W20260812 08:03:04.838457 22504 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 81ms of latency in SegmentAllocator::Sync()
I20260812 08:03:04.838577 23538 pending_rounds.cc:107] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Aborting uncommitted WRITE_OP operation due to leader change: 12.81
W20260812 08:03:04.842257 23687 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Injecting 15ms of latency in SegmentAllocator::Sync()
I20260812 08:03:04.843321 23682 mvcc.cc:204] Tried to move back new op lower bound from 7317593227673690112 to 7317593227472404480. Current Snapshot: MvccSnapshot[applied={T|T < 7317593227673690112 or (T in {7317593227673690112})}]
W20260812 08:03:04.851039 23431 connection.cc:573] server connection from 127.18.42.69:52977 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 08:03:04.851030 22266 connection.cc:573] server connection from 127.18.42.69:45079 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 08:03:04.851434 21956 connection.cc:573] server connection from 127.18.42.69:54643 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 08:03:04.851500 20378 connection.cc:573] client connection to 127.18.42.69:34819 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 08:03:04.851599 20378 meta_cache.cc:302] tablet 50b14eb091f7476d9d7682e503745c93: replica 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819) has failed: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 08:03:04.853960 18601 ts_itest-base.cc:575] TS 4 appears to have crashed. Restarting.
I20260812 08:03:04.854398 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.69:34819
--local_ip_for_outbound_sockets=127.18.42.69
--tserver_master_addrs=127.18.42.126:35213
--webserver_port=40235
--webserver_interface=127.18.42.69
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.05
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
W20260812 08:03:04.858079 23186 connection.cc:466] server connection from 127.18.42.69:37471 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.18.42.69:37471 (request call id 4) could send its response
W20260812 08:03:04.866827 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:04.868479 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:04.886166 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:04.886678 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:04.897009 22306 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.902299 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:04.920258 21679 connection.cc:466] server connection from 127.18.42.69:52569 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.18.42.69:52569 (request call id 4) could send its response
W20260812 08:03:04.927306 22306 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.930766 22306 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56854: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:04.932565 23449 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53158: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:04.938350 23449 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53158: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:04.966399 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:04.967267 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:04.971158 23691 flags.cc:432] Enabled unsafe flag: --fault_crash_on_leader_request_fraction=0.050000000000000003
W20260812 08:03:04.971371 23691 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:03:04.971390 23691 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:03:04.971406 23691 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:03:04.971449 23691 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:04.971470 23691 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:04.971514 23691 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:04.972538 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:04.974704 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:04.975584 23691 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:04.975692 23691 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.69
I20260812 08:03:04.979821 23691 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.050000000000000003
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.69:34819
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data/info.pb
--webserver_interface=127.18.42.69
--webserver_port=40235
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.23691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.69

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:04.980918 23691 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:04.982146 23691 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:03:04.984890 23691 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:03:04.988770 23703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:04.988724 23704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:04.988721 23707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:04.988971 23691 server_base.cc:1061] running on GCE node
I20260812 08:03:04.989555 23691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:04.990100 23691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:04.991276 23691 hybrid_clock.cc:648] HybridClock initialized: now 1786521784991257 us; error 34 us; skew 500 ppm
I20260812 08:03:04.993109 23691 webserver.cc:533] Webserver started at http://127.18.42.69:40235/ using document root <none> and password file <none>
I20260812 08:03:04.993773 23691 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:04.993865 23691 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:04.997236 23691 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:03:04.999370 23716 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:05.000463 23691 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260812 08:03:05.000639 23691 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
uuid: "285e078e6cfb46ae9dbe57c20be666d5"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:03:05.001127 23691 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:05.011180 23691 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:05.011742 23691 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:05.011885 23691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:05.012436 23691 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:05.013751 23727 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260812 08:03:05.014358 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:05.016835 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:03:05.017374 23691 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:05.017475 23691 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.001s	sys 0.000s
I20260812 08:03:05.017580 23691 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:05.019101 23691 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:05.019174 23691 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:03:05.019546 23727 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Bootstrap starting.
I20260812 08:03:05.030450 23691 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.69:34819
I20260812 08:03:05.030526 23893 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.69:34819 every 8 connection(s)
I20260812 08:03:05.031517 23691 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data/info.pb
I20260812 08:03:05.032742 23727 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Log is configured to *not* fsync() on all Append() calls
W20260812 08:03:05.037304 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:03:05.040570 23894 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:03:05.040812 23894 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:05.041448 23894 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:03:05.041786 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 23691
I20260812 08:03:05.043321 18664 ts_manager.cc:194] Re-registered known tserver with Master: 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819)
I20260812 08:03:05.044279 18664 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.69:47947
W20260812 08:03:05.047866 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:03:05.079607 23727 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Bootstrap replayed 1/1 log segments. Stats: ops{read=83 overwritten=2 applied=80 ignored=0} inserts{seen=73 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260812 08:03:05.080170 23727 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Bootstrap complete.
W20260812 08:03:05.080372 23727 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 2ms of latency in SegmentAllocator::Sync()
I20260812 08:03:05.082785 23727 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Time spent bootstrapping tablet: real 0.064s	user 0.061s	sys 0.000s
I20260812 08:03:05.084952 23727 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 14 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:05.085464 23727 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 14 FOLLOWER]: Becoming Follower/Learner. State: Replica: 285e078e6cfb46ae9dbe57c20be666d5, State: Initialized, Role: FOLLOWER
I20260812 08:03:05.085991 23727 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 80, Last appended: 14.81, Last appended by leader: 81, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:05.086694 23894 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
I20260812 08:03:05.086862 23727 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Time spent starting tablet: real 0.004s	user 0.000s	sys 0.004s
W20260812 08:03:05.091203 23763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44252: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:05.099944 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:05.111264 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:05.112388 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:03:05.119859 22832 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 285e078e6cfb46ae9dbe57c20be666d5)
I20260812 08:03:05.119926 23404 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 285e078e6cfb46ae9dbe57c20be666d5)
I20260812 08:03:05.120045 22832 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:05.120096 23404 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:05.120820 22832 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:05.120971 23404 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:05.121634 21840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:05.121734 23283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:05.122094 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:05.122095 23538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:05.122453 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:05.122570 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:05.122710 19620 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 14.
I20260812 08:03:05.122788 23535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:05.123085 19619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:05.123208 19619 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 14.
I20260812 08:03:05.123312 23690 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 285e078e6cfb46ae9dbe57c20be666d5)
I20260812 08:03:05.123435 23690 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:05.123521 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:05.123631 22384 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 14.
I20260812 08:03:05.123652 22059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:05.123791 22059 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 14.
I20260812 08:03:05.125207 23690 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785)
I20260812 08:03:05.129473 23819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:05.129642 23817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:05.129832 23819 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 14.
I20260812 08:03:05.130188 21956 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec; no voters: 0e10b84812514a76ab7225dfbf9329bf, 285e078e6cfb46ae9dbe57c20be666d5, 294e0e1970eb4e58bed426de35babb6b, a5813459b1d94e089ffbe751f4310de7
I20260812 08:03:05.130228 22269 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 285e078e6cfb46ae9dbe57c20be666d5, 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec; no voters: 0e10b84812514a76ab7225dfbf9329bf, 294e0e1970eb4e58bed426de35babb6b, a5813459b1d94e089ffbe751f4310de7
I20260812 08:03:05.130488 23404 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260812 08:03:05.130486 22832 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260812 08:03:05.130541 22832 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 285e078e6cfb46ae9dbe57c20be666d5)
I20260812 08:03:05.130585 22832 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 14 FOLLOWER]: Advancing to term 15
I20260812 08:03:05.131770 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:05.131762 22832 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:05.132470 22832 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 15 election: Requested vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
W20260812 08:03:05.133644 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:03:05.134716 19619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec"
I20260812 08:03:05.134835 19619 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 14 FOLLOWER]: Advancing to term 15
I20260812 08:03:05.135051 22059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d"
I20260812 08:03:05.135155 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:05.135365 23535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b"
I20260812 08:03:05.135182 22059 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 14 FOLLOWER]: Advancing to term 15
I20260812 08:03:05.135977 19619 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 15.
I20260812 08:03:05.136580 23819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:05.136730 22059 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 15.
I20260812 08:03:05.136735 23819 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e10b84812514a76ab7225dfbf9329bf in term 14.
I20260812 08:03:05.136891 23817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5"
I20260812 08:03:05.137053 23817 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 14 FOLLOWER]: Advancing to term 15
I20260812 08:03:05.137123 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:05.137276 19620 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e10b84812514a76ab7225dfbf9329bf in current term 15: Already voted for candidate 38681904c36540a4876519ec5ffac6b6 in this term.
I20260812 08:03:05.137492 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:05.137665 22384 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e10b84812514a76ab7225dfbf9329bf in current term 15: Already voted for candidate 38681904c36540a4876519ec5ffac6b6 in this term.
I20260812 08:03:05.137691 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf"
I20260812 08:03:05.137835 23284 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 14 FOLLOWER]: Advancing to term 15
I20260812 08:03:05.137885 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7"
I20260812 08:03:05.138008 21839 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 14 FOLLOWER]: Advancing to term 15
I20260812 08:03:05.139030 23284 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 15.
I20260812 08:03:05.139130 21839 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 15.
I20260812 08:03:05.139359 23817 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 15.
I20260812 08:03:05.139639 23186 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 285e078e6cfb46ae9dbe57c20be666d5; no voters: 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:05.139711 22269 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 285e078e6cfb46ae9dbe57c20be666d5, 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec; no voters: 294e0e1970eb4e58bed426de35babb6b
W20260812 08:03:05.139814 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:03:05.139892 23690 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260812 08:03:05.139889 22832 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 15 FOLLOWER]: Leader election won for term 15
I20260812 08:03:05.140110 22832 raft_consensus.cc:697] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 15 LEADER]: Becoming Leader. State: Replica: 38681904c36540a4876519ec5ffac6b6, State: Running, Role: LEADER
I20260812 08:03:05.140458 22832 consensus_queue.cc:237] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 80, Committed index: 80, Last appended: 14.82, Last appended by leader: 82, Current term: 15, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:05.140769 23652 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 285e078e6cfb46ae9dbe57c20be666d5)
I20260812 08:03:05.140870 23652 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:05.141600 23652 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:05.141758 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:05.141912 19620 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 294e0e1970eb4e58bed426de35babb6b in current term 15: Already voted for candidate 38681904c36540a4876519ec5ffac6b6 in this term.
W20260812 08:03:05.142341 22508 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Injecting 17ms of latency in SegmentAllocator::Sync()
I20260812 08:03:05.143049 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:05.143191 23284 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 294e0e1970eb4e58bed426de35babb6b in current term 15: Already voted for candidate 38681904c36540a4876519ec5ffac6b6 in this term.
I20260812 08:03:05.143685 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:05.143836 21839 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 294e0e1970eb4e58bed426de35babb6b in current term 15: Already voted for candidate 38681904c36540a4876519ec5ffac6b6 in this term.
I20260812 08:03:05.143855 23535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:05.143860 22059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:05.144029 22059 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 294e0e1970eb4e58bed426de35babb6b in current term 15: Already voted for candidate 38681904c36540a4876519ec5ffac6b6 in this term.
I20260812 08:03:05.144002 23535 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e10b84812514a76ab7225dfbf9329bf in term 14.
I20260812 08:03:05.144066 22384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:05.144068 18664 catalog_manager.cc:5719] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 reported cstate change: term changed from 14 to 15, leader changed from 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69) to 38681904c36540a4876519ec5ffac6b6 (127.18.42.70). New cstate: current_term: 15 leader_uuid: "38681904c36540a4876519ec5ffac6b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } health_report { overall_health: UNKNOWN } } }
I20260812 08:03:05.144754 23434 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 294e0e1970eb4e58bed426de35babb6b; no voters: 0e10b84812514a76ab7225dfbf9329bf, 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:05.144945 23652 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 14 FOLLOWER]: Advancing to term 15
I20260812 08:03:05.146134 23652 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260812 08:03:05.147312 23817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 82 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:05.147511 23817 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 294e0e1970eb4e58bed426de35babb6b in current term 15: Already voted for candidate 38681904c36540a4876519ec5ffac6b6 in this term.
W20260812 08:03:05.153683 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:05.156721 23449 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53158: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:05.172021 23220 tablet_service.cc:3080] error setting up scanner 9267912d07a3462bb298c328d272df4d with request: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: tablet hasn't heard from leader or there hasn't been a stable leader for 0.330s (maximum allowed 0.150s): new_scan_request { tablet_id: "50b14eb091f7476d9d7682e503745c93" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7317593227800997888 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\337\323\360\323\006\"3\n\005slave\022*\n f676baf4d90e47909c86ef51b3260926\020\001\030\001 \001(\001" signature: "<redacted>""3\272\377\3278\0220\236\'\036\225\3266\312\023\204L,\347\252\335\306\257\345\221\275\300\207\352\031\323W\277_ \003\305r\241\377 \344\323\265\031\320\027B*d\272\350xy\316h\030\272\227\026\236s\235\247" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "5ff8540deebd45bda469c5dfbf71d1a2"
W20260812 08:03:05.178363 23449 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53158: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:05.182652 23449 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53158: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:05.189695 23220 tablet_service.cc:3080] error setting up scanner 215bf11fe4f04ba0b1115022f79d8b39 with request: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: tablet hasn't heard from leader or there hasn't been a stable leader for 0.348s (maximum allowed 0.150s): new_scan_request { tablet_id: "50b14eb091f7476d9d7682e503745c93" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7317593227800997888 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\337\323\360\323\006\"3\n\005slave\022*\n f676baf4d90e47909c86ef51b3260926\020\001\030\001 \001(\001" signature: "<redacted>""3\272\377\3278\0220\236\'\036\225\3266\312\023\204L,\347\252\335\306\257\345\221\275\300\207\352\031\323W\277_ \003\305r\241\377 \344\323\265\031\320\027B*d\272\350xy\316h\030\272\227\026\236s\235\247" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "d4716c3919d347228becbc6c15e47625"
W20260812 08:03:05.192126 23449 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53158: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:05.209110 23925 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260812 08:03:05.209283 23925 fault_injection.cc:44] THIS SERVER MAY CRASH!
I20260812 08:03:05.210116 19620 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 15 FOLLOWER]: Refusing update from remote peer 38681904c36540a4876519ec5ffac6b6: Log matching property violated. Preceding OpId in replica: term: 12 index: 80. Preceding OpId from leader: term: 15 index: 84. (index mismatch)
I20260812 08:03:05.210177 23284 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 15 FOLLOWER]: Refusing update from remote peer 38681904c36540a4876519ec5ffac6b6: Log matching property violated. Preceding OpId in replica: term: 14 index: 82. Preceding OpId from leader: term: 15 index: 84. (index mismatch)
I20260812 08:03:05.210211 23535 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 15 FOLLOWER]: Refusing update from remote peer 38681904c36540a4876519ec5ffac6b6: Log matching property violated. Preceding OpId in replica: term: 14 index: 82. Preceding OpId from leader: term: 15 index: 84. (index mismatch)
I20260812 08:03:05.210269 22059 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 15 FOLLOWER]: Refusing update from remote peer 38681904c36540a4876519ec5ffac6b6: Log matching property violated. Preceding OpId in replica: term: 14 index: 82. Preceding OpId from leader: term: 15 index: 84. (index mismatch)
I20260812 08:03:05.210713 23925 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 83, Last known committed idx: 80, Time since last communication: 0.000s
I20260812 08:03:05.210912 23931 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 83, Last known committed idx: 80, Time since last communication: 0.000s
I20260812 08:03:05.211016 22832 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 83, Last known committed idx: 63, Time since last communication: 0.000s
I20260812 08:03:05.211103 23926 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 83, Last known committed idx: 80, Time since last communication: 0.000s
I20260812 08:03:05.211182 21839 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 15 FOLLOWER]: Refusing update from remote peer 38681904c36540a4876519ec5ffac6b6: Log matching property violated. Preceding OpId in replica: term: 14 index: 82. Preceding OpId from leader: term: 15 index: 84. (index mismatch)
I20260812 08:03:05.211308 23817 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 15 FOLLOWER]: Refusing update from remote peer 38681904c36540a4876519ec5ffac6b6: Log matching property violated. Preceding OpId in replica: term: 14 index: 81. Preceding OpId from leader: term: 15 index: 84. (index mismatch)
I20260812 08:03:05.212584 23926 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 83, Last known committed idx: 80, Time since last communication: 0.000s
W20260812 08:03:05.213115 23687 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Injecting 82ms of latency in SegmentAllocator::Sync()
I20260812 08:03:05.213152 22832 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 83, Last known committed idx: 80, Time since last communication: 0.000s
W20260812 08:03:05.214187 22504 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 36ms of latency in SegmentAllocator::Sync()
W20260812 08:03:05.215708 23940 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 9ms of latency in SegmentAllocator::Sync()
I20260812 08:03:05.217722 23936 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59600 (ReqId={client: 8e2317b289a14646951907d5a396d316, seq_no=63, attempt_no=1}) took 1788 ms. Trace:
I20260812 08:03:05.217834 23936 rpcz_store.cc:276] 0812 08:03:03.429729 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:03:03.429774 (+    45us) service_pool.cc:224] Handling call
0812 08:03:05.217695 (+1787921us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:03:05.218473 23936 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59600 (ReqId={client: 8e2317b289a14646951907d5a396d316, seq_no=64, attempt_no=1}) took 1793 ms. Trace:
I20260812 08:03:05.218600 23936 rpcz_store.cc:276] 0812 08:03:03.425509 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:03:03.425592 (+    83us) service_pool.cc:224] Handling call
0812 08:03:05.218461 (+1792869us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:03:05.219045 23937 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59600 (ReqId={client: 8e2317b289a14646951907d5a396d316, seq_no=66, attempt_no=1}) took 1793 ms. Trace:
I20260812 08:03:05.219115 23937 rpcz_store.cc:276] 0812 08:03:03.425351 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:03:03.425446 (+    95us) service_pool.cc:224] Handling call
0812 08:03:05.219021 (+1793575us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:03:05.219588 23937 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59600 (ReqId={client: 8e2317b289a14646951907d5a396d316, seq_no=67, attempt_no=1}) took 1792 ms. Trace:
I20260812 08:03:05.219651 23937 rpcz_store.cc:276] 0812 08:03:03.426666 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:03:03.426725 (+    59us) service_pool.cc:224] Handling call
0812 08:03:05.219578 (+1792853us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:03:05.220413 23935 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59600 (ReqId={client: 8e2317b289a14646951907d5a396d316, seq_no=65, attempt_no=1}) took 1790 ms. Trace:
I20260812 08:03:05.220476 23935 rpcz_store.cc:276] 0812 08:03:03.429909 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:03:03.429989 (+    80us) service_pool.cc:224] Handling call
0812 08:03:05.220378 (+1790389us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260812 08:03:05.221309 22508 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Injecting 74ms of latency in SegmentAllocator::Sync()
W20260812 08:03:05.223265 23689 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Injecting 139ms of latency in SegmentAllocator::Sync()
I20260812 08:03:05.225365 23674 mvcc.cc:204] Tried to move back new op lower bound from 7317593232214220800 to 7317593230440210432. Current Snapshot: MvccSnapshot[applied={T|T < 7317593230604115968}]
I20260812 08:03:05.227779 23943 mvcc.cc:204] Tried to move back new op lower bound from 7317593232214220800 to 7317593230440210432. Current Snapshot: MvccSnapshot[applied={T|T < 7317593230604115968}]
W20260812 08:03:05.230764 23944 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 51ms of latency in SegmentAllocator::Sync()
W20260812 08:03:05.231218 23940 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 126ms of latency in SegmentAllocator::Sync()
W20260812 08:03:05.233822 22503 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Injecting 119ms of latency in SegmentAllocator::Sync()
W20260812 08:03:05.254704 22504 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 87ms of latency in SegmentAllocator::Sync()
W20260812 08:03:05.296310 22508 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Injecting 54ms of latency in SegmentAllocator::Sync()
W20260812 08:03:05.304330 23687 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Injecting 77ms of latency in SegmentAllocator::Sync()
W20260812 08:03:05.304513 23944 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 17ms of latency in SegmentAllocator::Sync()
W20260812 08:03:05.324455 23944 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 38ms of latency in SegmentAllocator::Sync()
W20260812 08:03:05.349993 22504 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 186ms of latency in SegmentAllocator::Sync()
W20260812 08:03:05.353492 22508 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Injecting 59ms of latency in SegmentAllocator::Sync()
W20260812 08:03:05.357300 22503 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Injecting 12ms of latency in SegmentAllocator::Sync()
W20260812 08:03:05.362092 23940 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 6ms of latency in SegmentAllocator::Sync()
W20260812 08:03:05.367805 23944 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 12ms of latency in SegmentAllocator::Sync()
W20260812 08:03:05.368829 23689 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Injecting 151ms of latency in SegmentAllocator::Sync()
W20260812 08:03:05.375475 22503 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Injecting 84ms of latency in SegmentAllocator::Sync()
W20260812 08:03:05.377683 23940 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 95ms of latency in SegmentAllocator::Sync()
E20260812 08:03:05.380746 23926 fault_injection.cc:59] injecting fault for kudu (pid 22241): FLAGS_fault_crash_on_leader_request_fraction (process will exit)
W20260812 08:03:05.397970 23184 connection.cc:573] server connection from 127.18.42.70:39531 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 08:03:05.401248 18601 ts_itest-base.cc:575] TS 5 appears to have crashed. Restarting.
I20260812 08:03:05.401808 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.70:39785
--local_ip_for_outbound_sockets=127.18.42.70
--tserver_master_addrs=127.18.42.126:35213
--webserver_port=40739
--webserver_interface=127.18.42.70
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.05
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
W20260812 08:03:05.460187 21954 connection.cc:466] server connection from 127.18.42.70:54439 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.18.42.70:54439 (request call id 13) could send its response
W20260812 08:03:05.473277 19501 connection.cc:466] server connection from 127.18.42.70:49739 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.18.42.70:49739 (request call id 12) could send its response
W20260812 08:03:05.516954 23982 flags.cc:432] Enabled unsafe flag: --fault_crash_on_leader_request_fraction=0.050000000000000003
W20260812 08:03:05.517176 23982 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:03:05.517194 23982 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:03:05.517210 23982 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:03:05.517307 23982 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:05.517350 23982 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:05.517438 23982 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:05.520589 23436 connection.cc:466] server connection from 127.18.42.70:34327 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.18.42.70:34327 (request call id 7) could send its response
W20260812 08:03:05.521062 23982 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:05.521219 23982 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.70
I20260812 08:03:05.525115 23982 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.050000000000000003
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.70:39785
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data/info.pb
--webserver_interface=127.18.42.70
--webserver_port=40739
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.23982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.70

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:05.526289 23982 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:05.527436 23982 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:03:05.530347 23982 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:03:05.534052 23988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:05.534089 23987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:05.534062 23992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:05.534415 23982 server_base.cc:1061] running on GCE node
I20260812 08:03:05.534713 23982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:05.535228 23982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:05.536393 23982 hybrid_clock.cc:648] HybridClock initialized: now 1786521785536376 us; error 30 us; skew 500 ppm
W20260812 08:03:05.536643 21680 connection.cc:466] server connection from 127.18.42.70:36285 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.18.42.70:36285 (request call id 11) could send its response
I20260812 08:03:05.538307 23982 webserver.cc:533] Webserver started at http://127.18.42.70:40739/ using document root <none> and password file <none>
I20260812 08:03:05.538903 23982 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:05.538966 23982 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:05.542580 23982 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:03:05.544474 24002 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:05.545511 23982 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 08:03:05.545704 23982 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/wal
uuid: "38681904c36540a4876519ec5ffac6b6"
format_stamp: "Formatted at 2026-08-12 08:02:59 on dist-test-slave-zr1t"
I20260812 08:03:05.546093 23982 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:05.564848 23982 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:05.565613 23982 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:05.565820 23982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:05.566453 23982 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:05.567700 24015 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:03:05.571041 23982 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:05.571120 23982 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:03:05.571204 23982 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:05.572687 23982 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:05.572757 23982 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:03:05.573107 24015 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Bootstrap starting.
I20260812 08:03:05.584291 23982 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.70:39785
I20260812 08:03:05.584318 24170 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.70:39785 every 8 connection(s)
I20260812 08:03:05.584859 24015 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:05.585462 23982 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data/info.pb
I20260812 08:03:05.590809 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 23982
I20260812 08:03:05.595003 24173 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:03:05.595255 24173 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:05.595849 24173 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:03:05.597848 18661 ts_manager.cc:194] Re-registered known tserver with Master: 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785)
I20260812 08:03:05.598631 18661 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.70:37233
I20260812 08:03:05.604171 23690 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 38681904c36540a4876519ec5ffac6b6)
I20260812 08:03:05.604355 23690 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:05.605123 23690 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785)
I20260812 08:03:05.605581 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 95 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:05.605628 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 95 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:05.605670 22059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 95 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:05.606010 23819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 95 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:05.606686 23186 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf; no voters: 285e078e6cfb46ae9dbe57c20be666d5, 5b4c3ad48b794617893c88a077be1b8d, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:05.606796 23535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 95 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:05.606942 23690 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260812 08:03:05.611748 24103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 95 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
W20260812 08:03:05.613600 23186 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 16 pre-election: Tablet error from VoteRequest() call to peer 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:05.638994 24015 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Bootstrap replayed 1/1 log segments. Stats: ops{read=104 overwritten=2 applied=94 ignored=0} inserts{seen=85 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 8 replicates
I20260812 08:03:05.639467 24015 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Bootstrap complete.
W20260812 08:03:05.639631 24015 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Injecting 32ms of latency in SegmentAllocator::Sync()
I20260812 08:03:05.665244 23951 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 38681904c36540a4876519ec5ffac6b6)
I20260812 08:03:05.665405 23951 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:05.666988 23951 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:05.672128 24015 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Time spent bootstrapping tablet: real 0.099s	user 0.066s	sys 0.000s
I20260812 08:03:05.672863 22059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 16 candidate_status { last_received { term: 15 index: 104 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:05.672983 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 16 candidate_status { last_received { term: 15 index: 104 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:05.672972 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 16 candidate_status { last_received { term: 15 index: 104 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:05.673041 22059 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 15.
I20260812 08:03:05.673142 23284 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 15.
I20260812 08:03:05.674751 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 16 candidate_status { last_received { term: 15 index: 104 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:05.675320 24015 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 15 FOLLOWER]: Replica starting. Triggering 8 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:05.677367 24103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 16 candidate_status { last_received { term: 15 index: 104 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:05.677805 23535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 16 candidate_status { last_received { term: 15 index: 104 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:05.678193 23942 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 38681904c36540a4876519ec5ffac6b6)
I20260812 08:03:05.678308 23942 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:05.679131 23942 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:05.679280 22059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 107 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:05.679433 22059 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7def2bd3a74420d9deec19b32f541ec in term 15.
I20260812 08:03:05.679864 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 107 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:05.680063 23535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 107 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:05.680198 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 107 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:05.680333 23284 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7def2bd3a74420d9deec19b32f541ec in term 15.
I20260812 08:03:05.680435 24015 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 15 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38681904c36540a4876519ec5ffac6b6, State: Initialized, Role: FOLLOWER
I20260812 08:03:05.680958 24015 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 94, Last appended: 15.102, Last appended by leader: 102, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:05.681227 24103 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 15.
I20260812 08:03:05.682672 23721 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 285e078e6cfb46ae9dbe57c20be666d5, 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d; no voters: 294e0e1970eb4e58bed426de35babb6b, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:05.682929 23951 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260812 08:03:05.682986 23951 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 38681904c36540a4876519ec5ffac6b6)
I20260812 08:03:05.683022 23951 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 15 FOLLOWER]: Advancing to term 16
I20260812 08:03:05.684124 23951 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:05.684504 23819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 107 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:05.684666 23819 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b7def2bd3a74420d9deec19b32f541ec in current term 16: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:03:05.684932 23951 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 16 election: Requested vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:05.685230 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 16 candidate_status { last_received { term: 15 index: 104 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7"
I20260812 08:03:05.685333 24103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 107 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:05.685473 24103 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7def2bd3a74420d9deec19b32f541ec in term 15.
I20260812 08:03:05.685456 22059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 16 candidate_status { last_received { term: 15 index: 104 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d"
I20260812 08:03:05.685602 22059 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 15 FOLLOWER]: Advancing to term 16
I20260812 08:03:05.685750 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 16 candidate_status { last_received { term: 15 index: 104 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf"
I20260812 08:03:05.685906 23284 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 15 FOLLOWER]: Advancing to term 16
I20260812 08:03:05.686131 24103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 16 candidate_status { last_received { term: 15 index: 104 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6"
I20260812 08:03:05.686228 24103 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 15 FOLLOWER]: Advancing to term 16
I20260812 08:03:05.686259 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 16 candidate_status { last_received { term: 15 index: 104 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec"
I20260812 08:03:05.686370 19620 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 15 FOLLOWER]: Advancing to term 16
I20260812 08:03:05.686726 22059 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 16.
I20260812 08:03:05.687012 23284 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 16.
I20260812 08:03:05.687088 23535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 16 candidate_status { last_received { term: 15 index: 104 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b"
I20260812 08:03:05.687492 19620 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 285e078e6cfb46ae9dbe57c20be666d5 for term 16 because replica has last-logged OpId of term: 15 index: 107, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 104.
I20260812 08:03:05.687695 19501 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec; no voters: 285e078e6cfb46ae9dbe57c20be666d5, 294e0e1970eb4e58bed426de35babb6b, a5813459b1d94e089ffbe751f4310de7
I20260812 08:03:05.687908 23942 raft_consensus.cc:2764] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260812 08:03:05.688131 24015 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260812 08:03:05.688356 24103 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 16.
I20260812 08:03:05.688546 24173 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
I20260812 08:03:05.688638 23721 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 285e078e6cfb46ae9dbe57c20be666d5, 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d; no voters: 294e0e1970eb4e58bed426de35babb6b, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:05.688831 23951 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 16 FOLLOWER]: Leader election won for term 16
I20260812 08:03:05.688933 23951 raft_consensus.cc:697] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 16 LEADER]: Becoming Leader. State: Replica: 285e078e6cfb46ae9dbe57c20be666d5, State: Running, Role: LEADER
I20260812 08:03:05.689265 23951 consensus_queue.cc:237] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 94, Committed index: 94, Last appended: 15.104, Last appended by leader: 104, Current term: 16, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
W20260812 08:03:05.690613 23944 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 89ms of latency in SegmentAllocator::Sync()
I20260812 08:03:05.691835 18664 catalog_manager.cc:5719] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 reported cstate change: term changed from 15 to 16, leader changed from 38681904c36540a4876519ec5ffac6b6 (127.18.42.70) to 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69). New cstate: current_term: 16 leader_uuid: "285e078e6cfb46ae9dbe57c20be666d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } health_report { overall_health: UNKNOWN } } }
I20260812 08:03:05.750689 22534 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 38681904c36540a4876519ec5ffac6b6)
I20260812 08:03:05.750857 22534 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:05.751672 22534 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:05.752070 22059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 16 candidate_status { last_received { term: 15 index: 101 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:05.752285 22059 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5813459b1d94e089ffbe751f4310de7 in current term 16: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:03:05.752581 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 16 candidate_status { last_received { term: 15 index: 101 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:05.752643 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 16 candidate_status { last_received { term: 15 index: 101 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:05.752779 23284 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5813459b1d94e089ffbe751f4310de7 in current term 16: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:03:05.752873 19620 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5813459b1d94e089ffbe751f4310de7 for term 16 because replica has last-logged OpId of term: 15 index: 107, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 101.
I20260812 08:03:05.753228 23535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 16 candidate_status { last_received { term: 15 index: 101 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:05.753937 21680 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: a5813459b1d94e089ffbe751f4310de7; no voters: 0e10b84812514a76ab7225dfbf9329bf, 294e0e1970eb4e58bed426de35babb6b, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:05.754140 22534 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 15 FOLLOWER]: Advancing to term 16
I20260812 08:03:05.755182 22534 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260812 08:03:05.756820 23819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 16 candidate_status { last_received { term: 15 index: 101 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:05.757500 24103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 16 candidate_status { last_received { term: 15 index: 101 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:05.757746 24103 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5813459b1d94e089ffbe751f4310de7 in current term 16: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:03:05.768993 23652 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 38681904c36540a4876519ec5ffac6b6)
I20260812 08:03:05.769196 23652 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:05.770043 23652 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:05.770447 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 104 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:05.770510 23819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 104 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:05.770653 19620 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 294e0e1970eb4e58bed426de35babb6b for term 16 because replica has last-logged OpId of term: 15 index: 107, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 104.
I20260812 08:03:05.770637 22059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 104 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:05.770797 22059 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 294e0e1970eb4e58bed426de35babb6b in current term 16: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:03:05.771059 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 104 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:05.771232 23284 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 294e0e1970eb4e58bed426de35babb6b in current term 16: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
I20260812 08:03:05.771441 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 104 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:05.771615 21839 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 294e0e1970eb4e58bed426de35babb6b in term 16.
I20260812 08:03:05.771643 23436 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 294e0e1970eb4e58bed426de35babb6b; no voters: 0e10b84812514a76ab7225dfbf9329bf, 285e078e6cfb46ae9dbe57c20be666d5, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:05.771878 23652 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 15 FOLLOWER]: Advancing to term 16
I20260812 08:03:05.772848 23652 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260812 08:03:05.774716 24103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "294e0e1970eb4e58bed426de35babb6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 104 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:05.774905 24103 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 294e0e1970eb4e58bed426de35babb6b in current term 16: Already voted for candidate 285e078e6cfb46ae9dbe57c20be666d5 in this term.
W20260812 08:03:05.775940 23951 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260812 08:03:05.775995 23951 fault_injection.cc:44] THIS SERVER MAY CRASH!
I20260812 08:03:05.776784 19620 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 16 FOLLOWER]: Refusing update from remote peer 285e078e6cfb46ae9dbe57c20be666d5: Log matching property violated. Preceding OpId in replica: term: 15 index: 107. Preceding OpId from leader: term: 16 index: 105. (term mismatch)
I20260812 08:03:05.776849 19620 pending_rounds.cc:85] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Aborting all ops after (but not including) 104
I20260812 08:03:05.776867 19620 pending_rounds.cc:107] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Aborting uncommitted WRITE_OP operation due to leader change: 15.105
I20260812 08:03:05.777240 19620 pending_rounds.cc:107] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Aborting uncommitted WRITE_OP operation due to leader change: 15.106
I20260812 08:03:05.777614 19620 pending_rounds.cc:107] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Aborting uncommitted WRITE_OP operation due to leader change: 15.107
I20260812 08:03:05.778350 23951 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 105, Last known committed idx: 97, Time since last communication: 0.000s
I20260812 08:03:05.780287 22059 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 16 FOLLOWER]: Refusing update from remote peer 285e078e6cfb46ae9dbe57c20be666d5: Log matching property violated. Preceding OpId in replica: term: 15 index: 104. Preceding OpId from leader: term: 16 index: 105. (index mismatch)
W20260812 08:03:05.781028 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:03:05.781275 23951 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 105, Last known committed idx: 97, Time since last communication: 0.000s
W20260812 08:03:05.781028 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:05.784680 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
E20260812 08:03:05.785327 23951 fault_injection.cc:59] injecting fault for kudu (pid 23691): FLAGS_fault_crash_on_leader_request_fraction (process will exit)
W20260812 08:03:05.794032 21763 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:05.794265 21764 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:05.794025 21762 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:05.801635 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:05.803555 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:05.803565 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
I20260812 08:03:05.806617 18601 ts_itest-base.cc:575] TS 4 appears to have crashed. Restarting.
I20260812 08:03:05.807081 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.69:34819
--local_ip_for_outbound_sockets=127.18.42.69
--tserver_master_addrs=127.18.42.126:35213
--webserver_port=40235
--webserver_interface=127.18.42.69
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.05
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
W20260812 08:03:05.810529 23449 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53158: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:05.813731 23449 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53158: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:05.814714 23449 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53158: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:05.822780 23222 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:05.822780 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:05.831614 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:05.847028 24052 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56860: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:05.847028 24051 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56860: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:05.848459 24051 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56860: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:05.861253 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:05.862738 20378 meta_cache.cc:302] tablet 50b14eb091f7476d9d7682e503745c93: replica 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819) has failed: Network error: Client connection negotiation failed: client connection to 127.18.42.69:34819: connect: Connection refused (error 111)
W20260812 08:03:05.874357 21762 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:05.874357 21764 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:05.878288 21764 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:05.887893 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:05.889895 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:05.894125 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:05.903712 23449 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53158: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:05.904676 23449 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53158: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:05.907989 23449 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53158: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:05.922875 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:05.922911 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:05.926609 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:05.928851 24219 flags.cc:432] Enabled unsafe flag: --fault_crash_on_leader_request_fraction=0.050000000000000003
W20260812 08:03:05.929061 24219 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:03:05.929080 24219 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:03:05.929095 24219 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:03:05.929132 24219 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:05.929158 24219 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:05.929205 24219 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:05.932703 24219 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:05.932821 24219 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.69
I20260812 08:03:05.937022 24219 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.050000000000000003
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.69:34819
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data/info.pb
--webserver_interface=127.18.42.69
--webserver_port=40235
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.24219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.69

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:05.938277 24219 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:05.939421 24219 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:05.942021 24051 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56860: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
E20260812 08:03:05.942759 24219 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:03:05.946699 24228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:05.946743 24229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:05.946702 24232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:05.946966 24219 server_base.cc:1061] running on GCE node
I20260812 08:03:05.947397 24219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:05.947959 24219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:05.949126 24219 hybrid_clock.cc:648] HybridClock initialized: now 1786521785949111 us; error 29 us; skew 500 ppm
I20260812 08:03:05.951069 24219 webserver.cc:533] Webserver started at http://127.18.42.69:40235/ using document root <none> and password file <none>
I20260812 08:03:05.951647 24219 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:05.951705 24219 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:05.955276 24219 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:03:05.957334 24242 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:05.958705 24219 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
W20260812 08:03:05.959003 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
I20260812 08:03:05.958819 24219 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
uuid: "285e078e6cfb46ae9dbe57c20be666d5"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:03:05.959357 24219 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260812 08:03:05.962499 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:05.962815 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:03:05.978070 24219 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:05.978838 24219 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:05.979003 24219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:05.979575 24219 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
W20260812 08:03:05.981452 24051 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56860: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
I20260812 08:03:05.981725 24255 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:03:05.985352 24219 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:05.985440 24219 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.005s	user 0.001s	sys 0.000s
I20260812 08:03:05.985564 24219 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:05.987141 24219 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:05.987212 24219 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:03:05.987632 24255 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Bootstrap starting.
I20260812 08:03:05.997970 24219 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.69:34819
I20260812 08:03:05.999177 24418 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.69:34819 every 8 connection(s)
I20260812 08:03:05.999536 24219 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data/info.pb
W20260812 08:03:06.001868 19525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:03:06.002215 24255 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Log is configured to *not* fsync() on all Append() calls
W20260812 08:03:06.002921 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:03:06.005244 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 24219
I20260812 08:03:06.006078 24206 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:06.006207 24206 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:06.007874 24206 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:06.011138 24421 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:03:06.011391 24421 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:06.011992 24421 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:03:06.014329 18664 ts_manager.cc:194] Re-registered known tserver with Master: 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819)
I20260812 08:03:06.015143 22059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 17 candidate_status { last_received { term: 15 index: 102 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:06.015322 18664 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.69:40679
I20260812 08:03:06.015990 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 17 candidate_status { last_received { term: 15 index: 102 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:06.016130 23284 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 16.
I20260812 08:03:06.018173 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 17 candidate_status { last_received { term: 15 index: 102 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:06.018776 23535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 17 candidate_status { last_received { term: 15 index: 102 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:06.018944 23535 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38681904c36540a4876519ec5ffac6b6 for term 17 because replica has last-logged OpId of term: 15 index: 104, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 102.
I20260812 08:03:06.021248 24357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 17 candidate_status { last_received { term: 15 index: 102 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
W20260812 08:03:06.022801 24010 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 17 pre-election: Tablet error from VoteRequest() call to peer 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:06.022938 24010 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 38681904c36540a4876519ec5ffac6b6; no voters: 285e078e6cfb46ae9dbe57c20be666d5, 294e0e1970eb4e58bed426de35babb6b, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:06.023197 24206 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260812 08:03:06.023453 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 17 candidate_status { last_received { term: 15 index: 102 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:06.023648 21839 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 16.
W20260812 08:03:06.044552 19535 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59600: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:06.047807 21764 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:06.055362 21764 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:06.066397 21764 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:03:06.071163 24255 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Bootstrap replayed 1/1 log segments. Stats: ops{read=107 overwritten=2 applied=94 ignored=0} inserts{seen=85 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 11 replicates
W20260812 08:03:06.071264 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
I20260812 08:03:06.071769 24255 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Bootstrap complete.
W20260812 08:03:06.071977 24255 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 162ms of latency in SegmentAllocator::Sync()
W20260812 08:03:06.078387 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
I20260812 08:03:06.080279 23942 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 285e078e6cfb46ae9dbe57c20be666d5)
I20260812 08:03:06.080404 23942 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:06.081182 23942 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:06.081362 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:06.081650 22059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:06.081760 23535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:06.081825 24103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:06.081930 23535 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7def2bd3a74420d9deec19b32f541ec in term 16.
I20260812 08:03:06.081976 24103 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7def2bd3a74420d9deec19b32f541ec in term 16.
I20260812 08:03:06.082146 21839 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7def2bd3a74420d9deec19b32f541ec in term 16.
I20260812 08:03:06.082983 24218 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 285e078e6cfb46ae9dbe57c20be666d5)
I20260812 08:03:06.083035 19501 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 294e0e1970eb4e58bed426de35babb6b, 38681904c36540a4876519ec5ffac6b6, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec; no voters: 5b4c3ad48b794617893c88a077be1b8d
I20260812 08:03:06.083122 24218 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:06.083216 23942 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260812 08:03:06.083277 23942 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 16 FOLLOWER]: Starting leader election (detected failure of leader 285e078e6cfb46ae9dbe57c20be666d5)
I20260812 08:03:06.083318 23942 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 16 FOLLOWER]: Advancing to term 17
I20260812 08:03:06.083523 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:06.083649 23284 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7def2bd3a74420d9deec19b32f541ec in term 16.
I20260812 08:03:06.084022 24218 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:06.084746 23942 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:06.085067 23535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 17 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:06.085186 23535 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 16.
I20260812 08:03:06.085376 23942 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 17 election: Requested vote from peers a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:06.085783 22059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d"
I20260812 08:03:06.085902 22059 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 16 FOLLOWER]: Advancing to term 17
I20260812 08:03:06.086032 24103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6"
I20260812 08:03:06.086144 24103 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 16 FOLLOWER]: Advancing to term 17
I20260812 08:03:06.086158 19620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 17 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:06.086320 19620 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b4c3ad48b794617893c88a077be1b8d in current term 17: Already voted for candidate b7def2bd3a74420d9deec19b32f541ec in this term.
I20260812 08:03:06.086717 23535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b"
I20260812 08:03:06.086836 23535 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 16 FOLLOWER]: Advancing to term 17
I20260812 08:03:06.087030 22059 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7def2bd3a74420d9deec19b32f541ec in term 17.
I20260812 08:03:06.087215 24103 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7def2bd3a74420d9deec19b32f541ec in term 17.
I20260812 08:03:06.088037 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7"
I20260812 08:03:06.088070 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf"
I20260812 08:03:06.088160 21839 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 16 FOLLOWER]: Advancing to term 17
I20260812 08:03:06.088178 23284 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 16 FOLLOWER]: Advancing to term 17
I20260812 08:03:06.088187 23535 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7def2bd3a74420d9deec19b32f541ec in term 17.
I20260812 08:03:06.089166 23284 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7def2bd3a74420d9deec19b32f541ec in term 17.
I20260812 08:03:06.089190 21839 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7def2bd3a74420d9deec19b32f541ec in term 17.
I20260812 08:03:06.089213 19501 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 294e0e1970eb4e58bed426de35babb6b, 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec; no voters: 
I20260812 08:03:06.090970 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 17 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:06.091113 21839 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b4c3ad48b794617893c88a077be1b8d in current term 17: Already voted for candidate b7def2bd3a74420d9deec19b32f541ec in this term.
I20260812 08:03:06.091203 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 17 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:06.091364 23284 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b4c3ad48b794617893c88a077be1b8d in current term 17: Already voted for candidate b7def2bd3a74420d9deec19b32f541ec in this term.
I20260812 08:03:06.091509 23942 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 17 FOLLOWER]: Leader election won for term 17
I20260812 08:03:06.091934 23942 raft_consensus.cc:697] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 17 LEADER]: Becoming Leader. State: Replica: b7def2bd3a74420d9deec19b32f541ec, State: Running, Role: LEADER
I20260812 08:03:06.092356 23942 consensus_queue.cc:237] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 97, Committed index: 97, Last appended: 16.105, Last appended by leader: 105, Current term: 17, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:06.093256 24357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:06.093380 24353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5"
W20260812 08:03:06.094168 19502 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 17 pre-election: Tablet error from VoteRequest() call to peer 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819): Illegal state: must be running to vote when last-logged opid is not known
W20260812 08:03:06.094441 19502 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 17 election: Tablet error from VoteRequest() call to peer 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819): Illegal state: must be running to vote when last-logged opid is not known
W20260812 08:03:06.095153 23940 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 11ms of latency in SegmentAllocator::Sync()
W20260812 08:03:06.095368 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
I20260812 08:03:06.095772 24103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 17 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:06.095950 24103 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b4c3ad48b794617893c88a077be1b8d in current term 17: Already voted for candidate b7def2bd3a74420d9deec19b32f541ec in this term.
I20260812 08:03:06.096518 21955 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 294e0e1970eb4e58bed426de35babb6b, 5b4c3ad48b794617893c88a077be1b8d; no voters: 0e10b84812514a76ab7225dfbf9329bf, 38681904c36540a4876519ec5ffac6b6, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:06.096767 24218 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260812 08:03:06.097118 18664 catalog_manager.cc:5719] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec reported cstate change: term changed from 16 to 17, leader changed from 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69) to b7def2bd3a74420d9deec19b32f541ec (127.18.42.68). New cstate: current_term: 17 leader_uuid: "b7def2bd3a74420d9deec19b32f541ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } health_report { overall_health: UNKNOWN } } }
I20260812 08:03:06.097960 24353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 17 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
W20260812 08:03:06.098623 21956 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 17 pre-election: Tablet error from VoteRequest() call to peer 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819): Illegal state: must be running to vote when last-logged opid is not known
W20260812 08:03:06.098878 23449 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53158: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:06.103207 23449 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53158: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:06.122083 23449 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53158: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:06.123242 24051 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56860: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:06.127707 24051 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56860: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:06.148689 24051 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56860: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:06.151263 23940 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 57ms of latency in SegmentAllocator::Sync()
W20260812 08:03:06.151618 23942 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260812 08:03:06.151656 23942 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260812 08:03:06.151715 24450 fault_injection.cc:59] injecting fault for kudu (pid 19474): FLAGS_fault_crash_on_leader_request_fraction (process will exit)
W20260812 08:03:06.151638 23220 tablet_service.cc:3080] error setting up scanner 95bb756493e943a7bc233a277c3a7ad9 with request: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: tablet hasn't heard from leader or there hasn't been a stable leader for 0.850s (maximum allowed 0.150s): new_scan_request { tablet_id: "50b14eb091f7476d9d7682e503745c93" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7317593227800997888 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\337\323\360\323\006\"3\n\005slave\022*\n f676baf4d90e47909c86ef51b3260926\020\001\030\001 \001(\001" signature: "<redacted>""3\272\377\3278\0220\236\'\036\225\3266\312\023\204L,\347\252\335\306\257\345\221\275\300\207\352\031\323W\277_ \003\305r\241\377 \344\323\265\031\320\027B*d\272\350xy\316h\030\272\227\026\236s\235\247" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "d4716c3919d347228becbc6c15e47625"
W20260812 08:03:06.164076 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:06.176273 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:06.178385 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:06.179426 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:06.180763 23222 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:06.180842 23219 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:06.181109 23218 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:06.181264 23217 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:06.181816 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
I20260812 08:03:06.183164 18601 ts_itest-base.cc:575] TS 3 appears to have crashed. Restarting.
I20260812 08:03:06.183566 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.68:36499
--local_ip_for_outbound_sockets=127.18.42.68
--tserver_master_addrs=127.18.42.126:35213
--webserver_port=36373
--webserver_interface=127.18.42.68
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.05
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
W20260812 08:03:06.189248 21762 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:06.191354 21762 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:06.192215 21764 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:06.199484 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:06.221818 21764 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:06.229394 21764 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:03:06.234516 24255 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Time spent bootstrapping tablet: real 0.247s	user 0.065s	sys 0.006s
W20260812 08:03:06.235497 21764 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53648: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:03:06.237054 24255 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 16 FOLLOWER]: Replica starting. Triggering 11 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:06.240818 24255 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 16 FOLLOWER]: Becoming Follower/Learner. State: Replica: 285e078e6cfb46ae9dbe57c20be666d5, State: Initialized, Role: FOLLOWER
I20260812 08:03:06.241613 24255 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 94, Last appended: 16.105, Last appended by leader: 105, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:06.242468 24421 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
I20260812 08:03:06.243502 24255 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
W20260812 08:03:06.252609 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:06.261050 23449 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53158: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:06.265127 23449 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53158: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:06.282945 23449 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53158: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:06.292481 24286 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44322: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:06.298166 24286 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44322: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:06.309913 24455 flags.cc:432] Enabled unsafe flag: --fault_crash_on_leader_request_fraction=0.050000000000000003
W20260812 08:03:06.310163 24455 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:03:06.310181 24455 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:03:06.310197 24455 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:03:06.310236 24455 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:06.310262 24455 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:06.310307 24455 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:06.313844 24455 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:06.313964 24455 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.68
W20260812 08:03:06.316746 24286 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44322: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
I20260812 08:03:06.318316 24455 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_crash_on_leader_request_fraction=0.050000000000000003
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.68:36499
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/data/info.pb
--webserver_interface=127.18.42.68
--webserver_port=36373
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.24455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.68

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:06.319437 24455 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:06.320587 24455 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:03:06.323685 24455 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:03:06.325605 24051 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56860: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:06.327952 24469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:06.327946 24472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:06.327944 24470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:06.328567 24455 server_base.cc:1061] running on GCE node
W20260812 08:03:06.328778 24051 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56860: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
I20260812 08:03:06.329033 24455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:06.329720 24455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:06.330921 24455 hybrid_clock.cc:648] HybridClock initialized: now 1786521786330901 us; error 37 us; skew 500 ppm
I20260812 08:03:06.332971 24455 webserver.cc:533] Webserver started at http://127.18.42.68:36373/ using document root <none> and password file <none>
I20260812 08:03:06.333649 24455 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:06.333729 24455 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:06.337222 24455 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:03:06.339401 24483 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:06.340461 24455 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:06.340597 24455 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/wal
uuid: "b7def2bd3a74420d9deec19b32f541ec"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:03:06.340996 24455 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260812 08:03:06.351236 24051 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56860: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:06.358050 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:06.359886 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
I20260812 08:03:06.362324 24455 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:06.363008 24455 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:06.363199 24455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:06.363870 24455 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:06.365245 24491 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:03:06.368664 24455 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:06.368767 24455 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:03:06.368863 24455 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:06.370544 24455 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:06.370620 24455 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:03:06.371003 24491 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Bootstrap starting.
I20260812 08:03:06.382151 24491 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:06.382858 24455 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.68:36499
I20260812 08:03:06.382933 24665 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.68:36499 every 8 connection(s)
I20260812 08:03:06.383987 24455 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/data/info.pb
W20260812 08:03:06.386777 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:06.391503 21993 tablet_service.cc:3080] error setting up scanner cb437179fa5c4999be318c21d22c5808 with request: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: tablet hasn't heard from leader or there hasn't been a stable leader for 0.609s (maximum allowed 0.150s): new_scan_request { tablet_id: "50b14eb091f7476d9d7682e503745c93" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7317593227800997888 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\337\323\360\323\006\"3\n\005slave\022*\n f676baf4d90e47909c86ef51b3260926\020\001\030\001 \001(\001" signature: "<redacted>""3\272\377\3278\0220\236\'\036\225\3266\312\023\204L,\347\252\335\306\257\345\221\275\300\207\352\031\323W\277_ \003\305r\241\377 \344\323\265\031\320\027B*d\272\350xy\316h\030\272\227\026\236s\235\247" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "5ff8540deebd45bda469c5dfbf71d1a2"
I20260812 08:03:06.392314 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 24455
I20260812 08:03:06.395174 24666 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:03:06.395535 24666 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:06.396209 24666 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:03:06.399583 18664 ts_manager.cc:194] Re-registered known tserver with Master: b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499)
I20260812 08:03:06.400568 18664 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.68:42033
I20260812 08:03:06.402643 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 21615
W20260812 08:03:06.438504 21993 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:06.444164 20378 connection.cc:573] client connection to 127.18.42.65:45661 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 08:03:06.445143 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.65:45661
--local_ip_for_outbound_sockets=127.18.42.65
--tserver_master_addrs=127.18.42.126:35213
--webserver_port=38137
--webserver_interface=127.18.42.65
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
I20260812 08:03:06.452955 24491 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Bootstrap replayed 1/1 log segments. Stats: ops{read=113 overwritten=6 applied=97 ignored=0} inserts{seen=88 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 10 replicates
I20260812 08:03:06.453464 24491 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Bootstrap complete.
W20260812 08:03:06.453682 24491 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 57ms of latency in SegmentAllocator::Sync()
I20260812 08:03:06.475320 24672 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:06.475481 24672 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:06.476220 24672 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785)
I20260812 08:03:06.476730 22059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 18 candidate_status { last_received { term: 15 index: 95 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:06.476951 22059 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e10b84812514a76ab7225dfbf9329bf for term 18 because replica has last-logged OpId of term: 16 index: 105, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 95.
I20260812 08:03:06.477092 23535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 18 candidate_status { last_received { term: 15 index: 95 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:06.477257 23535 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e10b84812514a76ab7225dfbf9329bf for term 18 because replica has last-logged OpId of term: 15 index: 104, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 95.
W20260812 08:03:06.477476 23191 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.42.65:45661: connect: Connection refused (error 111)
I20260812 08:03:06.477713 24103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 18 candidate_status { last_received { term: 15 index: 95 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:06.477886 24103 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e10b84812514a76ab7225dfbf9329bf for term 18 because replica has last-logged OpId of term: 15 index: 102, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 95.
W20260812 08:03:06.478940 23191 leader_election.cc:336] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661): Network error: Client connection negotiation failed: client connection to 127.18.42.65:45661: connect: Connection refused (error 111)
I20260812 08:03:06.479018 23191 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf; no voters: 294e0e1970eb4e58bed426de35babb6b, 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d, a5813459b1d94e089ffbe751f4310de7
I20260812 08:03:06.479391 24672 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260812 08:03:06.482633 24353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 18 candidate_status { last_received { term: 15 index: 95 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:06.482893 24353 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e10b84812514a76ab7225dfbf9329bf for term 18 because replica has last-logged OpId of term: 16 index: 105, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 95.
I20260812 08:03:06.483372 24588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 18 candidate_status { last_received { term: 15 index: 95 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
W20260812 08:03:06.484972 23186 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 18 pre-election: Tablet error from VoteRequest() call to peer b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:06.509788 24218 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 08:03:06.510002 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
I20260812 08:03:06.509948 24218 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:06.510890 24218 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
W20260812 08:03:06.510931 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
I20260812 08:03:06.511343 24491 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Time spent bootstrapping tablet: real 0.141s	user 0.073s	sys 0.004s
I20260812 08:03:06.511617 23535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 18 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:06.511767 23535 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 17.
I20260812 08:03:06.512073 24103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 18 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:06.512212 24103 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 17.
I20260812 08:03:06.512722 24353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 18 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:06.512990 24353 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 16.
W20260812 08:03:06.513374 21956 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.42.65:45661: connect: Connection refused (error 111)
I20260812 08:03:06.513470 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 18 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:06.513643 23284 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 17.
I20260812 08:03:06.513783 21956 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 285e078e6cfb46ae9dbe57c20be666d5, 294e0e1970eb4e58bed426de35babb6b, 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d; no voters: 
I20260812 08:03:06.513971 24218 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260812 08:03:06.514024 24218 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:03:06.514062 24218 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 17 FOLLOWER]: Advancing to term 18
I20260812 08:03:06.514061 24491 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 17 FOLLOWER]: Replica starting. Triggering 10 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
W20260812 08:03:06.515059 21956 leader_election.cc:336] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661): Network error: Client connection negotiation failed: client connection to 127.18.42.65:45661: connect: Connection refused (error 111)
I20260812 08:03:06.515262 24218 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
W20260812 08:03:06.515852 22000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39676: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
I20260812 08:03:06.516904 24353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 18 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5"
I20260812 08:03:06.517038 23535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 18 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b"
I20260812 08:03:06.517055 24353 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 16 FOLLOWER]: Advancing to term 18
I20260812 08:03:06.517163 23535 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 17 FOLLOWER]: Advancing to term 18
I20260812 08:03:06.517505 24491 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 17 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7def2bd3a74420d9deec19b32f541ec, State: Initialized, Role: FOLLOWER
I20260812 08:03:06.518064 24491 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 97, Last appended: 17.107, Last appended by leader: 107, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:06.518330 23535 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 18.
I20260812 08:03:06.518599 24103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 18 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6"
I20260812 08:03:06.518718 24103 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 17 FOLLOWER]: Advancing to term 18
I20260812 08:03:06.519014 24491 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260812 08:03:06.519208 24353 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 18.
I20260812 08:03:06.519418 24666 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
I20260812 08:03:06.519762 24103 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 18.
I20260812 08:03:06.520085 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 18 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf"
I20260812 08:03:06.520232 23284 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 17 FOLLOWER]: Advancing to term 18
I20260812 08:03:06.521286 23284 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 18.
I20260812 08:03:06.522230 21955 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 285e078e6cfb46ae9dbe57c20be666d5, 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d; no voters: 
I20260812 08:03:06.522635 24685 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 18 FOLLOWER]: Leader election won for term 18
I20260812 08:03:06.522795 24685 raft_consensus.cc:697] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 18 LEADER]: Becoming Leader. State: Replica: 5b4c3ad48b794617893c88a077be1b8d, State: Running, Role: LEADER
I20260812 08:03:06.523100 24685 consensus_queue.cc:237] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 97, Committed index: 97, Last appended: 16.105, Last appended by leader: 105, Current term: 18, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
W20260812 08:03:06.523835 21956 leader_election.cc:336] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661): Network error: Client connection negotiation failed: client connection to 127.18.42.65:45661: connect: Connection refused (error 111)
I20260812 08:03:06.515868 24218 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 18 election: Requested vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
W20260812 08:03:06.525158 22503 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Injecting 41ms of latency in SegmentAllocator::Sync()
I20260812 08:03:06.526546 18664 catalog_manager.cc:5719] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d reported cstate change: term changed from 17 to 18, leader changed from b7def2bd3a74420d9deec19b32f541ec (127.18.42.68) to 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66). New cstate: current_term: 18 leader_uuid: "5b4c3ad48b794617893c88a077be1b8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } health_report { overall_health: UNKNOWN } } }
I20260812 08:03:06.527177 24588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 18 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:06.527253 24587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 18 candidate_status { last_received { term: 16 index: 105 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec"
I20260812 08:03:06.527369 24588 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b4c3ad48b794617893c88a077be1b8d for term 18 because replica has last-logged OpId of term: 17 index: 107, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 105.
W20260812 08:03:06.547015 23449 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53158: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:06.548020 23449 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53158: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:06.559337 23449 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53158: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
W20260812 08:03:06.569247 24671 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:03:06.569455 24671 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:03:06.569474 24671 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:03:06.569583 24671 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:06.569617 24671 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:06.569665 24671 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:06.572962 24671 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:06.573091 24671 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.65
I20260812 08:03:06.577159 24671 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.65:45661
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data/info.pb
--webserver_interface=127.18.42.65
--webserver_port=38137
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.24671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:06.578392 24671 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:06.579523 24671 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:03:06.582307 24671 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:03:06.584203 24286 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44322: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:06.586272 24699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:06.586272 24696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:06.586282 24694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:06.586757 24671 server_base.cc:1061] running on GCE node
I20260812 08:03:06.587126 24671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:06.587667 24671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:06.588857 24671 hybrid_clock.cc:648] HybridClock initialized: now 1786521786588841 us; error 32 us; skew 500 ppm
W20260812 08:03:06.589159 24286 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44322: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
I20260812 08:03:06.590976 24671 webserver.cc:533] Webserver started at http://127.18.42.65:38137/ using document root <none> and password file <none>
I20260812 08:03:06.591579 24671 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:06.591674 24671 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:06.595468 24671 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.002s	sys 0.002s
I20260812 08:03:06.597719 24706 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
W20260812 08:03:06.597736 24286 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44322: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
I20260812 08:03:06.598917 24671 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 08:03:06.599054 24671 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/wal
uuid: "a5813459b1d94e089ffbe751f4310de7"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:03:06.599501 24671 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260812 08:03:06.608669 24685 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260812 08:03:06.608752 24685 fault_injection.cc:44] THIS SERVER MAY CRASH!
I20260812 08:03:06.609820 24353 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 18 FOLLOWER]: Refusing update from remote peer 5b4c3ad48b794617893c88a077be1b8d: Log matching property violated. Preceding OpId in replica: term: 16 index: 105. Preceding OpId from leader: term: 18 index: 106. (index mismatch)
I20260812 08:03:06.610486 24685 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 106, Last known committed idx: 94, Time since last communication: 0.000s
I20260812 08:03:06.617864 24671 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:06.618595 24671 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:06.618820 24671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:06.619491 24671 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
E20260812 08:03:06.619757 24685 fault_injection.cc:59] injecting fault for kudu (pid 21922): FLAGS_fault_crash_on_leader_request_fraction (process will exit)
I20260812 08:03:06.634014 24719 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260812 08:03:06.634426 24052 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56860: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:06.634454 24051 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56860: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:06.637042 24051 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56860: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
I20260812 08:03:06.638505 24671 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:06.638588 24671 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.009s	user 0.001s	sys 0.000s
I20260812 08:03:06.638665 24671 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:06.641027 24671 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:06.641088 24671 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:03:06.641669 24719 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Bootstrap starting.
I20260812 08:03:06.654035 24671 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.65:45661
I20260812 08:03:06.654078 24880 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.65:45661 every 8 connection(s)
I20260812 08:03:06.655469 24671 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data/info.pb
I20260812 08:03:06.657577 24719 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Log is configured to *not* fsync() on all Append() calls
W20260812 08:03:06.657583 24052 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56860: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:06.658953 24052 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56860: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
I20260812 08:03:06.659943 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 24671
I20260812 08:03:06.660424 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.66:32855
--local_ip_for_outbound_sockets=127.18.42.66
--tserver_master_addrs=127.18.42.126:35213
--webserver_port=42027
--webserver_interface=127.18.42.66
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
I20260812 08:03:06.668949 24881 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:03:06.669176 24881 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:06.669719 24881 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:03:06.671430 18664 ts_manager.cc:194] Re-registered known tserver with Master: a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661)
I20260812 08:03:06.672286 18664 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.65:55343
W20260812 08:03:06.673880 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:06.675740 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:06.676904 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
I20260812 08:03:06.719048 24719 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Bootstrap replayed 1/1 log segments. Stats: ops{read=104 overwritten=3 applied=91 ignored=0} inserts{seen=82 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 10 replicates
I20260812 08:03:06.719645 24719 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Bootstrap complete.
I20260812 08:03:06.720045 24719 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Time spent bootstrapping tablet: real 0.079s	user 0.072s	sys 0.004s
I20260812 08:03:06.723156 24719 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 17 FOLLOWER]: Replica starting. Triggering 10 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:06.726118 24719 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 17 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5813459b1d94e089ffbe751f4310de7, State: Initialized, Role: FOLLOWER
I20260812 08:03:06.726693 24719 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 91, Last appended: 15.101, Last appended by leader: 101, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:06.727708 24719 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.002s
I20260812 08:03:06.728133 24881 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
W20260812 08:03:06.760273 24750 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53680: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:06.760275 24749 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53680: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:06.799398 24888 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:03:06.799604 24888 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:03:06.799623 24888 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:03:06.799659 24888 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:06.799733 24888 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:06.799819 24888 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:06.803325 24888 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:06.803442 24888 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.66
W20260812 08:03:06.806896 24523 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39692: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
I20260812 08:03:06.807654 24888 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.66:32855
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/data/info.pb
--webserver_interface=127.18.42.66
--webserver_port=42027
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.24888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:06.808750 24888 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:06.809923 24888 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:06.810609 24523 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39692: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
E20260812 08:03:06.812860 24888 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:03:06.816919 24902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:06.816919 24900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:06.816968 24906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:06.817283 24888 server_base.cc:1061] running on GCE node
I20260812 08:03:06.817777 24888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:06.818406 24888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:06.819591 24888 hybrid_clock.cc:648] HybridClock initialized: now 1786521786819569 us; error 33 us; skew 500 ppm
I20260812 08:03:06.821492 24888 webserver.cc:533] Webserver started at http://127.18.42.66:42027/ using document root <none> and password file <none>
I20260812 08:03:06.822257 24888 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:06.822355 24888 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:06.826212 24888 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.004s	sys 0.000s
I20260812 08:03:06.828475 24916 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:06.829885 24888 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:03:06.830017 24888 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/wal
uuid: "5b4c3ad48b794617893c88a077be1b8d"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:03:06.830538 24888 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:06.845284 24683 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:06.845551 24683 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:06.847139 24683 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
W20260812 08:03:06.848694 24486 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.42.66:32855: connect: Connection refused (error 111)
W20260812 08:03:06.851617 24486 leader_election.cc:336] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855): Network error: Client connection negotiation failed: client connection to 127.18.42.66:32855: connect: Connection refused (error 111)
I20260812 08:03:06.852461 24672 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:06.852586 24672 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
W20260812 08:03:06.853123 23449 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53158: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
I20260812 08:03:06.853225 24672 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785)
I20260812 08:03:06.854377 24814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 107 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:06.854684 24814 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7def2bd3a74420d9deec19b32f541ec in term 17.
I20260812 08:03:06.855518 24588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 19 candidate_status { last_received { term: 15 index: 95 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:06.855667 24588 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e10b84812514a76ab7225dfbf9329bf for term 19 because replica has last-logged OpId of term: 17 index: 107, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 95.
I20260812 08:03:06.855924 23535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 19 candidate_status { last_received { term: 15 index: 95 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:06.856103 23535 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e10b84812514a76ab7225dfbf9329bf for term 19 because replica has last-logged OpId of term: 15 index: 104, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 95.
I20260812 08:03:06.856189 23538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 107 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:06.856356 23538 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b7def2bd3a74420d9deec19b32f541ec in current term 18: Already voted for candidate 5b4c3ad48b794617893c88a077be1b8d in this term.
I20260812 08:03:06.856797 24103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 19 candidate_status { last_received { term: 15 index: 95 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:06.856966 24103 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e10b84812514a76ab7225dfbf9329bf for term 19 because replica has last-logged OpId of term: 15 index: 102, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 95.
W20260812 08:03:06.857877 23449 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53158: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
I20260812 08:03:06.857442 24353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 19 candidate_status { last_received { term: 15 index: 95 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:06.858907 23191 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf; no voters: 285e078e6cfb46ae9dbe57c20be666d5, 294e0e1970eb4e58bed426de35babb6b, 38681904c36540a4876519ec5ffac6b6, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:06.858939 24353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 107 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:06.859335 24672 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260812 08:03:06.859402 23185 leader_election.cc:336] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855): Network error: Client connection negotiation failed: client connection to 127.18.42.66:32855: connect: Connection refused (error 111)
I20260812 08:03:06.860606 24103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 107 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:06.860759 24103 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b7def2bd3a74420d9deec19b32f541ec in current term 18: Already voted for candidate 5b4c3ad48b794617893c88a077be1b8d in this term.
I20260812 08:03:06.861156 24487 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec; no voters: 285e078e6cfb46ae9dbe57c20be666d5, 294e0e1970eb4e58bed426de35babb6b, 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d
I20260812 08:03:06.861243 24888 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:06.861418 24683 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 17 FOLLOWER]: Advancing to term 18
I20260812 08:03:06.862133 24888 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:06.862347 24888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:06.862614 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "b7def2bd3a74420d9deec19b32f541ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 107 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:06.862756 23284 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b7def2bd3a74420d9deec19b32f541ec in current term 18: Already voted for candidate 5b4c3ad48b794617893c88a077be1b8d in this term.
I20260812 08:03:06.863133 24888 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:06.863212 24814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "0e10b84812514a76ab7225dfbf9329bf" candidate_term: 19 candidate_status { last_received { term: 15 index: 95 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:06.863441 24814 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e10b84812514a76ab7225dfbf9329bf for term 19 because replica has last-logged OpId of term: 15 index: 101, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 95.
I20260812 08:03:06.863921 24683 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260812 08:03:06.864744 24938 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:03:06.868125 24888 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:06.868191 24888 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.001s	sys 0.000s
I20260812 08:03:06.868278 24888 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:06.869871 24888 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:06.869916 24888 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.001s
I20260812 08:03:06.870229 24938 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Bootstrap starting.
I20260812 08:03:06.881973 24938 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:06.882292 24888 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.66:32855
I20260812 08:03:06.882349 25109 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.66:32855 every 8 connection(s)
I20260812 08:03:06.883435 24888 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/data/info.pb
I20260812 08:03:06.885769 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 24888
I20260812 08:03:06.885960 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 23407
W20260812 08:03:06.900223 24286 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44322: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
W20260812 08:03:06.902168 24286 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44322: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
I20260812 08:03:06.905485 25113 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:03:06.905781 25113 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:06.906324 25113 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:03:06.907961 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.67:40205
--local_ip_for_outbound_sockets=127.18.42.67
--tserver_master_addrs=127.18.42.126:35213
--webserver_port=39753
--webserver_interface=127.18.42.67
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
I20260812 08:03:06.910784 18664 ts_manager.cc:194] Re-registered known tserver with Master: 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855)
I20260812 08:03:06.911567 18664 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.66:39341
I20260812 08:03:06.911923 24459 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5b4c3ad48b794617893c88a077be1b8d)
I20260812 08:03:06.912087 24459 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:06.913908 24459 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
W20260812 08:03:06.920061 24246 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.42.67:40205: connect: Connection refused (error 111)
I20260812 08:03:06.920436 24588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 19 candidate_status { last_received { term: 18 index: 106 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:06.920622 24588 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 18.
I20260812 08:03:06.926048 24814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 19 candidate_status { last_received { term: 18 index: 106 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:06.926229 24814 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 17.
W20260812 08:03:06.928639 24246 leader_election.cc:336] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205): Network error: Client connection negotiation failed: client connection to 127.18.42.67:40205: connect: Connection refused (error 111)
I20260812 08:03:06.930193 24103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 19 candidate_status { last_received { term: 18 index: 106 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6" is_pre_election: true
I20260812 08:03:06.930353 24103 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 18.
I20260812 08:03:06.930773 24246 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 285e078e6cfb46ae9dbe57c20be666d5, 38681904c36540a4876519ec5ffac6b6, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec; no voters: 294e0e1970eb4e58bed426de35babb6b
I20260812 08:03:06.931039 24459 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260812 08:03:06.931090 24459 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5b4c3ad48b794617893c88a077be1b8d)
I20260812 08:03:06.931126 24459 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 18 FOLLOWER]: Advancing to term 19
I20260812 08:03:06.932242 24459 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:06.932524 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 19 candidate_status { last_received { term: 18 index: 106 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:06.932668 23284 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 18.
I20260812 08:03:06.932958 24459 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 19 election: Requested vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:06.933496 24814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 19 candidate_status { last_received { term: 18 index: 106 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7"
I20260812 08:03:06.933686 24814 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 17 FOLLOWER]: Advancing to term 19
I20260812 08:03:06.933787 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 19 candidate_status { last_received { term: 18 index: 106 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf"
I20260812 08:03:06.933916 23284 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 18 FOLLOWER]: Advancing to term 19
I20260812 08:03:06.934558 24588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 19 candidate_status { last_received { term: 18 index: 106 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec"
I20260812 08:03:06.934697 24588 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 18 FOLLOWER]: Advancing to term 19
I20260812 08:03:06.935050 23284 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 19.
I20260812 08:03:06.935907 24588 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 19.
I20260812 08:03:06.936033 24814 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 19.
I20260812 08:03:06.936947 24246 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 285e078e6cfb46ae9dbe57c20be666d5, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec; no voters: 
I20260812 08:03:06.936959 24103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 19 candidate_status { last_received { term: 18 index: 106 } } ignore_live_leader: false dest_uuid: "38681904c36540a4876519ec5ffac6b6"
I20260812 08:03:06.937089 24103 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 18 FOLLOWER]: Advancing to term 19
W20260812 08:03:06.938138 24246 leader_election.cc:336] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205): Network error: Client connection negotiation failed: client connection to 127.18.42.67:40205: connect: Connection refused (error 111)
I20260812 08:03:06.938239 24103 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285e078e6cfb46ae9dbe57c20be666d5 in term 19.
I20260812 08:03:06.938742 24459 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 19 FOLLOWER]: Leader election won for term 19
I20260812 08:03:06.938977 24459 raft_consensus.cc:697] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 19 LEADER]: Becoming Leader. State: Replica: 285e078e6cfb46ae9dbe57c20be666d5, State: Running, Role: LEADER
I20260812 08:03:06.937893 25035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 19 candidate_status { last_received { term: 18 index: 106 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:06.939319 24459 consensus_queue.cc:237] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 97, Committed index: 97, Last appended: 18.106, Last appended by leader: 106, Current term: 19, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:06.938529 25034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "285e078e6cfb46ae9dbe57c20be666d5" candidate_term: 19 candidate_status { last_received { term: 18 index: 106 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d"
W20260812 08:03:06.940218 24244 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 19 pre-election: Tablet error from VoteRequest() call to peer 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855): Illegal state: must be running to vote when last-logged opid is not known
W20260812 08:03:06.940603 24244 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [CANDIDATE]: Term 19 election: Tablet error from VoteRequest() call to peer 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855): Illegal state: must be running to vote when last-logged opid is not known
W20260812 08:03:06.941763 24715 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 29ms of latency in SegmentAllocator::Sync()
I20260812 08:03:06.943049 18664 catalog_manager.cc:5719] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 reported cstate change: term changed from 18 to 19, leader changed from 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66) to 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69). New cstate: current_term: 19 leader_uuid: "285e078e6cfb46ae9dbe57c20be666d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } health_report { overall_health: UNKNOWN } } }
W20260812 08:03:06.949189 24750 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53680: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:06.949201 24052 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56860: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
I20260812 08:03:06.954321 24938 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Bootstrap replayed 1/1 log segments. Stats: ops{read=107 overwritten=1 applied=97 ignored=0} inserts{seen=88 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 9 replicates
I20260812 08:03:06.954895 24938 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Bootstrap complete.
W20260812 08:03:06.955077 24938 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Injecting 101ms of latency in SegmentAllocator::Sync()
W20260812 08:03:06.998121 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
W20260812 08:03:06.998121 24052 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56860: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:07.021246 24459 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260812 08:03:07.021337 24459 fault_injection.cc:44] THIS SERVER MAY CRASH!
I20260812 08:03:07.022723 24103 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 19 FOLLOWER]: Refusing update from remote peer 285e078e6cfb46ae9dbe57c20be666d5: Log matching property violated. Preceding OpId in replica: term: 15 index: 102. Preceding OpId from leader: term: 19 index: 107. (index mismatch)
I20260812 08:03:07.023463 24459 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 107, Last known committed idx: 94, Time since last communication: 0.000s
W20260812 08:03:07.028090 25138 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Injecting 45ms of latency in SegmentAllocator::Sync()
I20260812 08:03:07.034849 24588 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 19 FOLLOWER]: Refusing update from remote peer 285e078e6cfb46ae9dbe57c20be666d5: Log matching property violated. Preceding OpId in replica: term: 17 index: 107. Preceding OpId from leader: term: 19 index: 107. (term mismatch)
I20260812 08:03:07.034940 24588 pending_rounds.cc:85] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Aborting all ops after (but not including) 106
I20260812 08:03:07.034988 24588 pending_rounds.cc:107] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Aborting uncommitted WRITE_OP operation due to leader change: 17.107
I20260812 08:03:07.036291 24459 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 107, Last known committed idx: 97, Time since last communication: 0.000s
I20260812 08:03:07.036453 24459 consensus_queue.cc:1243] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [LEADER]: Peer b7def2bd3a74420d9deec19b32f541ec log is divergent from this leader: its last log entry 17.106 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 97
I20260812 08:03:07.038172 24588 raft_consensus.cc:1217] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 19 FOLLOWER]: Deduplicated request from leader. Original: 15.97->[15.98-19.107]   Dedup: 16.105->[18.106-19.107]
I20260812 08:03:07.038328 24588 pending_rounds.cc:85] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Aborting all ops after (but not including) 105
I20260812 08:03:07.038372 24588 pending_rounds.cc:107] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Aborting uncommitted NO_OP operation due to leader change: 17.106
I20260812 08:03:07.038409 24588 raft_consensus.cc:2889] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260812 08:03:07.039711 25139 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 113ms of latency in SegmentAllocator::Sync()
W20260812 08:03:07.044997 25118 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:03:07.045248 25118 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:03:07.045275 25118 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:03:07.045327 25118 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:07.045352 25118 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:07.045408 25118 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260812 08:03:07.047407 24814 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 19 FOLLOWER]: Refusing update from remote peer 285e078e6cfb46ae9dbe57c20be666d5: Log matching property violated. Preceding OpId in replica: term: 15 index: 101. Preceding OpId from leader: term: 19 index: 107. (index mismatch)
I20260812 08:03:07.048053 24459 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 107, Last known committed idx: 91, Time since last communication: 0.000s
W20260812 08:03:07.052033 25118 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:07.052175 25118 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.67
E20260812 08:03:07.054733 24459 fault_injection.cc:59] injecting fault for kudu (pid 24219): FLAGS_fault_crash_on_leader_request_fraction (process will exit)
I20260812 08:03:07.066038 24938 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Time spent bootstrapping tablet: real 0.196s	user 0.075s	sys 0.004s
I20260812 08:03:07.069101 24938 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 18 FOLLOWER]: Replica starting. Triggering 9 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:07.083961 24938 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 18 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b4c3ad48b794617893c88a077be1b8d, State: Initialized, Role: FOLLOWER
I20260812 08:03:07.084558 24938 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 97, Last appended: 18.106, Last appended by leader: 106, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:07.085216 25113 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
I20260812 08:03:07.085912 24938 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260812 08:03:07.089807 25118 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.67:40205
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data/info.pb
--webserver_interface=127.18.42.67
--webserver_port=39753
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.25118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
W20260812 08:03:07.090847 24009 connection.cc:573] server connection from 127.18.42.69:39405 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 08:03:07.090845 24711 connection.cc:573] server connection from 127.18.42.69:59789 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 08:03:07.091259 25118 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
W20260812 08:03:07.091583 20378 meta_cache.cc:302] tablet 50b14eb091f7476d9d7682e503745c93: replica 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819) has failed: Network error: recv got EOF from 127.18.42.69:34819 (error 108)
I20260812 08:03:07.092468 25118 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:03:07.095180 25118 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:03:07.099082 25162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:07.099151 25166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:07.099092 25163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:07.099398 25118 server_base.cc:1061] running on GCE node
I20260812 08:03:07.099809 25118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:07.100373 25118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:07.101573 25118 hybrid_clock.cc:648] HybridClock initialized: now 1786521787101550 us; error 36 us; skew 500 ppm
I20260812 08:03:07.103410 25118 webserver.cc:533] Webserver started at http://127.18.42.67:39753/ using document root <none> and password file <none>
I20260812 08:03:07.103999 25118 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:07.104058 25118 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
W20260812 08:03:07.105425 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
I20260812 08:03:07.107824 25118 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.004s	sys 0.000s
I20260812 08:03:07.109843 25176 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:07.110852 25118 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:07.110991 25118 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal
uuid: "294e0e1970eb4e58bed426de35babb6b"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:03:07.111379 25118 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260812 08:03:07.126082 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
I20260812 08:03:07.135236 25118 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:07.135922 25118 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:07.136123 25118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:07.136808 25118 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:07.138161 25189 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260812 08:03:07.139647 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
I20260812 08:03:07.141932 25118 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:07.142042 25118 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.001s
I20260812 08:03:07.142158 25118 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:07.143787 25118 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:07.143884 25118 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:03:07.144207 25189 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Bootstrap starting.
W20260812 08:03:07.153976 24488 connection.cc:466] server connection from 127.18.42.69:46411 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.18.42.69:46411 (request call id 3) could send its response
I20260812 08:03:07.156764 25189 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:07.157696 25118 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.67:40205
I20260812 08:03:07.158936 25118 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data/info.pb
I20260812 08:03:07.159024 25370 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.67:40205 every 8 connection(s)
I20260812 08:03:07.165031 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 25118
I20260812 08:03:07.165210 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 24455
I20260812 08:03:07.189438 25371 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:03:07.189864 25371 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:07.190734 25371 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:03:07.193102 18664 ts_manager.cc:194] Re-registered known tserver with Master: 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205)
I20260812 08:03:07.194267 18664 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.67:54773
W20260812 08:03:07.198429 20377 connection.cc:573] client connection to 127.18.42.68:36499 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 08:03:07.199470 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.68:36499
--local_ip_for_outbound_sockets=127.18.42.68
--tserver_master_addrs=127.18.42.126:35213
--webserver_port=36373
--webserver_interface=127.18.42.68
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
W20260812 08:03:07.210630 24944 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39700: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
I20260812 08:03:07.240175 25189 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Bootstrap replayed 1/1 log segments. Stats: ops{read=105 overwritten=1 applied=94 ignored=0} inserts{seen=85 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 10 replicates
I20260812 08:03:07.240811 25189 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Bootstrap complete.
W20260812 08:03:07.241039 25189 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Injecting 30ms of latency in SegmentAllocator::Sync()
I20260812 08:03:07.271559 25189 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Time spent bootstrapping tablet: real 0.128s	user 0.058s	sys 0.012s
I20260812 08:03:07.273837 25189 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 18 FOLLOWER]: Replica starting. Triggering 10 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:07.276551 25189 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 18 FOLLOWER]: Becoming Follower/Learner. State: Replica: 294e0e1970eb4e58bed426de35babb6b, State: Initialized, Role: FOLLOWER
I20260812 08:03:07.277081 25189 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 94, Last appended: 15.104, Last appended by leader: 104, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:07.278530 25371 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
I20260812 08:03:07.278771 25189 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
W20260812 08:03:07.302134 24052 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56860: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:07.305910 24052 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56860: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:07.317811 24052 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56860: Illegal state: replica 38681904c36540a4876519ec5ffac6b6 is not leader of this config: current role FOLLOWER
W20260812 08:03:07.320194 25375 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:03:07.320420 25375 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:03:07.320439 25375 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:03:07.320539 25375 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:07.320583 25375 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:07.320648 25375 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:07.324712 25375 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:07.324900 25375 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.68
I20260812 08:03:07.325770 25132 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 285e078e6cfb46ae9dbe57c20be666d5)
I20260812 08:03:07.325934 25132 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:07.326665 25132 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:07.328130 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 20 candidate_status { last_received { term: 19 index: 107 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:07.328326 23284 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 19.
W20260812 08:03:07.329304 24010 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.42.69:34819: connect: Connection refused (error 111)
I20260812 08:03:07.331147 25375 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.68:36499
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/data/info.pb
--webserver_interface=127.18.42.68
--webserver_port=36373
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.25375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.68

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:07.332649 25375 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:07.332881 25034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 20 candidate_status { last_received { term: 19 index: 107 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:07.333106 24814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 20 candidate_status { last_received { term: 19 index: 107 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:07.333156 25034 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 18.
W20260812 08:03:07.333789 24010 leader_election.cc:336] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819): Network error: Client connection negotiation failed: client connection to 127.18.42.69:34819: connect: Connection refused (error 111)
I20260812 08:03:07.334170 25375 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:07.334285 24009 leader_election.cc:336] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499): Network error: Client connection negotiation failed: client connection to 127.18.42.68:36499: connect: Connection refused (error 111)
I20260812 08:03:07.335484 25297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 20 candidate_status { last_received { term: 19 index: 107 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:07.335772 25297 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 18.
I20260812 08:03:07.336197 24009 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 294e0e1970eb4e58bed426de35babb6b, 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d; no voters: 285e078e6cfb46ae9dbe57c20be666d5, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:07.336404 25132 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260812 08:03:07.336467 25132 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 285e078e6cfb46ae9dbe57c20be666d5)
I20260812 08:03:07.336514 25132 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 19 FOLLOWER]: Advancing to term 20
I20260812 08:03:07.337644 25132 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
E20260812 08:03:07.337738 25375 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
I20260812 08:03:07.338313 25132 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 20 election: Requested vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:07.338765 25297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 20 candidate_status { last_received { term: 19 index: 107 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b"
I20260812 08:03:07.338901 25297 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 18 FOLLOWER]: Advancing to term 20
I20260812 08:03:07.339154 24814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 20 candidate_status { last_received { term: 19 index: 107 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7"
W20260812 08:03:07.339358 24009 leader_election.cc:336] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499): Network error: Client connection negotiation failed: client connection to 127.18.42.68:36499: connect: Connection refused (error 111)
I20260812 08:03:07.339498 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 20 candidate_status { last_received { term: 19 index: 107 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf"
I20260812 08:03:07.339622 23284 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 19 FOLLOWER]: Advancing to term 20
I20260812 08:03:07.340803 23284 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 20.
I20260812 08:03:07.340943 25297 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 20.
I20260812 08:03:07.341089 25034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "38681904c36540a4876519ec5ffac6b6" candidate_term: 20 candidate_status { last_received { term: 19 index: 107 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d"
I20260812 08:03:07.341231 25034 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 18 FOLLOWER]: Advancing to term 20
W20260812 08:03:07.341919 25402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:07.342450 25400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:07.342509 24010 leader_election.cc:336] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819): Network error: Client connection negotiation failed: client connection to 127.18.42.69:34819: connect: Connection refused (error 111)
W20260812 08:03:07.342645 25399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:07.343396 25375 server_base.cc:1061] running on GCE node
I20260812 08:03:07.343699 25034 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38681904c36540a4876519ec5ffac6b6 in term 20.
I20260812 08:03:07.343804 25375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:07.344056 24008 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 294e0e1970eb4e58bed426de35babb6b, 38681904c36540a4876519ec5ffac6b6, 5b4c3ad48b794617893c88a077be1b8d; no voters: 285e078e6cfb46ae9dbe57c20be666d5, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:07.344251 25132 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 20 FOLLOWER]: Leader election won for term 20
I20260812 08:03:07.344460 25132 raft_consensus.cc:697] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 20 LEADER]: Becoming Leader. State: Replica: 38681904c36540a4876519ec5ffac6b6, State: Running, Role: LEADER
I20260812 08:03:07.344475 25375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:07.344723 25132 consensus_queue.cc:237] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 97, Committed index: 97, Last appended: 19.107, Last appended by leader: 107, Current term: 20, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
W20260812 08:03:07.346261 25138 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Injecting 16ms of latency in SegmentAllocator::Sync()
I20260812 08:03:07.347513 18664 catalog_manager.cc:5719] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 reported cstate change: term changed from 19 to 20, leader changed from 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69) to 38681904c36540a4876519ec5ffac6b6 (127.18.42.70). New cstate: current_term: 20 leader_uuid: "38681904c36540a4876519ec5ffac6b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } health_report { overall_health: UNKNOWN } } }
I20260812 08:03:07.347726 25375 hybrid_clock.cc:648] HybridClock initialized: now 1786521787347685 us; error 44 us; skew 500 ppm
I20260812 08:03:07.350134 25375 webserver.cc:533] Webserver started at http://127.18.42.68:36373/ using document root <none> and password file <none>
I20260812 08:03:07.350335 24895 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 285e078e6cfb46ae9dbe57c20be666d5)
I20260812 08:03:07.350472 24895 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:07.351001 25375 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:07.351091 25375 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:07.352085 24895 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
W20260812 08:03:07.352622 25132 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260812 08:03:07.352669 25132 fault_injection.cc:44] THIS SERVER MAY CRASH!
I20260812 08:03:07.353849 25297 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 20 FOLLOWER]: Refusing update from remote peer 38681904c36540a4876519ec5ffac6b6: Log matching property violated. Preceding OpId in replica: term: 15 index: 104. Preceding OpId from leader: term: 20 index: 109. (index mismatch)
I20260812 08:03:07.354244 23284 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 20 FOLLOWER]: Refusing update from remote peer 38681904c36540a4876519ec5ffac6b6: Log matching property violated. Preceding OpId in replica: term: 15 index: 95. Preceding OpId from leader: term: 20 index: 109. (index mismatch)
I20260812 08:03:07.354842 24814 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 19 FOLLOWER]: Advancing to term 20
I20260812 08:03:07.356060 24814 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 20 FOLLOWER]: Refusing update from remote peer 38681904c36540a4876519ec5ffac6b6: Log matching property violated. Preceding OpId in replica: term: 19 index: 107. Preceding OpId from leader: term: 20 index: 109. (index mismatch)
I20260812 08:03:07.356057 25034 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 20 FOLLOWER]: Refusing update from remote peer 38681904c36540a4876519ec5ffac6b6: Log matching property violated. Preceding OpId in replica: term: 18 index: 106. Preceding OpId from leader: term: 20 index: 109. (index mismatch)
W20260812 08:03:07.356737 24010 consensus_peers.cc:597] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 -> Peer 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819): Couldn't send request to peer 285e078e6cfb46ae9dbe57c20be666d5. Status: Network error: Client connection negotiation failed: client connection to 127.18.42.69:34819: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260812 08:03:07.356990 24710 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.42.68:36499: connect: Connection refused (error 111)
I20260812 08:03:07.357128 25132 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 108, Last known committed idx: 94, Time since last communication: 0.000s
W20260812 08:03:07.357558 24009 consensus_peers.cc:597] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 -> Peer b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499): Couldn't send request to peer b7def2bd3a74420d9deec19b32f541ec. Status: Network error: Client connection negotiation failed: client connection to 127.18.42.68:36499: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 08:03:07.357821 25132 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 108, Last known committed idx: 86, Time since last communication: 0.000s
I20260812 08:03:07.358400 25297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 20 candidate_status { last_received { term: 19 index: 107 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:07.358584 25297 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5813459b1d94e089ffbe751f4310de7 in current term 20: Already voted for candidate 38681904c36540a4876519ec5ffac6b6 in this term.
E20260812 08:03:07.359102 25132 fault_injection.cc:59] injecting fault for kudu (pid 23982): FLAGS_fault_crash_on_leader_request_fraction (process will exit)
I20260812 08:03:07.370618 25034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 20 candidate_status { last_received { term: 19 index: 107 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:07.370872 25034 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5813459b1d94e089ffbe751f4310de7 in current term 20: Already voted for candidate 38681904c36540a4876519ec5ffac6b6 in this term.
W20260812 08:03:07.372197 24710 leader_election.cc:336] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499): Network error: Client connection negotiation failed: client connection to 127.18.42.68:36499: connect: Connection refused (error 111)
W20260812 08:03:07.372361 24750 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53680: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:07.374329 24711 leader_election.cc:336] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819): Network error: Client connection negotiation failed: client connection to 127.18.42.69:34819: connect: Connection refused (error 111)
I20260812 08:03:07.374430 24711 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: a5813459b1d94e089ffbe751f4310de7; no voters: 285e078e6cfb46ae9dbe57c20be666d5, 294e0e1970eb4e58bed426de35babb6b, 5b4c3ad48b794617893c88a077be1b8d, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:07.374459 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 20 candidate_status { last_received { term: 19 index: 107 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:07.374682 23284 raft_consensus.cc:2393] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5813459b1d94e089ffbe751f4310de7 in current term 20: Already voted for candidate 38681904c36540a4876519ec5ffac6b6 in this term.
I20260812 08:03:07.374825 24895 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260812 08:03:07.375844 25375 fs_manager.cc:714] Time spent opening directory manager: real 0.023s	user 0.004s	sys 0.000s
I20260812 08:03:07.378319 25425 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:07.379580 25375 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260812 08:03:07.379716 25375 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/wal
uuid: "b7def2bd3a74420d9deec19b32f541ec"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:03:07.380156 25375 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260812 08:03:07.384652 25422 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:03:07.352727 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.18.42.70:39785 (local address 127.18.42.65:56047)
0812 08:03:07.357467 (+  4740us) negotiation.cc:107] Waiting for socket to connect
0812 08:03:07.357481 (+    14us) client_negotiation.cc:175] Beginning negotiation
0812 08:03:07.357792 (+   311us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 08:03:07.358173 (+   381us) client_negotiation.cc:281] Received NEGOTIATE NegotiatePB response
0812 08:03:07.358176 (+     3us) client_negotiation.cc:404] Received NEGOTIATE response from server
0812 08:03:07.358194 (+    18us) client_negotiation.cc:190] Negotiated authn=CERTIFICATE
0812 08:03:07.368895 (+ 10701us) client_negotiation.cc:530] Sending TLS_HANDSHAKE message to server
0812 08:03:07.368913 (+    18us) client_negotiation.cc:262] Sending TLS_HANDSHAKE NegotiatePB request
0812 08:03:07.383882 (+ 14969us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.42.70:39785: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":4664,"thread_start_us":99,"threads_started":1}
W20260812 08:03:07.385833 24710 leader_election.cc:336] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785): Network error: Client connection negotiation failed: client connection to 127.18.42.70:39785: connect: Connection refused (error 111)
W20260812 08:03:07.422428 24944 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39700: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
I20260812 08:03:07.424994 25375 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:07.425768 25375 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:07.425931 25375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:07.426510 25375 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:07.427887 25441 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:03:07.431490 25375 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:07.431578 25375 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:03:07.431664 25375 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
W20260812 08:03:07.432557 24749 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53680: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
I20260812 08:03:07.433404 25375 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:07.433449 25375 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:03:07.433813 25441 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Bootstrap starting.
W20260812 08:03:07.445180 24944 tablet_service.cc:3080] error setting up scanner b6edf86d7fbd4303a3f94846d70d090e with request: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: tablet hasn't heard from leader or there hasn't been a stable leader for 0.378s (maximum allowed 0.150s): new_scan_request { tablet_id: "50b14eb091f7476d9d7682e503745c93" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7317593227800997888 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\337\323\360\323\006\"3\n\005slave\022*\n f676baf4d90e47909c86ef51b3260926\020\001\030\001 \001(\001" signature: "<redacted>""3\272\377\3278\0220\236\'\036\225\3266\312\023\204L,\347\252\335\306\257\345\221\275\300\207\352\031\323W\277_ \003\305r\241\377 \344\323\265\031\320\027B*d\272\350xy\316h\030\272\227\026\236s\235\247" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "d4716c3919d347228becbc6c15e47625"
I20260812 08:03:07.445324 25375 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.68:36499
I20260812 08:03:07.445495 25610 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.68:36499 every 8 connection(s)
I20260812 08:03:07.446535 25375 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/data/info.pb
I20260812 08:03:07.447568 25441 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:07.449705 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 25375
I20260812 08:03:07.450237 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.69:34819
--local_ip_for_outbound_sockets=127.18.42.69
--tserver_master_addrs=127.18.42.126:35213
--webserver_port=40235
--webserver_interface=127.18.42.69
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
I20260812 08:03:07.456020 25612 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:03:07.456231 25612 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:07.456669 25612 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:03:07.458389 18661 ts_manager.cc:194] Re-registered known tserver with Master: b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499)
I20260812 08:03:07.459393 18661 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.68:58087
W20260812 08:03:07.476645 24749 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53680: Illegal state: replica a5813459b1d94e089ffbe751f4310de7 is not leader of this config: current role FOLLOWER
W20260812 08:03:07.482491 24944 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39700: Illegal state: replica 5b4c3ad48b794617893c88a077be1b8d is not leader of this config: current role FOLLOWER
W20260812 08:03:07.483697 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
I20260812 08:03:07.502251 25441 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Bootstrap replayed 1/1 log segments. Stats: ops{read=115 overwritten=8 applied=97 ignored=0} inserts{seen=88 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 10 replicates
I20260812 08:03:07.502822 25441 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Bootstrap complete.
W20260812 08:03:07.503019 25441 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 11ms of latency in SegmentAllocator::Sync()
I20260812 08:03:07.514459 25441 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Time spent bootstrapping tablet: real 0.081s	user 0.069s	sys 0.000s
I20260812 08:03:07.516765 25441 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 19 FOLLOWER]: Replica starting. Triggering 10 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:07.518851 25441 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 19 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7def2bd3a74420d9deec19b32f541ec, State: Initialized, Role: FOLLOWER
I20260812 08:03:07.519352 25441 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 97, Last appended: 19.107, Last appended by leader: 107, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:07.520033 25612 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
I20260812 08:03:07.520246 25441 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
W20260812 08:03:07.543412 25476 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39698: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:07.543607 25473 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39698: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:07.572111 25621 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:03:07.572316 25621 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:03:07.572335 25621 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:03:07.572396 25621 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:07.572439 25621 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:07.572546 25621 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:07.575930 25621 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:07.576041 25621 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.69
I20260812 08:03:07.580031 25621 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.69:34819
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data/info.pb
--webserver_interface=127.18.42.69
--webserver_port=40235
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.25621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.69

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:07.581151 25621 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:07.582384 25621 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:03:07.585251 25621 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:03:07.589190 25638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:07.589196 25631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:07.589277 25621 server_base.cc:1061] running on GCE node
W20260812 08:03:07.589196 25632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:07.589916 25621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:07.590431 25621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:07.591603 25621 hybrid_clock.cc:648] HybridClock initialized: now 1786521787591589 us; error 26 us; skew 500 ppm
I20260812 08:03:07.593312 25621 webserver.cc:533] Webserver started at http://127.18.42.69:40235/ using document root <none> and password file <none>
I20260812 08:03:07.593956 25621 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:07.594017 25621 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
W20260812 08:03:07.594918 25476 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39698: Illegal state: replica b7def2bd3a74420d9deec19b32f541ec is not leader of this config: current role FOLLOWER
W20260812 08:03:07.596860 25226 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53172: Illegal state: replica 294e0e1970eb4e58bed426de35babb6b is not leader of this config: current role FOLLOWER
I20260812 08:03:07.598151 25621 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.002s	sys 0.000s
I20260812 08:03:07.600152 25650 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:07.601238 25621 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:07.601423 25621 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
uuid: "285e078e6cfb46ae9dbe57c20be666d5"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-zr1t"
I20260812 08:03:07.601909 25621 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:07.618674 25621 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:07.619387 25621 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:07.619587 25621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:07.620208 25621 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:07.621469 25660 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:03:07.624966 25621 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:07.625027 25621 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:03:07.625108 25621 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:07.626678 25621 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:07.626724 25621 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:03:07.627142 25660 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Bootstrap starting.
I20260812 08:03:07.638780 25621 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.69:34819
I20260812 08:03:07.638804 25841 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.69:34819 every 8 connection(s)
I20260812 08:03:07.639676 25660 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:07.639883 25621 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data/info.pb
I20260812 08:03:07.643060 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 25621
I20260812 08:03:07.643599 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.70:39785
--local_ip_for_outbound_sockets=127.18.42.70
--tserver_master_addrs=127.18.42.126:35213
--webserver_port=40739
--webserver_interface=127.18.42.70
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
I20260812 08:03:07.649739 25844 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:03:07.650056 25844 heartbeater.cc:461] Registering TS with master...
W20260812 08:03:07.650280 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
I20260812 08:03:07.650730 25844 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:03:07.652710 18661 ts_manager.cc:194] Re-registered known tserver with Master: 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819)
I20260812 08:03:07.653805 18661 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.69:48055
W20260812 08:03:07.654538 23220 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43188: Illegal state: replica 0e10b84812514a76ab7225dfbf9329bf is not leader of this config: current role FOLLOWER
I20260812 08:03:07.663419 25849 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:07.663664 25849 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:07.672290 25849 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), a5813459b1d94e089ffbe751f4310de7 (127.18.42.65:45661), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:07.673645 25540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 21 candidate_status { last_received { term: 18 index: 106 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:07.673995 25540 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b4c3ad48b794617893c88a077be1b8d for term 21 because replica has last-logged OpId of term: 19 index: 107, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 106.
W20260812 08:03:07.676355 24919 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.42.70:39785: connect: Connection refused (error 111)
I20260812 08:03:07.678604 25297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 21 candidate_status { last_received { term: 18 index: 106 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:07.678774 25297 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 20.
I20260812 08:03:07.680299 24814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 21 candidate_status { last_received { term: 18 index: 106 } } ignore_live_leader: false dest_uuid: "a5813459b1d94e089ffbe751f4310de7" is_pre_election: true
I20260812 08:03:07.680485 24814 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b4c3ad48b794617893c88a077be1b8d for term 21 because replica has last-logged OpId of term: 19 index: 107, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 106.
I20260812 08:03:07.683032 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 21 candidate_status { last_received { term: 18 index: 106 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:07.683185 23284 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b4c3ad48b794617893c88a077be1b8d in term 20.
W20260812 08:03:07.683743 24919 leader_election.cc:336] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785): Network error: Client connection negotiation failed: client connection to 127.18.42.70:39785: connect: Connection refused (error 111)
I20260812 08:03:07.685910 25767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "5b4c3ad48b794617893c88a077be1b8d" candidate_term: 21 candidate_status { last_received { term: 18 index: 106 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
W20260812 08:03:07.689437 24924 leader_election.cc:343] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 21 pre-election: Tablet error from VoteRequest() call to peer 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:03:07.689594 24924 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 294e0e1970eb4e58bed426de35babb6b, 5b4c3ad48b794617893c88a077be1b8d; no voters: 285e078e6cfb46ae9dbe57c20be666d5, 38681904c36540a4876519ec5ffac6b6, a5813459b1d94e089ffbe751f4310de7, b7def2bd3a74420d9deec19b32f541ec
I20260812 08:03:07.689838 25849 raft_consensus.cc:2749] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260812 08:03:07.706239 25660 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Bootstrap replayed 1/1 log segments. Stats: ops{read=110 overwritten=2 applied=97 ignored=0} inserts{seen=88 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 11 replicates
I20260812 08:03:07.706872 25660 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Bootstrap complete.
W20260812 08:03:07.707065 25660 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 14ms of latency in SegmentAllocator::Sync()
I20260812 08:03:07.721598 25660 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Time spent bootstrapping tablet: real 0.095s	user 0.063s	sys 0.012s
I20260812 08:03:07.723214 25660 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 19 FOLLOWER]: Replica starting. Triggering 11 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:07.726101 25660 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 19 FOLLOWER]: Becoming Follower/Learner. State: Replica: 285e078e6cfb46ae9dbe57c20be666d5, State: Initialized, Role: FOLLOWER
I20260812 08:03:07.726599 25660 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 97, Last appended: 19.108, Last appended by leader: 108, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:07.727321 25844 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
I20260812 08:03:07.727537 25660 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
W20260812 08:03:07.731138 25699 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44370: Illegal state: replica 285e078e6cfb46ae9dbe57c20be666d5 is not leader of this config: current role FOLLOWER
I20260812 08:03:07.732996 24895 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:07.733108 24895 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:07.733922 24895 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:07.734184 25034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 21 candidate_status { last_received { term: 19 index: 107 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d" is_pre_election: true
I20260812 08:03:07.734336 25034 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 20.
I20260812 08:03:07.735066 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 21 candidate_status { last_received { term: 19 index: 107 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf" is_pre_election: true
I20260812 08:03:07.735143 25297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 21 candidate_status { last_received { term: 19 index: 107 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b" is_pre_election: true
I20260812 08:03:07.735211 23284 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 20.
I20260812 08:03:07.735317 25297 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 20.
I20260812 08:03:07.736266 24710 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 294e0e1970eb4e58bed426de35babb6b, 5b4c3ad48b794617893c88a077be1b8d, a5813459b1d94e089ffbe751f4310de7; no voters: 
I20260812 08:03:07.737140 24895 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260812 08:03:07.737197 24895 raft_consensus.cc:493] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:03:07.737231 24895 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 20 FOLLOWER]: Advancing to term 21
I20260812 08:03:07.738447 24895 raft_consensus.cc:515] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:07.739042 24895 leader_election.cc:290] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 21 election: Requested vote from peers b7def2bd3a74420d9deec19b32f541ec (127.18.42.68:36499), 5b4c3ad48b794617893c88a077be1b8d (127.18.42.66:32855), 294e0e1970eb4e58bed426de35babb6b (127.18.42.67:40205), 285e078e6cfb46ae9dbe57c20be666d5 (127.18.42.69:34819), 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785), 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
W20260812 08:03:07.739363 24710 leader_election.cc:336] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785): Network error: Client connection negotiation failed: client connection to 127.18.42.70:39785: connect: Connection refused (error 111)
I20260812 08:03:07.739434 23284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 21 candidate_status { last_received { term: 19 index: 107 } } ignore_live_leader: false dest_uuid: "0e10b84812514a76ab7225dfbf9329bf"
I20260812 08:03:07.739492 25034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 21 candidate_status { last_received { term: 19 index: 107 } } ignore_live_leader: false dest_uuid: "5b4c3ad48b794617893c88a077be1b8d"
I20260812 08:03:07.739560 23284 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 20 FOLLOWER]: Advancing to term 21
I20260812 08:03:07.739612 25034 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 20 FOLLOWER]: Advancing to term 21
I20260812 08:03:07.740239 25540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 21 candidate_status { last_received { term: 19 index: 107 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec" is_pre_election: true
I20260812 08:03:07.740430 25540 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 19.
I20260812 08:03:07.740681 25034 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 21.
I20260812 08:03:07.740818 23284 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 21.
I20260812 08:03:07.741138 25538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 21 candidate_status { last_received { term: 19 index: 107 } } ignore_live_leader: false dest_uuid: "b7def2bd3a74420d9deec19b32f541ec"
I20260812 08:03:07.741297 25538 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 19 FOLLOWER]: Advancing to term 21
I20260812 08:03:07.741829 25297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 21 candidate_status { last_received { term: 19 index: 107 } } ignore_live_leader: false dest_uuid: "294e0e1970eb4e58bed426de35babb6b"
I20260812 08:03:07.741976 25297 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 20 FOLLOWER]: Advancing to term 21
I20260812 08:03:07.742982 25538 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 21.
I20260812 08:03:07.743005 25297 raft_consensus.cc:2468] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5813459b1d94e089ffbe751f4310de7 in term 21.
I20260812 08:03:07.743379 24711 leader_election.cc:304] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0e10b84812514a76ab7225dfbf9329bf, 294e0e1970eb4e58bed426de35babb6b, 5b4c3ad48b794617893c88a077be1b8d, a5813459b1d94e089ffbe751f4310de7; no voters: 
I20260812 08:03:07.743562 24895 raft_consensus.cc:2804] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 21 FOLLOWER]: Leader election won for term 21
I20260812 08:03:07.743767 24895 raft_consensus.cc:697] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [term 21 LEADER]: Becoming Leader. State: Replica: a5813459b1d94e089ffbe751f4310de7, State: Running, Role: LEADER
I20260812 08:03:07.744095 24895 consensus_queue.cc:237] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 97, Committed index: 97, Last appended: 19.107, Last appended by leader: 107, Current term: 21, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:07.744480 25767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 21 candidate_status { last_received { term: 19 index: 107 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5" is_pre_election: true
I20260812 08:03:07.744721 25767 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5813459b1d94e089ffbe751f4310de7 for term 21 because replica has last-logged OpId of term: 19 index: 108, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 107.
I20260812 08:03:07.744501 25766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b14eb091f7476d9d7682e503745c93" candidate_uuid: "a5813459b1d94e089ffbe751f4310de7" candidate_term: 21 candidate_status { last_received { term: 19 index: 107 } } ignore_live_leader: false dest_uuid: "285e078e6cfb46ae9dbe57c20be666d5"
I20260812 08:03:07.745023 25766 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 19 FOLLOWER]: Advancing to term 21
W20260812 08:03:07.745788 25153 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 10ms of latency in SegmentAllocator::Sync()
I20260812 08:03:07.746920 25766 raft_consensus.cc:2410] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a5813459b1d94e089ffbe751f4310de7 for term 21 because replica has last-logged OpId of term: 19 index: 108, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 107.
W20260812 08:03:07.748001 24710 leader_election.cc:336] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785): Network error: Client connection negotiation failed: client connection to 127.18.42.70:39785: connect: Connection refused (error 111)
I20260812 08:03:07.747926 18661 catalog_manager.cc:5719] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 reported cstate change: term changed from 20 to 21, leader changed from 38681904c36540a4876519ec5ffac6b6 (127.18.42.70) to a5813459b1d94e089ffbe751f4310de7 (127.18.42.65). New cstate: current_term: 21 leader_uuid: "a5813459b1d94e089ffbe751f4310de7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } health_report { overall_health: UNKNOWN } } }
W20260812 08:03:07.772437 25153 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 110ms of latency in SegmentAllocator::Sync()
I20260812 08:03:07.773545 23284 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 21 FOLLOWER]: Refusing update from remote peer a5813459b1d94e089ffbe751f4310de7: Log matching property violated. Preceding OpId in replica: term: 15 index: 95. Preceding OpId from leader: term: 21 index: 109. (index mismatch)
W20260812 08:03:07.773835 24710 consensus_peers.cc:597] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 -> Peer 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785): Couldn't send request to peer 38681904c36540a4876519ec5ffac6b6. Status: Network error: Client connection negotiation failed: client connection to 127.18.42.70:39785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 08:03:07.773869 25766 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 21 FOLLOWER]: Refusing update from remote peer a5813459b1d94e089ffbe751f4310de7: Log matching property violated. Preceding OpId in replica: term: 19 index: 108. Preceding OpId from leader: term: 21 index: 109. (index mismatch)
W20260812 08:03:07.774080 25848 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:03:07.774323 25848 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:03:07.774369 25848 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:03:07.774446 25848 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:07.774492 25848 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:07.774555 25848 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260812 08:03:07.774892 25297 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b [term 21 FOLLOWER]: Refusing update from remote peer a5813459b1d94e089ffbe751f4310de7: Log matching property violated. Preceding OpId in replica: term: 15 index: 104. Preceding OpId from leader: term: 21 index: 109. (index mismatch)
I20260812 08:03:07.774950 25867 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 108, Last known committed idx: 86, Time since last communication: 0.000s
I20260812 08:03:07.775264 24895 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 108, Last known committed idx: 97, Time since last communication: 0.000s
I20260812 08:03:07.775369 24895 consensus_queue.cc:1243] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [LEADER]: Peer 285e078e6cfb46ae9dbe57c20be666d5 log is divergent from this leader: its last log entry 19.108 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 97
I20260812 08:03:07.775676 25882 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 108, Last known committed idx: 94, Time since last communication: 0.000s
I20260812 08:03:07.776888 25766 raft_consensus.cc:1217] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5 [term 21 FOLLOWER]: Deduplicated request from leader. Original: 15.97->[15.98-21.109]   Dedup: 19.107->[21.108-21.109]
I20260812 08:03:07.777032 25766 pending_rounds.cc:85] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Aborting all ops after (but not including) 107
I20260812 08:03:07.777081 25766 pending_rounds.cc:107] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Aborting uncommitted WRITE_OP operation due to leader change: 19.108
I20260812 08:03:07.777119 25538 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec [term 21 FOLLOWER]: Refusing update from remote peer a5813459b1d94e089ffbe751f4310de7: Log matching property violated. Preceding OpId in replica: term: 19 index: 107. Preceding OpId from leader: term: 21 index: 109. (index mismatch)
I20260812 08:03:07.777788 25034 raft_consensus.cc:1275] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d [term 21 FOLLOWER]: Refusing update from remote peer a5813459b1d94e089ffbe751f4310de7: Log matching property violated. Preceding OpId in replica: term: 18 index: 106. Preceding OpId from leader: term: 21 index: 109. (index mismatch)
I20260812 08:03:07.778439 25880 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 108, Last known committed idx: 97, Time since last communication: 0.000s
W20260812 08:03:07.778903 25848 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:07.779055 25848 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.70
I20260812 08:03:07.780794 24895 consensus_queue.cc:1048] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 108, Last known committed idx: 97, Time since last communication: 0.000s
I20260812 08:03:07.784560 25848 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.70:39785
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data/info.pb
--webserver_interface=127.18.42.70
--webserver_port=40739
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.25848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.70

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
W20260812 08:03:07.785705 25889 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 50ms of latency in SegmentAllocator::Sync()
I20260812 08:03:07.786257 25848 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:07.787560 25848 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:07.787573 25903 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 3ms of latency in SegmentAllocator::Sync()
W20260812 08:03:07.791534 25904 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Injecting 69ms of latency in SegmentAllocator::Sync()
E20260812 08:03:07.792273 25848 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:03:07.803809 25901 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Injecting 37ms of latency in SegmentAllocator::Sync()
W20260812 08:03:07.805478 25907 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Injecting 66ms of latency in SegmentAllocator::Sync()
W20260812 08:03:07.806190 25941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:07.806548 25936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:07.807024 25938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:07.808120 25848 server_base.cc:1061] running on GCE node
I20260812 08:03:07.808609 25848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20260812 08:03:07.808790 25903 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 24ms of latency in SegmentAllocator::Sync()
I20260812 08:03:07.809262 25848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:07.810434 25848 hybrid_clock.cc:648] HybridClock initialized: now 1786521787810398 us; error 45 us; skew 500 ppm
I20260812 08:03:07.812129 25848 webserver.cc:533] Webserver started at http://127.18.42.70:40739/ using document root <none> and password file <none>
I20260812 08:03:07.812758 25848 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:07.812840 25848 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:07.816123 25848 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:07.818105 25963 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:07.819098 25848 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:07.819214 25848 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/wal
uuid: "38681904c36540a4876519ec5ffac6b6"
format_stamp: "Formatted at 2026-08-12 08:02:59 on dist-test-slave-zr1t"
I20260812 08:03:07.819590 25848 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:07.836932 25848 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:07.837740 25848 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:07.837954 25848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:07.838655 25848 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
W20260812 08:03:07.843007 25903 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 105ms of latency in SegmentAllocator::Sync()
I20260812 08:03:07.843130 25971 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260812 08:03:07.843843 25889 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 9ms of latency in SegmentAllocator::Sync()
I20260812 08:03:07.844172 25865 mvcc.cc:204] Tried to move back new op lower bound from 7317593242713776128 to 7317593239311720448. Current Snapshot: MvccSnapshot[applied={T|T < 7317593242713776128 or (T in {7317593242713776128})}]
W20260812 08:03:07.847199 25901 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Injecting 46ms of latency in SegmentAllocator::Sync()
I20260812 08:03:07.847919 25848 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:07.847993 25848 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.008s	user 0.001s	sys 0.000s
I20260812 08:03:07.848070 25848 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:07.849632 25848 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:07.849699 25848 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:07.850028 25971 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Bootstrap starting.
W20260812 08:03:07.856689 25889 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 69ms of latency in SegmentAllocator::Sync()
I20260812 08:03:07.862092 25848 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.70:39785
I20260812 08:03:07.863431 25848 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data/info.pb
I20260812 08:03:07.866189 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 25848
W20260812 08:03:07.866226 25904 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Injecting 116ms of latency in SegmentAllocator::Sync()
I20260812 08:03:07.866358 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 23162
I20260812 08:03:07.865139 25971 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:07.876670 26150 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.70:39785 every 8 connection(s)
W20260812 08:03:07.883517 25907 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Injecting 13ms of latency in SegmentAllocator::Sync()
W20260812 08:03:07.884405 25153 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 54ms of latency in SegmentAllocator::Sync()
W20260812 08:03:07.892565 25422 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:03:07.891825 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.18.42.71:35647 (local address 127.18.42.65:39393)
0812 08:03:07.892011 (+   186us) negotiation.cc:107] Waiting for socket to connect
0812 08:03:07.892026 (+    15us) client_negotiation.cc:175] Beginning negotiation
0812 08:03:07.892098 (+    72us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 08:03:07.892468 (+   370us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.42.71:35647: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":106}
W20260812 08:03:07.892747 24711 consensus_peers.cc:597] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7 -> Peer 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647): Couldn't send request to peer 0e10b84812514a76ab7225dfbf9329bf. Status: Network error: Client connection negotiation failed: client connection to 127.18.42.71:35647: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
I20260812 08:03:07.894233 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.71:35647
--local_ip_for_outbound_sockets=127.18.42.71
--tserver_master_addrs=127.18.42.126:35213
--webserver_port=37531
--webserver_interface=127.18.42.71
--builtin_ntp_servers=127.18.42.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_inject_latency
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--raft_heartbeat_interval_ms=100
--log_preallocate_segments=false with env {}
W20260812 08:03:07.907478 25907 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Injecting 99ms of latency in SegmentAllocator::Sync()
I20260812 08:03:07.909459 26159 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:03:07.909725 26159 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:07.910387 26159 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
I20260812 08:03:07.912889 18661 ts_manager.cc:194] Re-registered known tserver with Master: 38681904c36540a4876519ec5ffac6b6 (127.18.42.70:39785)
I20260812 08:03:07.914237 18661 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.70:52339
W20260812 08:03:07.916708 25901 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Injecting 62ms of latency in SegmentAllocator::Sync()
I20260812 08:03:07.929630 20493 meta_cache.cc:1510] marking tablet server 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647) as failed
W20260812 08:03:07.930514 25889 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 64ms of latency in SegmentAllocator::Sync()
W20260812 08:03:07.939936 25153 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 72ms of latency in SegmentAllocator::Sync()
I20260812 08:03:07.946234 25971 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Bootstrap replayed 1/1 log segments. Stats: ops{read=110 overwritten=2 applied=97 ignored=0} inserts{seen=88 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 11 replicates
I20260812 08:03:07.946954 25971 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Bootstrap complete.
W20260812 08:03:07.947211 25971 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Injecting 77ms of latency in SegmentAllocator::Sync()
W20260812 08:03:07.956974 25903 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 3ms of latency in SegmentAllocator::Sync()
W20260812 08:03:07.981717 25901 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Injecting 47ms of latency in SegmentAllocator::Sync()
W20260812 08:03:08.013175 25907 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Injecting 26ms of latency in SegmentAllocator::Sync()
W20260812 08:03:08.014072 25153 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 55ms of latency in SegmentAllocator::Sync()
W20260812 08:03:08.018481 25889 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 57ms of latency in SegmentAllocator::Sync()
I20260812 08:03:08.024787 25971 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Time spent bootstrapping tablet: real 0.175s	user 0.078s	sys 0.000s
W20260812 08:03:08.025787 20493 scanner-internal.cc:458] Time spent opening tablet: real 3.743s	user 0.004s	sys 0.000s
I20260812 08:03:08.028398 25971 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 20 FOLLOWER]: Replica starting. Triggering 11 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:08.032572 25971 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 20 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38681904c36540a4876519ec5ffac6b6, State: Initialized, Role: FOLLOWER
W20260812 08:03:08.032876 25901 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 294e0e1970eb4e58bed426de35babb6b: Injecting 125ms of latency in SegmentAllocator::Sync()
I20260812 08:03:08.033119 25971 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 97, Last appended: 20.108, Last appended by leader: 108, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:08.034125 25971 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260812 08:03:08.034720 26159 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
W20260812 08:03:08.038228 25903 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 68ms of latency in SegmentAllocator::Sync()
W20260812 08:03:08.038679 26166 flags.cc:432] Enabled unsafe flag: --log_inject_latency=true
W20260812 08:03:08.038932 26166 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_mean=30
W20260812 08:03:08.038970 26166 flags.cc:432] Enabled unsafe flag: --log_inject_latency_ms_stddev=60
W20260812 08:03:08.039032 26166 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:08.039074 26166 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:08.039139 26166 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:08.043690 26166 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:08.043828 26166 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.71
W20260812 08:03:08.045190 25907 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Injecting 38ms of latency in SegmentAllocator::Sync()
I20260812 08:03:08.049322 26166 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_inject_latency=true
--log_inject_latency_ms_mean=30
--log_inject_latency_ms_stddev=60
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.71:35647
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data/info.pb
--webserver_interface=127.18.42.71
--webserver_port=37531
--tserver_master_addrs=127.18.42.126:35213
--never_fsync=true
--heap_profile_path=/tmp/kudu.26166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.71

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:08.050712 26166 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:08.052081 26166 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:03:08.055536 26166 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:03:08.059612 26189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:08.059612 26184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:08.059940 26186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:08.060568 26166 server_base.cc:1061] running on GCE node
I20260812 08:03:08.060961 26166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:08.061601 26166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:08.062785 26166 hybrid_clock.cc:648] HybridClock initialized: now 1786521788062754 us; error 44 us; skew 500 ppm
I20260812 08:03:08.065001 26166 webserver.cc:533] Webserver started at http://127.18.42.71:37531/ using document root <none> and password file <none>
I20260812 08:03:08.065739 26166 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:08.065814 26166 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:08.070080 26166 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.003s	sys 0.000s
W20260812 08:03:08.070152 25153 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 144ms of latency in SegmentAllocator::Sync()
I20260812 08:03:08.072373 26201 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:08.073570 26166 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:03:08.073681 26166 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/wal
uuid: "0e10b84812514a76ab7225dfbf9329bf"
format_stamp: "Formatted at 2026-08-12 08:02:59 on dist-test-slave-zr1t"
I20260812 08:03:08.074146 26166 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260812 08:03:08.080384 25889 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 50ms of latency in SegmentAllocator::Sync()
W20260812 08:03:08.087420 25907 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 5b4c3ad48b794617893c88a077be1b8d: Injecting 28ms of latency in SegmentAllocator::Sync()
I20260812 08:03:08.101481 26166 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:08.102350 26166 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:08.102555 26166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:08.103308 26166 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:08.104857 26210 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260812 08:03:08.109758 25903 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P b7def2bd3a74420d9deec19b32f541ec: Injecting 45ms of latency in SegmentAllocator::Sync()
I20260812 08:03:08.109845 26166 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:08.109894 26166 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.005s	user 0.000s	sys 0.000s
I20260812 08:03:08.109968 26166 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:08.111861 26166 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:08.111918 26166 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:03:08.112321 26210 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Bootstrap starting.
I20260812 08:03:08.124711 26166 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.71:35647
I20260812 08:03:08.125732 26387 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.71:35647 every 8 connection(s)
I20260812 08:03:08.125990 26166 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data/info.pb
I20260812 08:03:08.128741 26210 log.cc:826] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:08.129170 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 26166
I20260812 08:03:08.132952 26047 raft_consensus.cc:3060] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 20 FOLLOWER]: Advancing to term 21
W20260812 08:03:08.133759 25889 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 285e078e6cfb46ae9dbe57c20be666d5: Injecting 11ms of latency in SegmentAllocator::Sync()
I20260812 08:03:08.135020 26047 pending_rounds.cc:85] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Aborting all ops after (but not including) 107
I20260812 08:03:08.135106 26047 pending_rounds.cc:107] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Aborting uncommitted NO_OP operation due to leader change: 20.108
I20260812 08:03:08.135180 26047 raft_consensus.cc:2889] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260812 08:03:08.153470 26388 heartbeater.cc:344] Connected to a master server at 127.18.42.126:35213
I20260812 08:03:08.153771 26388 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:08.154388 26388 heartbeater.cc:507] Master 127.18.42.126:35213 requested a full tablet report, sending...
W20260812 08:03:08.155452 26400 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 38681904c36540a4876519ec5ffac6b6: Injecting 1ms of latency in SegmentAllocator::Sync()
I20260812 08:03:08.157179 18661 ts_manager.cc:194] Re-registered known tserver with Master: 0e10b84812514a76ab7225dfbf9329bf (127.18.42.71:35647)
I20260812 08:03:08.158164 18661 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.71:34435
I20260812 08:03:08.202216 26210 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Bootstrap replayed 1/1 log segments. Stats: ops{read=123 overwritten=3 applied=109 ignored=0} inserts{seen=96 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 11 replicates
I20260812 08:03:08.202718 26210 tablet_bootstrap.cc:492] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Bootstrap complete.
W20260812 08:03:08.202900 26210 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Injecting 21ms of latency in SegmentAllocator::Sync()
W20260812 08:03:08.214967 25153 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P a5813459b1d94e089ffbe751f4310de7: Injecting 7ms of latency in SegmentAllocator::Sync()
I20260812 08:03:08.224328 26210 ts_tablet_manager.cc:1403] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Time spent bootstrapping tablet: real 0.112s	user 0.082s	sys 0.004s
I20260812 08:03:08.226647 26210 raft_consensus.cc:359] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 21 FOLLOWER]: Replica starting. Triggering 11 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:08.230029 26210 raft_consensus.cc:740] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 21 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e10b84812514a76ab7225dfbf9329bf, State: Initialized, Role: FOLLOWER
I20260812 08:03:08.230540 26210 consensus_queue.cc:260] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 109, Last appended: 21.120, Last appended by leader: 120, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7def2bd3a74420d9deec19b32f541ec" member_type: VOTER last_known_addr { host: "127.18.42.68" port: 36499 } } peers { permanent_uuid: "a5813459b1d94e089ffbe751f4310de7" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45661 } } peers { permanent_uuid: "5b4c3ad48b794617893c88a077be1b8d" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 32855 } } peers { permanent_uuid: "294e0e1970eb4e58bed426de35babb6b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 40205 } } peers { permanent_uuid: "285e078e6cfb46ae9dbe57c20be666d5" member_type: VOTER last_known_addr { host: "127.18.42.69" port: 34819 } } peers { permanent_uuid: "38681904c36540a4876519ec5ffac6b6" member_type: VOTER last_known_addr { host: "127.18.42.70" port: 39785 } } peers { permanent_uuid: "0e10b84812514a76ab7225dfbf9329bf" member_type: VOTER last_known_addr { host: "127.18.42.71" port: 35647 } }
I20260812 08:03:08.231248 26388 heartbeater.cc:499] Master 127.18.42.126:35213 was elected leader, sending a full tablet report...
I20260812 08:03:08.231455 26210 ts_tablet_manager.cc:1434] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260812 08:03:08.255270 26306 raft_consensus.cc:1217] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf [term 21 FOLLOWER]: Deduplicated request from leader. Original: 21.109->[21.110-21.141]   Dedup: 21.120->[21.121-21.141]
W20260812 08:03:08.260771 26407 log.cc:510] T 50b14eb091f7476d9d7682e503745c93 P 0e10b84812514a76ab7225dfbf9329bf: Injecting 13ms of latency in SegmentAllocator::Sync()
W20260812 08:03:11.289139 20495 scanner-internal.cc:458] Time spent opening tablet: real 7.007s	user 0.005s	sys 0.000s
I20260812 08:03:11.351784 26270 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:03:11.351864 24993 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:03:11.356233 26045 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:03:11.362177 25731 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:03:11.363590 25505 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:03:11.366325 25265 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:03:11.375650 24753 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
Master Summary
               UUID               |       Address       | Status
----------------------------------+---------------------+---------
 af9c6f91894742328cd99747cf1207fc | 127.18.42.126:35213 | HEALTHY

Unusual flags for Master:
               Flag               |                                                                                                 Value                                                                                                 |      Tags       |         Master
----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 ipki_ca_key_size                 | 768                                                                                                                                                                                                   | experimental    | all 1 server(s) checked
 ipki_server_key_size             | 768                                                                                                                                                                                                   | experimental    | all 1 server(s) checked
 never_fsync                      | true                                                                                                                                                                                                  | unsafe,advanced | all 1 server(s) checked
 openssl_security_level_override  | 0                                                                                                                                                                                                     | unsafe,hidden   | all 1 server(s) checked
 rpc_reuseport                    | true                                                                                                                                                                                                  | experimental    | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports | true                                                                                                                                                                                                  | unsafe          | all 1 server(s) checked
 server_dump_info_format          | pb                                                                                                                                                                                                    | hidden          | all 1 server(s) checked
 server_dump_info_path            | /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/master-0/data/info.pb | hidden          | all 1 server(s) checked
 tsk_num_rsa_bits                 | 512                                                                                                                                                                                                   | experimental    | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |       Value        |         Master
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.18.42.84:46431 | all 1 server(s) checked
 time_source         | builtin            | all 1 server(s) checked

Tablet Server Summary
               UUID               |      Address       | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
 0e10b84812514a76ab7225dfbf9329bf | 127.18.42.71:35647 | HEALTHY | <none>   |       0        |       0
 285e078e6cfb46ae9dbe57c20be666d5 | 127.18.42.69:34819 | HEALTHY | <none>   |       0        |       0
 294e0e1970eb4e58bed426de35babb6b | 127.18.42.67:40205 | HEALTHY | <none>   |       0        |       0
 38681904c36540a4876519ec5ffac6b6 | 127.18.42.70:39785 | HEALTHY | <none>   |       0        |       0
 5b4c3ad48b794617893c88a077be1b8d | 127.18.42.66:32855 | HEALTHY | <none>   |       0        |       0
 a5813459b1d94e089ffbe751f4310de7 | 127.18.42.65:45661 | HEALTHY | <none>   |       1        |       0
 b7def2bd3a74420d9deec19b32f541ec | 127.18.42.68:36499 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       7

Unusual flags for Tablet Server:
               Flag               |                                                                                               Value                                                                                               |      Tags       |      Tablet Server
----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 ipki_server_key_size             | 768                                                                                                                                                                                               | experimental    | all 7 server(s) checked
 local_ip_for_outbound_sockets    | 127.18.42.65                                                                                                                                                                                      | experimental    | 127.18.42.65:45661
 local_ip_for_outbound_sockets    | 127.18.42.66                                                                                                                                                                                      | experimental    | 127.18.42.66:32855
 local_ip_for_outbound_sockets    | 127.18.42.67                                                                                                                                                                                      | experimental    | 127.18.42.67:40205
 local_ip_for_outbound_sockets    | 127.18.42.68                                                                                                                                                                                      | experimental    | 127.18.42.68:36499
 local_ip_for_outbound_sockets    | 127.18.42.69                                                                                                                                                                                      | experimental    | 127.18.42.69:34819
 local_ip_for_outbound_sockets    | 127.18.42.70                                                                                                                                                                                      | experimental    | 127.18.42.70:39785
 local_ip_for_outbound_sockets    | 127.18.42.71                                                                                                                                                                                      | experimental    | 127.18.42.71:35647
 log_inject_latency               | true                                                                                                                                                                                              | runtime,unsafe  | all 7 server(s) checked
 log_inject_latency_ms_mean       | 30                                                                                                                                                                                                | runtime,unsafe  | all 7 server(s) checked
 log_inject_latency_ms_stddev     | 60                                                                                                                                                                                                | runtime,unsafe  | all 7 server(s) checked
 never_fsync                      | true                                                                                                                                                                                              | unsafe,advanced | all 7 server(s) checked
 openssl_security_level_override  | 0                                                                                                                                                                                                 | unsafe,hidden   | all 7 server(s) checked
 rpc_server_allow_ephemeral_ports | true                                                                                                                                                                                              | unsafe          | all 7 server(s) checked
 server_dump_info_format          | pb                                                                                                                                                                                                | hidden          | all 7 server(s) checked
 server_dump_info_path            | /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-0/data/info.pb | hidden          | 127.18.42.65:45661
 server_dump_info_path            | /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-1/data/info.pb | hidden          | 127.18.42.66:32855
 server_dump_info_path            | /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-2/data/info.pb | hidden          | 127.18.42.67:40205
 server_dump_info_path            | /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-3/data/info.pb | hidden          | 127.18.42.68:36499
 server_dump_info_path            | /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-4/data/info.pb | hidden          | 127.18.42.69:34819
 server_dump_info_path            | /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-5/data/info.pb | hidden          | 127.18.42.70:39785
 server_dump_info_path            | /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.InsertUniqueKeysWithCrashyNodes.1786521777590645-18601-0/raft_consensus-itest-crashy-nodes-cluster/ts-6/data/info.pb | hidden          | 127.18.42.71:35647

Flags of checked categories for Tablet Server:
        Flag         |       Value        |      Tablet Server
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.18.42.84:46431 | all 7 server(s) checked
 time_source         | builtin            | all 7 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 8 server(s) checked

Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
     Name      | RF | Status  | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
 test-workload | 7  | HEALTHY | 1             | 1       | 0          | 0                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 7
 Tables         | 1
 Tablets        | 1
 Replicas       | 7

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

OK
I20260812 08:03:11.412519 18601 log_verifier.cc:126] Checking tablet 50b14eb091f7476d9d7682e503745c93
I20260812 08:03:11.481143 18601 log_verifier.cc:177] Verified matching terms for 141 ops in tablet 50b14eb091f7476d9d7682e503745c93
I20260812 08:03:11.531414 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 24671
I20260812 08:03:11.556951 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 24888
I20260812 08:03:11.572115 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 25118
I20260812 08:03:11.596729 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 25375
I20260812 08:03:11.611204 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 25621
I20260812 08:03:11.635281 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 25848
I20260812 08:03:11.647190 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 26166
I20260812 08:03:11.656808 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 18622
2026-08-12T08:03:11Z chronyd exiting
[       OK ] RaftConsensusITest.InsertUniqueKeysWithCrashyNodes (14084 ms)
[ RUN      ] RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue
I20260812 08:03:11.688800 18601 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:03:11.688868 18601 ts_itest-base.cc:116] --------------
I20260812 08:03:11.688887 18601 ts_itest-base.cc:117] 3 tablet servers
I20260812 08:03:11.688905 18601 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:03:11.688932 18601 ts_itest-base.cc:119] --------------
2026-08-12T08:03:11Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:03:11Z Disabled control of system clock
I20260812 08:03:11.698216 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.42.126:34973
--webserver_interface=127.18.42.126
--webserver_port=0
--builtin_ntp_servers=127.18.42.84:45543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.42.126:34973
--catalog_manager_wait_for_new_tablets_to_elect_leader=false with env {}
W20260812 08:03:11.800958 26488 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:11.801162 26488 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:11.801200 26488 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:11.804375 26488 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:03:11.804433 26488 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:11.804453 26488 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:03:11.804471 26488 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:03:11.808127 26488 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:45543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
--catalog_manager_wait_for_new_tablets_to_elect_leader=false
--ipki_ca_key_size=768
--master_addresses=127.18.42.126:34973
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.42.126:34973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.18.42.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.26488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:11.809163 26488 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:11.810227 26488 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:11.815762 26500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:11.815775 26508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:11.815764 26504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:11.816260 26488 server_base.cc:1061] running on GCE node
I20260812 08:03:11.816852 26488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:11.817838 26488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:11.819031 26488 hybrid_clock.cc:648] HybridClock initialized: now 1786521791819008 us; error 44 us; skew 500 ppm
I20260812 08:03:11.820883 26488 webserver.cc:533] Webserver started at http://127.18.42.126:43029/ using document root <none> and password file <none>
I20260812 08:03:11.821471 26488 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:11.821587 26488 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:11.821817 26488 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:11.823390 26488 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "743cd59589ee455ca4cb6db686cdfb49"
format_stamp: "Formatted at 2026-08-12 08:03:11 on dist-test-slave-zr1t"
I20260812 08:03:11.823881 26488 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "743cd59589ee455ca4cb6db686cdfb49"
format_stamp: "Formatted at 2026-08-12 08:03:11 on dist-test-slave-zr1t"
I20260812 08:03:11.827164 26488 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:03:11.829303 26517 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:11.830421 26488 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:03:11.830576 26488 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
uuid: "743cd59589ee455ca4cb6db686cdfb49"
format_stamp: "Formatted at 2026-08-12 08:03:11 on dist-test-slave-zr1t"
I20260812 08:03:11.830689 26488 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:11.852257 26488 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:11.852989 26488 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:11.853199 26488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:11.861192 26591 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.126:34973 every 8 connection(s)
I20260812 08:03:11.861204 26488 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.126:34973
I20260812 08:03:11.862373 26488 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:03:11.863233 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 26488
I20260812 08:03:11.863369 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:03:11.865698 26592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:11.870687 26592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 743cd59589ee455ca4cb6db686cdfb49: Bootstrap starting.
I20260812 08:03:11.872985 26592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 743cd59589ee455ca4cb6db686cdfb49: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:11.873795 26592 log.cc:826] T 00000000000000000000000000000000 P 743cd59589ee455ca4cb6db686cdfb49: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:11.875372 26592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 743cd59589ee455ca4cb6db686cdfb49: No bootstrap required, opened a new log
I20260812 08:03:11.877971 26592 raft_consensus.cc:359] T 00000000000000000000000000000000 P 743cd59589ee455ca4cb6db686cdfb49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "743cd59589ee455ca4cb6db686cdfb49" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 34973 } }
I20260812 08:03:11.878165 26592 raft_consensus.cc:385] T 00000000000000000000000000000000 P 743cd59589ee455ca4cb6db686cdfb49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:11.878258 26592 raft_consensus.cc:740] T 00000000000000000000000000000000 P 743cd59589ee455ca4cb6db686cdfb49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 743cd59589ee455ca4cb6db686cdfb49, State: Initialized, Role: FOLLOWER
I20260812 08:03:11.878696 26592 consensus_queue.cc:260] T 00000000000000000000000000000000 P 743cd59589ee455ca4cb6db686cdfb49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "743cd59589ee455ca4cb6db686cdfb49" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 34973 } }
I20260812 08:03:11.878836 26592 raft_consensus.cc:399] T 00000000000000000000000000000000 P 743cd59589ee455ca4cb6db686cdfb49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:03:11.878901 26592 raft_consensus.cc:493] T 00000000000000000000000000000000 P 743cd59589ee455ca4cb6db686cdfb49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:03:11.879004 26592 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 743cd59589ee455ca4cb6db686cdfb49 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:11.879756 26592 raft_consensus.cc:515] T 00000000000000000000000000000000 P 743cd59589ee455ca4cb6db686cdfb49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "743cd59589ee455ca4cb6db686cdfb49" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 34973 } }
I20260812 08:03:11.880077 26592 leader_election.cc:304] T 00000000000000000000000000000000 P 743cd59589ee455ca4cb6db686cdfb49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 743cd59589ee455ca4cb6db686cdfb49; no voters: 
I20260812 08:03:11.880363 26592 leader_election.cc:290] T 00000000000000000000000000000000 P 743cd59589ee455ca4cb6db686cdfb49 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:03:11.880541 26602 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 743cd59589ee455ca4cb6db686cdfb49 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:11.880765 26602 raft_consensus.cc:697] T 00000000000000000000000000000000 P 743cd59589ee455ca4cb6db686cdfb49 [term 1 LEADER]: Becoming Leader. State: Replica: 743cd59589ee455ca4cb6db686cdfb49, State: Running, Role: LEADER
I20260812 08:03:11.881060 26602 consensus_queue.cc:237] T 00000000000000000000000000000000 P 743cd59589ee455ca4cb6db686cdfb49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "743cd59589ee455ca4cb6db686cdfb49" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 34973 } }
I20260812 08:03:11.881476 26592 sys_catalog.cc:565] T 00000000000000000000000000000000 P 743cd59589ee455ca4cb6db686cdfb49 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:11.882623 26603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 743cd59589ee455ca4cb6db686cdfb49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "743cd59589ee455ca4cb6db686cdfb49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "743cd59589ee455ca4cb6db686cdfb49" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 34973 } } }
I20260812 08:03:11.882766 26603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 743cd59589ee455ca4cb6db686cdfb49 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:11.883154 26619 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:03:11.883167 26607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 743cd59589ee455ca4cb6db686cdfb49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 743cd59589ee455ca4cb6db686cdfb49. Latest consensus state: current_term: 1 leader_uuid: "743cd59589ee455ca4cb6db686cdfb49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "743cd59589ee455ca4cb6db686cdfb49" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 34973 } } }
I20260812 08:03:11.883333 26607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 743cd59589ee455ca4cb6db686cdfb49 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:11.886605 26619 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:03:11.892189 26619 catalog_manager.cc:1383] Generated new cluster ID: 0895756feb844e209f2b312d1943640b
I20260812 08:03:11.892254 26619 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:03:11.909667 26619 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:03:11.910502 26619 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:03:11.919945 26619 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 743cd59589ee455ca4cb6db686cdfb49: Generated new TSK 0
I20260812 08:03:11.920580 26619 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:03:11.930835 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.65:0
--local_ip_for_outbound_sockets=127.18.42.65
--webserver_interface=127.18.42.65
--webserver_port=0
--tserver_master_addrs=127.18.42.126:34973
--builtin_ntp_servers=127.18.42.84:45543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false with env {}
W20260812 08:03:12.036561 26633 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 08:03:12.036818 26633 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:12.036890 26633 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:12.036972 26633 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:12.040290 26633 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:12.040446 26633 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.65
I20260812 08:03:12.044291 26633 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:45543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.18.42.65
--webserver_port=0
--tserver_master_addrs=127.18.42.126:34973
--never_fsync=true
--heap_profile_path=/tmp/kudu.26633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:12.045300 26633 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:12.046447 26633 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:12.052536 26644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:12.052829 26633 server_base.cc:1061] running on GCE node
W20260812 08:03:12.052536 26645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:12.052984 26649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:12.053341 26633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:12.053978 26633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:12.055163 26633 hybrid_clock.cc:648] HybridClock initialized: now 1786521792055139 us; error 44 us; skew 500 ppm
I20260812 08:03:12.056953 26633 webserver.cc:533] Webserver started at http://127.18.42.65:43329/ using document root <none> and password file <none>
I20260812 08:03:12.057452 26633 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:12.057503 26633 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:12.057746 26633 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:12.059186 26633 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "cf864e9e0db643aea3ab448b5afe38e9"
format_stamp: "Formatted at 2026-08-12 08:03:12 on dist-test-slave-zr1t"
I20260812 08:03:12.059600 26633 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "cf864e9e0db643aea3ab448b5afe38e9"
format_stamp: "Formatted at 2026-08-12 08:03:12 on dist-test-slave-zr1t"
I20260812 08:03:12.062676 26633 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:03:12.064790 26656 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:12.065841 26633 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:03:12.065960 26633 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "cf864e9e0db643aea3ab448b5afe38e9"
format_stamp: "Formatted at 2026-08-12 08:03:12 on dist-test-slave-zr1t"
I20260812 08:03:12.066063 26633 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:12.079538 26633 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:12.080173 26633 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:12.080374 26633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:12.080948 26633 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:12.082001 26633 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:12.082048 26633 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:12.082086 26633 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:12.082144 26633 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:12.092031 26633 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.65:35227
I20260812 08:03:12.092149 26837 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.65:35227 every 8 connection(s)
I20260812 08:03:12.092989 26633 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:03:12.097747 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 26633
I20260812 08:03:12.097855 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:03:12.100126 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.66:0
--local_ip_for_outbound_sockets=127.18.42.66
--webserver_interface=127.18.42.66
--webserver_port=0
--tserver_master_addrs=127.18.42.126:34973
--builtin_ntp_servers=127.18.42.84:45543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false with env {}
I20260812 08:03:12.102082 26839 heartbeater.cc:344] Connected to a master server at 127.18.42.126:34973
I20260812 08:03:12.102329 26839 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:12.102823 26839 heartbeater.cc:507] Master 127.18.42.126:34973 requested a full tablet report, sending...
I20260812 08:03:12.104386 26546 ts_manager.cc:194] Registered new tserver with Master: cf864e9e0db643aea3ab448b5afe38e9 (127.18.42.65:35227)
I20260812 08:03:12.106057 26546 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.65:58327
W20260812 08:03:12.206250 26843 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 08:03:12.206470 26843 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:12.206492 26843 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:12.206530 26843 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:12.209759 26843 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:12.209854 26843 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.66
I20260812 08:03:12.213454 26843 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:45543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.18.42.66
--webserver_port=0
--tserver_master_addrs=127.18.42.126:34973
--never_fsync=true
--heap_profile_path=/tmp/kudu.26843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:12.214452 26843 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:12.215612 26843 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:12.221947 26858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:12.221930 26851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:12.221930 26852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:12.222048 26843 server_base.cc:1061] running on GCE node
I20260812 08:03:12.222477 26843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:12.223028 26843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:12.224179 26843 hybrid_clock.cc:648] HybridClock initialized: now 1786521792224168 us; error 27 us; skew 500 ppm
I20260812 08:03:12.225907 26843 webserver.cc:533] Webserver started at http://127.18.42.66:45989/ using document root <none> and password file <none>
I20260812 08:03:12.226473 26843 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:12.226529 26843 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:12.226727 26843 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:12.228250 26843 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "0e3d82e90a4f4dd99ac90434fcd766dc"
format_stamp: "Formatted at 2026-08-12 08:03:12 on dist-test-slave-zr1t"
I20260812 08:03:12.228736 26843 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "0e3d82e90a4f4dd99ac90434fcd766dc"
format_stamp: "Formatted at 2026-08-12 08:03:12 on dist-test-slave-zr1t"
I20260812 08:03:12.232059 26843 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:03:12.234176 26865 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:12.235159 26843 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:12.235277 26843 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "0e3d82e90a4f4dd99ac90434fcd766dc"
format_stamp: "Formatted at 2026-08-12 08:03:12 on dist-test-slave-zr1t"
I20260812 08:03:12.235412 26843 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:12.248891 26843 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:12.249439 26843 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:12.249665 26843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:12.250159 26843 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:12.251072 26843 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:12.251118 26843 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:12.251155 26843 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:12.251171 26843 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:12.261379 26843 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.66:33959
I20260812 08:03:12.261408 27064 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.66:33959 every 8 connection(s)
I20260812 08:03:12.262363 26843 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:03:12.268921 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 26843
I20260812 08:03:12.269054 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:03:12.271510 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.67:0
--local_ip_for_outbound_sockets=127.18.42.67
--webserver_interface=127.18.42.67
--webserver_port=0
--tserver_master_addrs=127.18.42.126:34973
--builtin_ntp_servers=127.18.42.84:45543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false with env {}
I20260812 08:03:12.271883 27068 heartbeater.cc:344] Connected to a master server at 127.18.42.126:34973
I20260812 08:03:12.272130 27068 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:12.272657 27068 heartbeater.cc:507] Master 127.18.42.126:34973 requested a full tablet report, sending...
I20260812 08:03:12.273797 26546 ts_manager.cc:194] Registered new tserver with Master: 0e3d82e90a4f4dd99ac90434fcd766dc (127.18.42.66:33959)
I20260812 08:03:12.274407 26546 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.66:58951
W20260812 08:03:12.377511 27073 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 08:03:12.377851 27073 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:12.377919 27073 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:12.377997 27073 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:12.381330 27073 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:12.381505 27073 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.67
I20260812 08:03:12.385620 27073 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:45543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.18.42.67
--webserver_port=0
--tserver_master_addrs=127.18.42.126:34973
--never_fsync=true
--heap_profile_path=/tmp/kudu.27073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:12.386751 27073 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:12.388049 27073 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:12.394654 27086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:12.394694 27081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:12.394659 27082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:12.395015 27073 server_base.cc:1061] running on GCE node
I20260812 08:03:12.395387 27073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:12.395951 27073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:12.397125 27073 hybrid_clock.cc:648] HybridClock initialized: now 1786521792397109 us; error 36 us; skew 500 ppm
I20260812 08:03:12.398859 27073 webserver.cc:533] Webserver started at http://127.18.42.67:41163/ using document root <none> and password file <none>
I20260812 08:03:12.399459 27073 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:12.399551 27073 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:12.399775 27073 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:12.401281 27073 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "1804a95025a148479dc9eac17a48a0bb"
format_stamp: "Formatted at 2026-08-12 08:03:12 on dist-test-slave-zr1t"
I20260812 08:03:12.401794 27073 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "1804a95025a148479dc9eac17a48a0bb"
format_stamp: "Formatted at 2026-08-12 08:03:12 on dist-test-slave-zr1t"
I20260812 08:03:12.404943 27073 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.000s	sys 0.003s
I20260812 08:03:12.407122 27099 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:12.408135 27073 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:03:12.408278 27073 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "1804a95025a148479dc9eac17a48a0bb"
format_stamp: "Formatted at 2026-08-12 08:03:12 on dist-test-slave-zr1t"
I20260812 08:03:12.408406 27073 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:12.428017 27073 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:12.428735 27073 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:12.428936 27073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:12.429603 27073 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:12.430640 27073 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:12.430716 27073 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:12.430788 27073 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:12.430840 27073 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:12.440984 27073 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.67:36993
I20260812 08:03:12.441013 27284 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.67:36993 every 8 connection(s)
I20260812 08:03:12.442015 27073 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:03:12.448455 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 27073
I20260812 08:03:12.448578 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:03:12.450796 27286 heartbeater.cc:344] Connected to a master server at 127.18.42.126:34973
I20260812 08:03:12.451041 27286 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:12.451493 27286 heartbeater.cc:507] Master 127.18.42.126:34973 requested a full tablet report, sending...
I20260812 08:03:12.452452 26544 ts_manager.cc:194] Registered new tserver with Master: 1804a95025a148479dc9eac17a48a0bb (127.18.42.67:36993)
I20260812 08:03:12.452984 26544 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.67:53803
I20260812 08:03:12.463861 18601 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:03:12.472841 26544 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56018:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
W20260812 08:03:12.475178 26544 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 08:03:12.491804 26962 tablet_service.cc:1511] Processing CreateTablet for tablet 1f27d477426f48ae90b695d9ccdb0b16 (DEFAULT_TABLE table=TestTable [id=ddd3d72d85e546eba1c3b395727b93ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:12.491983 26733 tablet_service.cc:1511] Processing CreateTablet for tablet 1f27d477426f48ae90b695d9ccdb0b16 (DEFAULT_TABLE table=TestTable [id=ddd3d72d85e546eba1c3b395727b93ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:12.492784 26962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f27d477426f48ae90b695d9ccdb0b16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:12.492826 26733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f27d477426f48ae90b695d9ccdb0b16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:12.496861 27170 tablet_service.cc:1511] Processing CreateTablet for tablet 1f27d477426f48ae90b695d9ccdb0b16 (DEFAULT_TABLE table=TestTable [id=ddd3d72d85e546eba1c3b395727b93ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:12.497766 27315 tablet_bootstrap.cc:492] T 1f27d477426f48ae90b695d9ccdb0b16 P cf864e9e0db643aea3ab448b5afe38e9: Bootstrap starting.
I20260812 08:03:12.497846 27170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f27d477426f48ae90b695d9ccdb0b16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:12.498940 27316 tablet_bootstrap.cc:492] T 1f27d477426f48ae90b695d9ccdb0b16 P 0e3d82e90a4f4dd99ac90434fcd766dc: Bootstrap starting.
I20260812 08:03:12.500150 27315 tablet_bootstrap.cc:654] T 1f27d477426f48ae90b695d9ccdb0b16 P cf864e9e0db643aea3ab448b5afe38e9: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:12.500872 27315 log.cc:826] T 1f27d477426f48ae90b695d9ccdb0b16 P cf864e9e0db643aea3ab448b5afe38e9: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:12.501052 27316 tablet_bootstrap.cc:654] T 1f27d477426f48ae90b695d9ccdb0b16 P 0e3d82e90a4f4dd99ac90434fcd766dc: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:12.501899 27316 log.cc:826] T 1f27d477426f48ae90b695d9ccdb0b16 P 0e3d82e90a4f4dd99ac90434fcd766dc: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:12.502627 27315 tablet_bootstrap.cc:492] T 1f27d477426f48ae90b695d9ccdb0b16 P cf864e9e0db643aea3ab448b5afe38e9: No bootstrap required, opened a new log
I20260812 08:03:12.502830 27315 ts_tablet_manager.cc:1403] T 1f27d477426f48ae90b695d9ccdb0b16 P cf864e9e0db643aea3ab448b5afe38e9: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:03:12.503515 27320 tablet_bootstrap.cc:492] T 1f27d477426f48ae90b695d9ccdb0b16 P 1804a95025a148479dc9eac17a48a0bb: Bootstrap starting.
I20260812 08:03:12.503675 27316 tablet_bootstrap.cc:492] T 1f27d477426f48ae90b695d9ccdb0b16 P 0e3d82e90a4f4dd99ac90434fcd766dc: No bootstrap required, opened a new log
I20260812 08:03:12.503868 27316 ts_tablet_manager.cc:1403] T 1f27d477426f48ae90b695d9ccdb0b16 P 0e3d82e90a4f4dd99ac90434fcd766dc: Time spent bootstrapping tablet: real 0.005s	user 0.000s	sys 0.004s
I20260812 08:03:12.505877 27320 tablet_bootstrap.cc:654] T 1f27d477426f48ae90b695d9ccdb0b16 P 1804a95025a148479dc9eac17a48a0bb: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:12.506019 27315 raft_consensus.cc:359] T 1f27d477426f48ae90b695d9ccdb0b16 P cf864e9e0db643aea3ab448b5afe38e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3d82e90a4f4dd99ac90434fcd766dc" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 33959 } } peers { permanent_uuid: "cf864e9e0db643aea3ab448b5afe38e9" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 35227 } } peers { permanent_uuid: "1804a95025a148479dc9eac17a48a0bb" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 36993 } }
I20260812 08:03:12.506250 27315 raft_consensus.cc:740] T 1f27d477426f48ae90b695d9ccdb0b16 P cf864e9e0db643aea3ab448b5afe38e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf864e9e0db643aea3ab448b5afe38e9, State: Initialized, Role: FOLLOWER
I20260812 08:03:12.506568 27315 consensus_queue.cc:260] T 1f27d477426f48ae90b695d9ccdb0b16 P cf864e9e0db643aea3ab448b5afe38e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3d82e90a4f4dd99ac90434fcd766dc" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 33959 } } peers { permanent_uuid: "cf864e9e0db643aea3ab448b5afe38e9" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 35227 } } peers { permanent_uuid: "1804a95025a148479dc9eac17a48a0bb" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 36993 } }
I20260812 08:03:12.506664 27320 log.cc:826] T 1f27d477426f48ae90b695d9ccdb0b16 P 1804a95025a148479dc9eac17a48a0bb: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:12.506588 27316 raft_consensus.cc:359] T 1f27d477426f48ae90b695d9ccdb0b16 P 0e3d82e90a4f4dd99ac90434fcd766dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3d82e90a4f4dd99ac90434fcd766dc" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 33959 } } peers { permanent_uuid: "cf864e9e0db643aea3ab448b5afe38e9" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 35227 } } peers { permanent_uuid: "1804a95025a148479dc9eac17a48a0bb" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 36993 } }
I20260812 08:03:12.506800 27316 raft_consensus.cc:740] T 1f27d477426f48ae90b695d9ccdb0b16 P 0e3d82e90a4f4dd99ac90434fcd766dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e3d82e90a4f4dd99ac90434fcd766dc, State: Initialized, Role: FOLLOWER
I20260812 08:03:12.507088 27316 consensus_queue.cc:260] T 1f27d477426f48ae90b695d9ccdb0b16 P 0e3d82e90a4f4dd99ac90434fcd766dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3d82e90a4f4dd99ac90434fcd766dc" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 33959 } } peers { permanent_uuid: "cf864e9e0db643aea3ab448b5afe38e9" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 35227 } } peers { permanent_uuid: "1804a95025a148479dc9eac17a48a0bb" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 36993 } }
I20260812 08:03:12.507472 27315 ts_tablet_manager.cc:1434] T 1f27d477426f48ae90b695d9ccdb0b16 P cf864e9e0db643aea3ab448b5afe38e9: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 08:03:12.507762 27068 heartbeater.cc:499] Master 127.18.42.126:34973 was elected leader, sending a full tablet report...
I20260812 08:03:12.507949 27316 ts_tablet_manager.cc:1434] T 1f27d477426f48ae90b695d9ccdb0b16 P 0e3d82e90a4f4dd99ac90434fcd766dc: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.001s
I20260812 08:03:12.508503 27320 tablet_bootstrap.cc:492] T 1f27d477426f48ae90b695d9ccdb0b16 P 1804a95025a148479dc9eac17a48a0bb: No bootstrap required, opened a new log
I20260812 08:03:12.508687 27320 ts_tablet_manager.cc:1403] T 1f27d477426f48ae90b695d9ccdb0b16 P 1804a95025a148479dc9eac17a48a0bb: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:03:12.508973 26839 heartbeater.cc:499] Master 127.18.42.126:34973 was elected leader, sending a full tablet report...
I20260812 08:03:12.511554 27320 raft_consensus.cc:359] T 1f27d477426f48ae90b695d9ccdb0b16 P 1804a95025a148479dc9eac17a48a0bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3d82e90a4f4dd99ac90434fcd766dc" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 33959 } } peers { permanent_uuid: "cf864e9e0db643aea3ab448b5afe38e9" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 35227 } } peers { permanent_uuid: "1804a95025a148479dc9eac17a48a0bb" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 36993 } }
I20260812 08:03:12.511761 27320 raft_consensus.cc:740] T 1f27d477426f48ae90b695d9ccdb0b16 P 1804a95025a148479dc9eac17a48a0bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1804a95025a148479dc9eac17a48a0bb, State: Initialized, Role: FOLLOWER
I20260812 08:03:12.512056 27320 consensus_queue.cc:260] T 1f27d477426f48ae90b695d9ccdb0b16 P 1804a95025a148479dc9eac17a48a0bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3d82e90a4f4dd99ac90434fcd766dc" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 33959 } } peers { permanent_uuid: "cf864e9e0db643aea3ab448b5afe38e9" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 35227 } } peers { permanent_uuid: "1804a95025a148479dc9eac17a48a0bb" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 36993 } }
I20260812 08:03:12.512998 27320 ts_tablet_manager.cc:1434] T 1f27d477426f48ae90b695d9ccdb0b16 P 1804a95025a148479dc9eac17a48a0bb: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
W20260812 08:03:12.513281 27070 tablet.cc:2406] T 1f27d477426f48ae90b695d9ccdb0b16 P 0e3d82e90a4f4dd99ac90434fcd766dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:03:12.513335 27286 heartbeater.cc:499] Master 127.18.42.126:34973 was elected leader, sending a full tablet report...
I20260812 08:03:12.516957 18601 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:03:12.518558 18601 ts_itest-base.cc:246] Waiting for 1 tablets on tserver cf864e9e0db643aea3ab448b5afe38e9 to finish bootstrapping
I20260812 08:03:12.524518 18601 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 0e3d82e90a4f4dd99ac90434fcd766dc to finish bootstrapping
I20260812 08:03:12.529754 18601 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 1804a95025a148479dc9eac17a48a0bb to finish bootstrapping
I20260812 08:03:12.538866 27199 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1f27d477426f48ae90b695d9ccdb0b16"
dest_uuid: "1804a95025a148479dc9eac17a48a0bb"
 from {username='slave'} at 127.0.0.1:56308
I20260812 08:03:12.539182 27199 raft_consensus.cc:493] T 1f27d477426f48ae90b695d9ccdb0b16 P 1804a95025a148479dc9eac17a48a0bb [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 08:03:12.539364 27199 raft_consensus.cc:3060] T 1f27d477426f48ae90b695d9ccdb0b16 P 1804a95025a148479dc9eac17a48a0bb [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:12.540432 27199 raft_consensus.cc:515] T 1f27d477426f48ae90b695d9ccdb0b16 P 1804a95025a148479dc9eac17a48a0bb [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3d82e90a4f4dd99ac90434fcd766dc" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 33959 } } peers { permanent_uuid: "cf864e9e0db643aea3ab448b5afe38e9" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 35227 } } peers { permanent_uuid: "1804a95025a148479dc9eac17a48a0bb" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 36993 } }
I20260812 08:03:12.541622 27199 leader_election.cc:290] T 1f27d477426f48ae90b695d9ccdb0b16 P 1804a95025a148479dc9eac17a48a0bb [CANDIDATE]: Term 1 election: Requested vote from peers 0e3d82e90a4f4dd99ac90434fcd766dc (127.18.42.66:33959), cf864e9e0db643aea3ab448b5afe38e9 (127.18.42.65:35227)
I20260812 08:03:12.546252 26993 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f27d477426f48ae90b695d9ccdb0b16" candidate_uuid: "1804a95025a148479dc9eac17a48a0bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0e3d82e90a4f4dd99ac90434fcd766dc"
I20260812 08:03:12.546255 26773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f27d477426f48ae90b695d9ccdb0b16" candidate_uuid: "1804a95025a148479dc9eac17a48a0bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "cf864e9e0db643aea3ab448b5afe38e9"
I20260812 08:03:12.546522 26773 raft_consensus.cc:3060] T 1f27d477426f48ae90b695d9ccdb0b16 P cf864e9e0db643aea3ab448b5afe38e9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:12.546530 26993 raft_consensus.cc:3060] T 1f27d477426f48ae90b695d9ccdb0b16 P 0e3d82e90a4f4dd99ac90434fcd766dc [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:12.547962 26993 raft_consensus.cc:2468] T 1f27d477426f48ae90b695d9ccdb0b16 P 0e3d82e90a4f4dd99ac90434fcd766dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1804a95025a148479dc9eac17a48a0bb in term 1.
I20260812 08:03:12.547960 26773 raft_consensus.cc:2468] T 1f27d477426f48ae90b695d9ccdb0b16 P cf864e9e0db643aea3ab448b5afe38e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1804a95025a148479dc9eac17a48a0bb in term 1.
I20260812 08:03:12.548485 27103 leader_election.cc:304] T 1f27d477426f48ae90b695d9ccdb0b16 P 1804a95025a148479dc9eac17a48a0bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1804a95025a148479dc9eac17a48a0bb, cf864e9e0db643aea3ab448b5afe38e9; no voters: 
I20260812 08:03:12.548801 27329 raft_consensus.cc:2804] T 1f27d477426f48ae90b695d9ccdb0b16 P 1804a95025a148479dc9eac17a48a0bb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:12.549058 27329 raft_consensus.cc:697] T 1f27d477426f48ae90b695d9ccdb0b16 P 1804a95025a148479dc9eac17a48a0bb [term 1 LEADER]: Becoming Leader. State: Replica: 1804a95025a148479dc9eac17a48a0bb, State: Running, Role: LEADER
I20260812 08:03:12.549418 27329 consensus_queue.cc:237] T 1f27d477426f48ae90b695d9ccdb0b16 P 1804a95025a148479dc9eac17a48a0bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3d82e90a4f4dd99ac90434fcd766dc" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 33959 } } peers { permanent_uuid: "cf864e9e0db643aea3ab448b5afe38e9" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 35227 } } peers { permanent_uuid: "1804a95025a148479dc9eac17a48a0bb" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 36993 } }
I20260812 08:03:12.553361 26543 catalog_manager.cc:5719] T 1f27d477426f48ae90b695d9ccdb0b16 P 1804a95025a148479dc9eac17a48a0bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 1804a95025a148479dc9eac17a48a0bb (127.18.42.67). New cstate: current_term: 1 leader_uuid: "1804a95025a148479dc9eac17a48a0bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3d82e90a4f4dd99ac90434fcd766dc" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 33959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf864e9e0db643aea3ab448b5afe38e9" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 35227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1804a95025a148479dc9eac17a48a0bb" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 36993 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:12.554565 18601 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 0.0 1.1
W20260812 08:03:12.594094 26840 tablet.cc:2406] T 1f27d477426f48ae90b695d9ccdb0b16 P cf864e9e0db643aea3ab448b5afe38e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:03:12.657709 18601 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 0.0 1.1
W20260812 08:03:12.692993 27287 tablet.cc:2406] T 1f27d477426f48ae90b695d9ccdb0b16 P 1804a95025a148479dc9eac17a48a0bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:03:12.860527 18601 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 0.0 1.1
I20260812 08:03:13.070576 27345 consensus_queue.cc:1048] T 1f27d477426f48ae90b695d9ccdb0b16 P 1804a95025a148479dc9eac17a48a0bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf864e9e0db643aea3ab448b5afe38e9" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 35227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:13.076515 27345 consensus_queue.cc:1048] T 1f27d477426f48ae90b695d9ccdb0b16 P 1804a95025a148479dc9eac17a48a0bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e3d82e90a4f4dd99ac90434fcd766dc" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 33959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:13.163790 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 26843
I20260812 08:03:13.185179 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 27073
I20260812 08:03:13.198880 18601 raft_consensus-itest.cc:648] ================================== Cluster setup complete.
I20260812 08:03:13.203828 26773 raft_consensus.cc:3060] T 1f27d477426f48ae90b695d9ccdb0b16 P cf864e9e0db643aea3ab448b5afe38e9 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:03:13.208030 26773 raft_consensus.cc:3060] T 1f27d477426f48ae90b695d9ccdb0b16 P cf864e9e0db643aea3ab448b5afe38e9 [term 2 FOLLOWER]: Advancing to term 3
I20260812 08:03:13.209066 26773 pending_rounds.cc:85] T 1f27d477426f48ae90b695d9ccdb0b16 P cf864e9e0db643aea3ab448b5afe38e9: Aborting all ops after (but not including) 3
I20260812 08:03:13.209161 26773 pending_rounds.cc:107] T 1f27d477426f48ae90b695d9ccdb0b16 P cf864e9e0db643aea3ab448b5afe38e9: Aborting uncommitted WRITE_OP operation due to leader change: 2.4
I20260812 08:03:13.211282 18601 raft_consensus-itest.cc:251] Waiting for row count 2...
I20260812 08:03:13.218490 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 26633
I20260812 08:03:13.239853 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 26488
2026-08-12T08:03:13Z chronyd exiting
[       OK ] RaftConsensusITest.TestReplaceOperationStuckInPrepareQueue (1585 ms)
[ RUN      ] RaftConsensusITest.TestMasterNotifiedOnConfigChange
I20260812 08:03:13.274225 18601 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:03:13.274273 18601 ts_itest-base.cc:116] --------------
I20260812 08:03:13.274286 18601 ts_itest-base.cc:117] 3 tablet servers
I20260812 08:03:13.274296 18601 ts_itest-base.cc:118] 2 replicas per TS
I20260812 08:03:13.274307 18601 ts_itest-base.cc:119] --------------
2026-08-12T08:03:13Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:03:13Z Disabled control of system clock
I20260812 08:03:13.283512 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.42.126:40195
--webserver_interface=127.18.42.126
--webserver_port=0
--builtin_ntp_servers=127.18.42.84:36283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.42.126:40195
--master_add_server_when_underreplicated=false
--allow_unsafe_replication_factor=true
--catalog_manager_evict_excess_replicas=false with env {}
W20260812 08:03:13.385562 27378 flags.cc:432] Enabled unsafe flag: --allow_unsafe_replication_factor=true
W20260812 08:03:13.385897 27378 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:13.385986 27378 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:13.386063 27378 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:13.389382 27378 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:03:13.389483 27378 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:13.389519 27378 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:03:13.389585 27378 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:03:13.393561 27378 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:36283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
--allow_unsafe_replication_factor=true
--catalog_manager_evict_excess_replicas=false
--master_add_server_when_underreplicated=false
--ipki_ca_key_size=768
--master_addresses=127.18.42.126:40195
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.42.126:40195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.18.42.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.27378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:13.394731 27378 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:13.395879 27378 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:13.401324 27395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:13.401333 27391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:13.401497 27378 server_base.cc:1061] running on GCE node
W20260812 08:03:13.401348 27398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:13.402072 27378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:13.402880 27378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:13.404044 27378 hybrid_clock.cc:648] HybridClock initialized: now 1786521793404018 us; error 39 us; skew 500 ppm
I20260812 08:03:13.405715 27378 webserver.cc:533] Webserver started at http://127.18.42.126:36603/ using document root <none> and password file <none>
I20260812 08:03:13.406265 27378 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:13.406319 27378 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:13.406487 27378 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:13.407912 27378 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "157ba5254a2847d390e0826c49ae1831"
format_stamp: "Formatted at 2026-08-12 08:03:13 on dist-test-slave-zr1t"
I20260812 08:03:13.408341 27378 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "157ba5254a2847d390e0826c49ae1831"
format_stamp: "Formatted at 2026-08-12 08:03:13 on dist-test-slave-zr1t"
I20260812 08:03:13.411388 27378 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:03:13.413272 27406 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:13.414233 27378 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:13.414351 27378 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
uuid: "157ba5254a2847d390e0826c49ae1831"
format_stamp: "Formatted at 2026-08-12 08:03:13 on dist-test-slave-zr1t"
I20260812 08:03:13.414446 27378 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:13.427136 27378 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:13.427733 27378 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:13.427932 27378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:13.435483 27499 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.126:40195 every 8 connection(s)
I20260812 08:03:13.435492 27378 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.126:40195
I20260812 08:03:13.436509 27378 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:03:13.438405 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 27378
I20260812 08:03:13.438547 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:03:13.439474 27501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:13.444871 27501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 157ba5254a2847d390e0826c49ae1831: Bootstrap starting.
I20260812 08:03:13.446940 27501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 157ba5254a2847d390e0826c49ae1831: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:13.447556 27501 log.cc:826] T 00000000000000000000000000000000 P 157ba5254a2847d390e0826c49ae1831: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:13.449066 27501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 157ba5254a2847d390e0826c49ae1831: No bootstrap required, opened a new log
I20260812 08:03:13.451751 27501 raft_consensus.cc:359] T 00000000000000000000000000000000 P 157ba5254a2847d390e0826c49ae1831 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "157ba5254a2847d390e0826c49ae1831" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 40195 } }
I20260812 08:03:13.451954 27501 raft_consensus.cc:385] T 00000000000000000000000000000000 P 157ba5254a2847d390e0826c49ae1831 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:13.452014 27501 raft_consensus.cc:740] T 00000000000000000000000000000000 P 157ba5254a2847d390e0826c49ae1831 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 157ba5254a2847d390e0826c49ae1831, State: Initialized, Role: FOLLOWER
I20260812 08:03:13.452454 27501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 157ba5254a2847d390e0826c49ae1831 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "157ba5254a2847d390e0826c49ae1831" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 40195 } }
I20260812 08:03:13.452612 27501 raft_consensus.cc:399] T 00000000000000000000000000000000 P 157ba5254a2847d390e0826c49ae1831 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:03:13.452663 27501 raft_consensus.cc:493] T 00000000000000000000000000000000 P 157ba5254a2847d390e0826c49ae1831 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:03:13.452740 27501 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 157ba5254a2847d390e0826c49ae1831 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:13.453461 27501 raft_consensus.cc:515] T 00000000000000000000000000000000 P 157ba5254a2847d390e0826c49ae1831 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "157ba5254a2847d390e0826c49ae1831" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 40195 } }
I20260812 08:03:13.453778 27501 leader_election.cc:304] T 00000000000000000000000000000000 P 157ba5254a2847d390e0826c49ae1831 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 157ba5254a2847d390e0826c49ae1831; no voters: 
I20260812 08:03:13.454010 27501 leader_election.cc:290] T 00000000000000000000000000000000 P 157ba5254a2847d390e0826c49ae1831 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:03:13.454152 27510 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 157ba5254a2847d390e0826c49ae1831 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:13.454362 27510 raft_consensus.cc:697] T 00000000000000000000000000000000 P 157ba5254a2847d390e0826c49ae1831 [term 1 LEADER]: Becoming Leader. State: Replica: 157ba5254a2847d390e0826c49ae1831, State: Running, Role: LEADER
I20260812 08:03:13.454694 27510 consensus_queue.cc:237] T 00000000000000000000000000000000 P 157ba5254a2847d390e0826c49ae1831 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "157ba5254a2847d390e0826c49ae1831" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 40195 } }
I20260812 08:03:13.455163 27501 sys_catalog.cc:565] T 00000000000000000000000000000000 P 157ba5254a2847d390e0826c49ae1831 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:13.456212 27511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 157ba5254a2847d390e0826c49ae1831 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "157ba5254a2847d390e0826c49ae1831" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "157ba5254a2847d390e0826c49ae1831" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 40195 } } }
I20260812 08:03:13.456337 27511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 157ba5254a2847d390e0826c49ae1831 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:13.456897 27521 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:03:13.456714 27512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 157ba5254a2847d390e0826c49ae1831 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 157ba5254a2847d390e0826c49ae1831. Latest consensus state: current_term: 1 leader_uuid: "157ba5254a2847d390e0826c49ae1831" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "157ba5254a2847d390e0826c49ae1831" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 40195 } } }
I20260812 08:03:13.457332 27512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 157ba5254a2847d390e0826c49ae1831 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:13.459667 27521 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:03:13.464377 27521 catalog_manager.cc:1383] Generated new cluster ID: 25fccb16f6cc4bb3aea574669a4489f4
I20260812 08:03:13.464439 27521 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:03:13.481715 27521 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:03:13.482547 27521 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:03:13.492506 27521 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 157ba5254a2847d390e0826c49ae1831: Generated new TSK 0
I20260812 08:03:13.493070 27521 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:03:13.497320 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.65:0
--local_ip_for_outbound_sockets=127.18.42.65
--webserver_interface=127.18.42.65
--webserver_port=0
--tserver_master_addrs=127.18.42.126:40195
--builtin_ntp_servers=127.18.42.84:36283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=50 with env {}
W20260812 08:03:13.604295 27539 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:13.604509 27539 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:13.604552 27539 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:13.608062 27539 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:13.608172 27539 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.65
I20260812 08:03:13.612078 27539 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:36283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=50
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.18.42.65
--webserver_port=0
--tserver_master_addrs=127.18.42.126:40195
--never_fsync=true
--heap_profile_path=/tmp/kudu.27539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:13.613274 27539 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:13.614476 27539 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:13.620843 27547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:13.620846 27548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:13.620846 27551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:13.621373 27539 server_base.cc:1061] running on GCE node
I20260812 08:03:13.621827 27539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:13.622490 27539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:13.623682 27539 hybrid_clock.cc:648] HybridClock initialized: now 1786521793623654 us; error 45 us; skew 500 ppm
I20260812 08:03:13.625871 27539 webserver.cc:533] Webserver started at http://127.18.42.65:44545/ using document root <none> and password file <none>
I20260812 08:03:13.626433 27539 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:13.626492 27539 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:13.626659 27539 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:13.628262 27539 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "15851e32b4dc453eb481c88ac7edca78"
format_stamp: "Formatted at 2026-08-12 08:03:13 on dist-test-slave-zr1t"
I20260812 08:03:13.628717 27539 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "15851e32b4dc453eb481c88ac7edca78"
format_stamp: "Formatted at 2026-08-12 08:03:13 on dist-test-slave-zr1t"
I20260812 08:03:13.632154 27539 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.003s
I20260812 08:03:13.634373 27563 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:13.635412 27539 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260812 08:03:13.635548 27539 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "15851e32b4dc453eb481c88ac7edca78"
format_stamp: "Formatted at 2026-08-12 08:03:13 on dist-test-slave-zr1t"
I20260812 08:03:13.635664 27539 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:13.658857 27539 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:13.659586 27539 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:13.659807 27539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:13.660430 27539 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:13.661504 27539 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:13.661600 27539 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:13.661684 27539 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:13.661743 27539 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:13.672175 27539 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.65:39381
I20260812 08:03:13.672237 27749 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.65:39381 every 8 connection(s)
I20260812 08:03:13.673219 27539 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:03:13.674433 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 27539
I20260812 08:03:13.674553 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:03:13.676829 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.66:0
--local_ip_for_outbound_sockets=127.18.42.66
--webserver_interface=127.18.42.66
--webserver_port=0
--tserver_master_addrs=127.18.42.126:40195
--builtin_ntp_servers=127.18.42.84:36283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=50 with env {}
I20260812 08:03:13.683990 27750 heartbeater.cc:344] Connected to a master server at 127.18.42.126:40195
I20260812 08:03:13.684285 27750 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:13.684989 27750 heartbeater.cc:507] Master 127.18.42.126:40195 requested a full tablet report, sending...
I20260812 08:03:13.686842 27444 ts_manager.cc:194] Registered new tserver with Master: 15851e32b4dc453eb481c88ac7edca78 (127.18.42.65:39381)
I20260812 08:03:13.688565 27444 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.65:36221
W20260812 08:03:13.785651 27754 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:13.785856 27754 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:13.785898 27754 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:13.789094 27754 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:13.789188 27754 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.66
I20260812 08:03:13.792867 27754 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:36283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=50
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.18.42.66
--webserver_port=0
--tserver_master_addrs=127.18.42.126:40195
--never_fsync=true
--heap_profile_path=/tmp/kudu.27754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:13.793921 27754 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:13.794950 27754 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:13.801534 27763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:13.801506 27766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:13.801614 27764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:13.801998 27754 server_base.cc:1061] running on GCE node
I20260812 08:03:13.802336 27754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:13.802860 27754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:13.804034 27754 hybrid_clock.cc:648] HybridClock initialized: now 1786521793804013 us; error 30 us; skew 500 ppm
I20260812 08:03:13.805866 27754 webserver.cc:533] Webserver started at http://127.18.42.66:37731/ using document root <none> and password file <none>
I20260812 08:03:13.806427 27754 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:13.806485 27754 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:13.806680 27754 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:13.808127 27754 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "46a7d85f674d461882efa722b8b90976"
format_stamp: "Formatted at 2026-08-12 08:03:13 on dist-test-slave-zr1t"
I20260812 08:03:13.808600 27754 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "46a7d85f674d461882efa722b8b90976"
format_stamp: "Formatted at 2026-08-12 08:03:13 on dist-test-slave-zr1t"
I20260812 08:03:13.811779 27754 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:03:13.814041 27775 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:13.815078 27754 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:03:13.815238 27754 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "46a7d85f674d461882efa722b8b90976"
format_stamp: "Formatted at 2026-08-12 08:03:13 on dist-test-slave-zr1t"
I20260812 08:03:13.815351 27754 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:13.826823 27754 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:13.827497 27754 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:13.827703 27754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:13.828306 27754 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:13.829378 27754 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:13.829455 27754 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:13.829586 27754 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:13.829636 27754 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:13.840348 27754 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.66:45397
I20260812 08:03:13.840394 27959 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.66:45397 every 8 connection(s)
I20260812 08:03:13.841336 27754 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:03:13.844205 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 27754
I20260812 08:03:13.844329 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:03:13.846555 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.67:0
--local_ip_for_outbound_sockets=127.18.42.67
--webserver_interface=127.18.42.67
--webserver_port=0
--tserver_master_addrs=127.18.42.126:40195
--builtin_ntp_servers=127.18.42.84:36283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=50 with env {}
I20260812 08:03:13.851217 27960 heartbeater.cc:344] Connected to a master server at 127.18.42.126:40195
I20260812 08:03:13.851490 27960 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:13.852029 27960 heartbeater.cc:507] Master 127.18.42.126:40195 requested a full tablet report, sending...
I20260812 08:03:13.853206 27444 ts_manager.cc:194] Registered new tserver with Master: 46a7d85f674d461882efa722b8b90976 (127.18.42.66:45397)
I20260812 08:03:13.854056 27444 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.66:39663
W20260812 08:03:13.953182 27968 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:13.953369 27968 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:13.953410 27968 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:13.956954 27968 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:13.957079 27968 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.67
I20260812 08:03:13.960878 27968 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:36283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=50
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.18.42.67
--webserver_port=0
--tserver_master_addrs=127.18.42.126:40195
--never_fsync=true
--heap_profile_path=/tmp/kudu.27968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:13.961968 27968 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:13.963086 27968 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:13.969342 27978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:13.969424 27976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:13.969357 27975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:13.969981 27968 server_base.cc:1061] running on GCE node
I20260812 08:03:13.970314 27968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:13.970824 27968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:13.972005 27968 hybrid_clock.cc:648] HybridClock initialized: now 1786521793972016 us; error 54 us; skew 500 ppm
I20260812 08:03:13.973706 27968 webserver.cc:533] Webserver started at http://127.18.42.67:44425/ using document root <none> and password file <none>
I20260812 08:03:13.974246 27968 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:13.974344 27968 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:13.974576 27968 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:13.976109 27968 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "1ea6088e6e0e475c825bdd9585613759"
format_stamp: "Formatted at 2026-08-12 08:03:13 on dist-test-slave-zr1t"
I20260812 08:03:13.976593 27968 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "1ea6088e6e0e475c825bdd9585613759"
format_stamp: "Formatted at 2026-08-12 08:03:13 on dist-test-slave-zr1t"
I20260812 08:03:13.979766 27968 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.003s
I20260812 08:03:13.982079 27991 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:13.983237 27968 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:13.983363 27968 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "1ea6088e6e0e475c825bdd9585613759"
format_stamp: "Formatted at 2026-08-12 08:03:13 on dist-test-slave-zr1t"
I20260812 08:03:13.983472 27968 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:14.013654 27968 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:14.014284 27968 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:14.014482 27968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:14.015036 27968 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:14.015990 27968 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:14.016058 27968 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:14.016129 27968 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:14.016181 27968 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:14.025926 27968 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.67:37699
I20260812 08:03:14.025974 28183 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.67:37699 every 8 connection(s)
I20260812 08:03:14.026842 27968 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:03:14.033407 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 27968
I20260812 08:03:14.033558 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:03:14.035494 28184 heartbeater.cc:344] Connected to a master server at 127.18.42.126:40195
I20260812 08:03:14.035717 28184 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:14.036257 28184 heartbeater.cc:507] Master 127.18.42.126:40195 requested a full tablet report, sending...
I20260812 08:03:14.037230 27444 ts_manager.cc:194] Registered new tserver with Master: 1ea6088e6e0e475c825bdd9585613759 (127.18.42.67:37699)
I20260812 08:03:14.037746 27444 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.67:36827
I20260812 08:03:14.047614 18601 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:03:14.055778 27444 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42840:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
I20260812 08:03:14.078111 27646 tablet_service.cc:1511] Processing CreateTablet for tablet df9755806eb24d1688f0c28cdce1e074 (DEFAULT_TABLE table=TestTable [id=2c412547babe4429ae1185c9ea3783b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:14.078114 27862 tablet_service.cc:1511] Processing CreateTablet for tablet df9755806eb24d1688f0c28cdce1e074 (DEFAULT_TABLE table=TestTable [id=2c412547babe4429ae1185c9ea3783b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:14.079110 27862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df9755806eb24d1688f0c28cdce1e074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:14.079113 27646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df9755806eb24d1688f0c28cdce1e074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:14.083652 28215 tablet_bootstrap.cc:492] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78: Bootstrap starting.
I20260812 08:03:14.084492 28214 tablet_bootstrap.cc:492] T df9755806eb24d1688f0c28cdce1e074 P 46a7d85f674d461882efa722b8b90976: Bootstrap starting.
I20260812 08:03:14.085767 28215 tablet_bootstrap.cc:654] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:14.086334 28214 tablet_bootstrap.cc:654] T df9755806eb24d1688f0c28cdce1e074 P 46a7d85f674d461882efa722b8b90976: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:14.086526 28215 log.cc:826] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:14.087054 28214 log.cc:826] T df9755806eb24d1688f0c28cdce1e074 P 46a7d85f674d461882efa722b8b90976: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:14.088186 28215 tablet_bootstrap.cc:492] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78: No bootstrap required, opened a new log
I20260812 08:03:14.088377 28215 ts_tablet_manager.cc:1403] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.000s
I20260812 08:03:14.088729 28214 tablet_bootstrap.cc:492] T df9755806eb24d1688f0c28cdce1e074 P 46a7d85f674d461882efa722b8b90976: No bootstrap required, opened a new log
I20260812 08:03:14.088919 28214 ts_tablet_manager.cc:1403] T df9755806eb24d1688f0c28cdce1e074 P 46a7d85f674d461882efa722b8b90976: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.000s
I20260812 08:03:14.091041 28215 raft_consensus.cc:359] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a7d85f674d461882efa722b8b90976" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45397 } } peers { permanent_uuid: "15851e32b4dc453eb481c88ac7edca78" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 39381 } }
I20260812 08:03:14.091203 28215 raft_consensus.cc:385] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:14.091261 28215 raft_consensus.cc:740] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15851e32b4dc453eb481c88ac7edca78, State: Initialized, Role: FOLLOWER
I20260812 08:03:14.091452 28214 raft_consensus.cc:359] T df9755806eb24d1688f0c28cdce1e074 P 46a7d85f674d461882efa722b8b90976 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a7d85f674d461882efa722b8b90976" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45397 } } peers { permanent_uuid: "15851e32b4dc453eb481c88ac7edca78" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 39381 } }
I20260812 08:03:14.091634 28214 raft_consensus.cc:385] T df9755806eb24d1688f0c28cdce1e074 P 46a7d85f674d461882efa722b8b90976 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:14.091711 28214 raft_consensus.cc:740] T df9755806eb24d1688f0c28cdce1e074 P 46a7d85f674d461882efa722b8b90976 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a7d85f674d461882efa722b8b90976, State: Initialized, Role: FOLLOWER
I20260812 08:03:14.091733 28215 consensus_queue.cc:260] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a7d85f674d461882efa722b8b90976" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45397 } } peers { permanent_uuid: "15851e32b4dc453eb481c88ac7edca78" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 39381 } }
I20260812 08:03:14.092097 28214 consensus_queue.cc:260] T df9755806eb24d1688f0c28cdce1e074 P 46a7d85f674d461882efa722b8b90976 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a7d85f674d461882efa722b8b90976" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45397 } } peers { permanent_uuid: "15851e32b4dc453eb481c88ac7edca78" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 39381 } }
I20260812 08:03:14.092613 27750 heartbeater.cc:499] Master 127.18.42.126:40195 was elected leader, sending a full tablet report...
I20260812 08:03:14.092721 28215 ts_tablet_manager.cc:1434] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:03:14.092736 27960 heartbeater.cc:499] Master 127.18.42.126:40195 was elected leader, sending a full tablet report...
I20260812 08:03:14.092922 28214 ts_tablet_manager.cc:1434] T df9755806eb24d1688f0c28cdce1e074 P 46a7d85f674d461882efa722b8b90976: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
W20260812 08:03:14.174396 27751 tablet.cc:2406] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:03:14.342499 27963 tablet.cc:2406] T df9755806eb24d1688f0c28cdce1e074 P 46a7d85f674d461882efa722b8b90976: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:03:14.382473 28218 raft_consensus.cc:493] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:14.382711 28218 raft_consensus.cc:515] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a7d85f674d461882efa722b8b90976" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45397 } } peers { permanent_uuid: "15851e32b4dc453eb481c88ac7edca78" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 39381 } }
I20260812 08:03:14.383796 28218 leader_election.cc:290] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46a7d85f674d461882efa722b8b90976 (127.18.42.66:45397)
I20260812 08:03:14.388145 27895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df9755806eb24d1688f0c28cdce1e074" candidate_uuid: "15851e32b4dc453eb481c88ac7edca78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a7d85f674d461882efa722b8b90976" is_pre_election: true
I20260812 08:03:14.388433 27895 raft_consensus.cc:2468] T df9755806eb24d1688f0c28cdce1e074 P 46a7d85f674d461882efa722b8b90976 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15851e32b4dc453eb481c88ac7edca78 in term 0.
I20260812 08:03:14.388890 27571 leader_election.cc:304] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 15851e32b4dc453eb481c88ac7edca78, 46a7d85f674d461882efa722b8b90976; no voters: 
I20260812 08:03:14.389175 28218 raft_consensus.cc:2804] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:03:14.389246 28218 raft_consensus.cc:493] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:03:14.389277 28218 raft_consensus.cc:3060] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:14.390333 28218 raft_consensus.cc:515] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a7d85f674d461882efa722b8b90976" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45397 } } peers { permanent_uuid: "15851e32b4dc453eb481c88ac7edca78" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 39381 } }
I20260812 08:03:14.390654 28218 leader_election.cc:290] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 [CANDIDATE]: Term 1 election: Requested vote from peers 46a7d85f674d461882efa722b8b90976 (127.18.42.66:45397)
I20260812 08:03:14.391083 27895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df9755806eb24d1688f0c28cdce1e074" candidate_uuid: "15851e32b4dc453eb481c88ac7edca78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a7d85f674d461882efa722b8b90976"
I20260812 08:03:14.391187 27895 raft_consensus.cc:3060] T df9755806eb24d1688f0c28cdce1e074 P 46a7d85f674d461882efa722b8b90976 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:14.392431 27895 raft_consensus.cc:2468] T df9755806eb24d1688f0c28cdce1e074 P 46a7d85f674d461882efa722b8b90976 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15851e32b4dc453eb481c88ac7edca78 in term 1.
I20260812 08:03:14.392737 27571 leader_election.cc:304] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 15851e32b4dc453eb481c88ac7edca78, 46a7d85f674d461882efa722b8b90976; no voters: 
I20260812 08:03:14.392958 28218 raft_consensus.cc:2804] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:14.393170 28218 raft_consensus.cc:697] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 [term 1 LEADER]: Becoming Leader. State: Replica: 15851e32b4dc453eb481c88ac7edca78, State: Running, Role: LEADER
I20260812 08:03:14.393546 28218 consensus_queue.cc:237] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a7d85f674d461882efa722b8b90976" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45397 } } peers { permanent_uuid: "15851e32b4dc453eb481c88ac7edca78" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 39381 } }
I20260812 08:03:14.396641 27444 catalog_manager.cc:5719] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 15851e32b4dc453eb481c88ac7edca78 (127.18.42.65). New cstate: current_term: 1 leader_uuid: "15851e32b4dc453eb481c88ac7edca78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a7d85f674d461882efa722b8b90976" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15851e32b4dc453eb481c88ac7edca78" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 39381 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:14.430423 18601 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:03:14.431849 18601 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 15851e32b4dc453eb481c88ac7edca78 to finish bootstrapping
I20260812 08:03:14.438097 18601 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 46a7d85f674d461882efa722b8b90976 to finish bootstrapping
I20260812 08:03:14.443425 18601 raft_consensus-itest.cc:1792] Finding tablet leader and waiting for things to start...
I20260812 08:03:14.443491 18601 raft_consensus-itest.cc:1812] Waiting for Master to see the current replicas...
I20260812 08:03:14.444624 18601 raft_consensus-itest.cc:1819] Tablet locations:
tablet_locations {
  tablet_id: "df9755806eb24d1688f0c28cdce1e074"
  DEPRECATED_stale: false
  partition {
    partition_key_start: ""
    partition_key_end: ""
  }
  interned_replicas {
    ts_info_idx: 0
    role: FOLLOWER
  }
  interned_replicas {
    ts_info_idx: 1
    role: LEADER
  }
}
ts_infos {
  permanent_uuid: "46a7d85f674d461882efa722b8b90976"
  rpc_addresses {
    host: "127.18.42.66"
    port: 45397
  }
}
ts_infos {
  permanent_uuid: "15851e32b4dc453eb481c88ac7edca78"
  rpc_addresses {
    host: "127.18.42.65"
    port: 39381
  }
}
I20260812 08:03:14.969115 28237 consensus_queue.cc:1048] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a7d85f674d461882efa722b8b90976" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:14.983734 18601 raft_consensus-itest.cc:1828] Adding tserver with uuid 1ea6088e6e0e475c825bdd9585613759
I20260812 08:03:14.985080 27682 consensus_queue.cc:237] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "46a7d85f674d461882efa722b8b90976" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45397 } } peers { permanent_uuid: "15851e32b4dc453eb481c88ac7edca78" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 39381 } } peers { permanent_uuid: "1ea6088e6e0e475c825bdd9585613759" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37699 } attrs { } }
I20260812 08:03:14.986604 27895 raft_consensus.cc:1275] T df9755806eb24d1688f0c28cdce1e074 P 46a7d85f674d461882efa722b8b90976 [term 1 FOLLOWER]: Refusing update from remote peer 15851e32b4dc453eb481c88ac7edca78: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:03:14.987190 28242 consensus_queue.cc:1048] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a7d85f674d461882efa722b8b90976" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:14.989240 28237 raft_consensus.cc:2955] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1ea6088e6e0e475c825bdd9585613759 (127.18.42.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "46a7d85f674d461882efa722b8b90976" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45397 } } peers { permanent_uuid: "15851e32b4dc453eb481c88ac7edca78" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 39381 } } peers { permanent_uuid: "1ea6088e6e0e475c825bdd9585613759" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37699 } attrs { } } }
I20260812 08:03:14.989770 27895 raft_consensus.cc:2955] T df9755806eb24d1688f0c28cdce1e074 P 46a7d85f674d461882efa722b8b90976 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1ea6088e6e0e475c825bdd9585613759 (127.18.42.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "46a7d85f674d461882efa722b8b90976" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45397 } } peers { permanent_uuid: "15851e32b4dc453eb481c88ac7edca78" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 39381 } } peers { permanent_uuid: "1ea6088e6e0e475c825bdd9585613759" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37699 } attrs { } } }
W20260812 08:03:14.991680 27565 consensus_peers.cc:597] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 -> Peer 1ea6088e6e0e475c825bdd9585613759 (127.18.42.67:37699): Couldn't send request to peer 1ea6088e6e0e475c825bdd9585613759. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df9755806eb24d1688f0c28cdce1e074. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:03:14.992375 27441 catalog_manager.cc:5719] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 reported cstate change: config changed from index -1 to 2, VOTER 1ea6088e6e0e475c825bdd9585613759 (127.18.42.67) added. New cstate: current_term: 1 leader_uuid: "15851e32b4dc453eb481c88ac7edca78" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "46a7d85f674d461882efa722b8b90976" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15851e32b4dc453eb481c88ac7edca78" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 39381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ea6088e6e0e475c825bdd9585613759" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37699 } attrs { } health_report { overall_health: UNKNOWN } } }
I20260812 08:03:14.994604 18601 cluster_itest_util.cc:258] Not converged past 2 yet: 1.2 <uninitialized op> 1.2
I20260812 08:03:15.039402 28184 heartbeater.cc:499] Master 127.18.42.126:40195 was elected leader, sending a full tablet report...
I20260812 08:03:15.097579 18601 cluster_itest_util.cc:258] Not converged past 2 yet: 1.2 <uninitialized op> 1.2
I20260812 08:03:15.300784 18601 cluster_itest_util.cc:258] Not converged past 2 yet: 1.2 <uninitialized op> 1.2
I20260812 08:03:15.602466 28256 ts_tablet_manager.cc:933] T df9755806eb24d1688f0c28cdce1e074 P 1ea6088e6e0e475c825bdd9585613759: Initiating tablet copy from peer 15851e32b4dc453eb481c88ac7edca78 (127.18.42.65:39381)
I20260812 08:03:15.603279 28256 tablet_copy_client.cc:323] T df9755806eb24d1688f0c28cdce1e074 P 1ea6088e6e0e475c825bdd9585613759: tablet copy: Beginning tablet copy session from remote peer at address 127.18.42.65:39381
I20260812 08:03:15.604321 18601 cluster_itest_util.cc:258] Not converged past 2 yet: 1.2 <uninitialized op> 1.2
I20260812 08:03:15.607388 27711 tablet_copy_service.cc:140] P 15851e32b4dc453eb481c88ac7edca78: Received BeginTabletCopySession request for tablet df9755806eb24d1688f0c28cdce1e074 from peer 1ea6088e6e0e475c825bdd9585613759 ({username='slave'} at 127.18.42.67:37667)
I20260812 08:03:15.607542 27711 tablet_copy_service.cc:161] P 15851e32b4dc453eb481c88ac7edca78: Beginning new tablet copy session on tablet df9755806eb24d1688f0c28cdce1e074 from peer 1ea6088e6e0e475c825bdd9585613759 at {username='slave'} at 127.18.42.67:37667: session id = 1ea6088e6e0e475c825bdd9585613759-df9755806eb24d1688f0c28cdce1e074
I20260812 08:03:15.608753 27711 tablet_copy_source_session.cc:215] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 08:03:15.611963 28256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df9755806eb24d1688f0c28cdce1e074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:15.616375 28256 tablet_copy_client.cc:806] T df9755806eb24d1688f0c28cdce1e074 P 1ea6088e6e0e475c825bdd9585613759: tablet copy: Starting download of 0 data blocks...
I20260812 08:03:15.616590 28256 tablet_copy_client.cc:670] T df9755806eb24d1688f0c28cdce1e074 P 1ea6088e6e0e475c825bdd9585613759: tablet copy: Starting download of 1 WAL segments...
I20260812 08:03:15.618623 28256 tablet_copy_client.cc:538] T df9755806eb24d1688f0c28cdce1e074 P 1ea6088e6e0e475c825bdd9585613759: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 08:03:15.620232 28256 tablet_bootstrap.cc:492] T df9755806eb24d1688f0c28cdce1e074 P 1ea6088e6e0e475c825bdd9585613759: Bootstrap starting.
I20260812 08:03:15.624614 28256 log.cc:826] T df9755806eb24d1688f0c28cdce1e074 P 1ea6088e6e0e475c825bdd9585613759: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:15.626920 28256 tablet_bootstrap.cc:492] T df9755806eb24d1688f0c28cdce1e074 P 1ea6088e6e0e475c825bdd9585613759: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:03:15.627235 28256 tablet_bootstrap.cc:492] T df9755806eb24d1688f0c28cdce1e074 P 1ea6088e6e0e475c825bdd9585613759: Bootstrap complete.
I20260812 08:03:15.627523 28256 ts_tablet_manager.cc:1403] T df9755806eb24d1688f0c28cdce1e074 P 1ea6088e6e0e475c825bdd9585613759: Time spent bootstrapping tablet: real 0.007s	user 0.007s	sys 0.000s
I20260812 08:03:15.629878 28256 raft_consensus.cc:359] T df9755806eb24d1688f0c28cdce1e074 P 1ea6088e6e0e475c825bdd9585613759 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "46a7d85f674d461882efa722b8b90976" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45397 } } peers { permanent_uuid: "15851e32b4dc453eb481c88ac7edca78" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 39381 } } peers { permanent_uuid: "1ea6088e6e0e475c825bdd9585613759" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37699 } attrs { } }
I20260812 08:03:15.630092 28256 raft_consensus.cc:740] T df9755806eb24d1688f0c28cdce1e074 P 1ea6088e6e0e475c825bdd9585613759 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ea6088e6e0e475c825bdd9585613759, State: Initialized, Role: FOLLOWER
I20260812 08:03:15.630550 28256 consensus_queue.cc:260] T df9755806eb24d1688f0c28cdce1e074 P 1ea6088e6e0e475c825bdd9585613759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "46a7d85f674d461882efa722b8b90976" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45397 } } peers { permanent_uuid: "15851e32b4dc453eb481c88ac7edca78" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 39381 } } peers { permanent_uuid: "1ea6088e6e0e475c825bdd9585613759" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37699 } attrs { } }
I20260812 08:03:15.631269 28256 ts_tablet_manager.cc:1434] T df9755806eb24d1688f0c28cdce1e074 P 1ea6088e6e0e475c825bdd9585613759: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:03:15.632165 27711 tablet_copy_service.cc:342] P 15851e32b4dc453eb481c88ac7edca78: Request end of tablet copy session 1ea6088e6e0e475c825bdd9585613759-df9755806eb24d1688f0c28cdce1e074 received from {username='slave'} at 127.18.42.67:37667
I20260812 08:03:15.632308 27711 tablet_copy_service.cc:434] P 15851e32b4dc453eb481c88ac7edca78: ending tablet copy session 1ea6088e6e0e475c825bdd9585613759-df9755806eb24d1688f0c28cdce1e074 on tablet df9755806eb24d1688f0c28cdce1e074 with peer 1ea6088e6e0e475c825bdd9585613759
I20260812 08:03:16.007583 18601 raft_consensus-itest.cc:1835] Waiting for Master to see config change...
I20260812 08:03:16.009014 18601 raft_consensus-itest.cc:1841] Tablet locations:
tablet_locations {
  tablet_id: "df9755806eb24d1688f0c28cdce1e074"
  DEPRECATED_stale: false
  partition {
    partition_key_start: ""
    partition_key_end: ""
  }
  interned_replicas {
    ts_info_idx: 0
    role: FOLLOWER
  }
  interned_replicas {
    ts_info_idx: 1
    role: LEADER
  }
  interned_replicas {
    ts_info_idx: 2
    role: FOLLOWER
  }
}
ts_infos {
  permanent_uuid: "46a7d85f674d461882efa722b8b90976"
  rpc_addresses {
    host: "127.18.42.66"
    port: 45397
  }
}
ts_infos {
  permanent_uuid: "15851e32b4dc453eb481c88ac7edca78"
  rpc_addresses {
    host: "127.18.42.65"
    port: 39381
  }
}
ts_infos {
  permanent_uuid: "1ea6088e6e0e475c825bdd9585613759"
  rpc_addresses {
    host: "127.18.42.67"
    port: 37699
  }
}
I20260812 08:03:16.009135 18601 raft_consensus-itest.cc:1844] Removing tserver with uuid 1ea6088e6e0e475c825bdd9585613759
I20260812 08:03:16.010008 27682 consensus_queue.cc:237] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "46a7d85f674d461882efa722b8b90976" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45397 } } peers { permanent_uuid: "15851e32b4dc453eb481c88ac7edca78" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 39381 } }
I20260812 08:03:16.011301 27894 raft_consensus.cc:1275] T df9755806eb24d1688f0c28cdce1e074 P 46a7d85f674d461882efa722b8b90976 [term 1 FOLLOWER]: Refusing update from remote peer 15851e32b4dc453eb481c88ac7edca78: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 08:03:16.011897 28237 consensus_queue.cc:1048] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a7d85f674d461882efa722b8b90976" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 08:03:16.014184 28272 raft_consensus.cc:2955] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1ea6088e6e0e475c825bdd9585613759 (127.18.42.67) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "46a7d85f674d461882efa722b8b90976" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45397 } } peers { permanent_uuid: "15851e32b4dc453eb481c88ac7edca78" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 39381 } } }
I20260812 08:03:16.014729 27894 raft_consensus.cc:2955] T df9755806eb24d1688f0c28cdce1e074 P 46a7d85f674d461882efa722b8b90976 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1ea6088e6e0e475c825bdd9585613759 (127.18.42.67) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "46a7d85f674d461882efa722b8b90976" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45397 } } peers { permanent_uuid: "15851e32b4dc453eb481c88ac7edca78" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 39381 } } }
I20260812 08:03:16.017107 27444 catalog_manager.cc:5719] T df9755806eb24d1688f0c28cdce1e074 P 15851e32b4dc453eb481c88ac7edca78 reported cstate change: config changed from index 2 to 3, VOTER 1ea6088e6e0e475c825bdd9585613759 (127.18.42.67) evicted. New cstate: current_term: 1 leader_uuid: "15851e32b4dc453eb481c88ac7edca78" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "46a7d85f674d461882efa722b8b90976" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15851e32b4dc453eb481c88ac7edca78" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 39381 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:16.017668 18601 raft_consensus-itest.cc:1851] Waiting for Master to see config change...
I20260812 08:03:16.025086 28072 tablet_service.cc:1558] Processing DeleteTablet for tablet df9755806eb24d1688f0c28cdce1e074 with delete_type TABLET_DATA_TOMBSTONED (TS 1ea6088e6e0e475c825bdd9585613759 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:48606
I20260812 08:03:16.025725 28284 tablet_replica.cc:333] T df9755806eb24d1688f0c28cdce1e074 P 1ea6088e6e0e475c825bdd9585613759: stopping tablet replica
I20260812 08:03:16.026114 28284 raft_consensus.cc:2243] T df9755806eb24d1688f0c28cdce1e074 P 1ea6088e6e0e475c825bdd9585613759 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 08:03:16.026386 28284 raft_consensus.cc:2272] T df9755806eb24d1688f0c28cdce1e074 P 1ea6088e6e0e475c825bdd9585613759 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 08:03:16.027341 28284 ts_tablet_manager.cc:1916] T df9755806eb24d1688f0c28cdce1e074 P 1ea6088e6e0e475c825bdd9585613759: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 08:03:16.029742 28284 ts_tablet_manager.cc:1929] T df9755806eb24d1688f0c28cdce1e074 P 1ea6088e6e0e475c825bdd9585613759: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20260812 08:03:16.029837 28284 log.cc:1199] T df9755806eb24d1688f0c28cdce1e074 P 1ea6088e6e0e475c825bdd9585613759: Deleting WAL directory at /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestMasterNotifiedOnConfigChange.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal/wals/df9755806eb24d1688f0c28cdce1e074
I20260812 08:03:16.030588 27409 catalog_manager.cc:5050] TS 1ea6088e6e0e475c825bdd9585613759 (127.18.42.67:37699): tablet df9755806eb24d1688f0c28cdce1e074 (table TestTable [id=2c412547babe4429ae1185c9ea3783b4]) successfully deleted
I20260812 08:03:16.040452 18601 raft_consensus-itest.cc:1857] Tablet locations:
tablet_locations {
  tablet_id: "df9755806eb24d1688f0c28cdce1e074"
  DEPRECATED_stale: false
  partition {
    partition_key_start: ""
    partition_key_end: ""
  }
  interned_replicas {
    ts_info_idx: 0
    role: FOLLOWER
  }
  interned_replicas {
    ts_info_idx: 1
    role: LEADER
  }
}
ts_infos {
  permanent_uuid: "46a7d85f674d461882efa722b8b90976"
  rpc_addresses {
    host: "127.18.42.66"
    port: 45397
  }
}
ts_infos {
  permanent_uuid: "15851e32b4dc453eb481c88ac7edca78"
  rpc_addresses {
    host: "127.18.42.65"
    port: 39381
  }
}
I20260812 08:03:16.041055 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 27539
I20260812 08:03:16.054553 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 27754
I20260812 08:03:16.068002 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 27968
I20260812 08:03:16.085309 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 27378
2026-08-12T08:03:16Z chronyd exiting
[       OK ] RaftConsensusITest.TestMasterNotifiedOnConfigChange (2849 ms)
[ RUN      ] RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection
I20260812 08:03:16.123726 18601 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:03:16.123787 18601 ts_itest-base.cc:116] --------------
I20260812 08:03:16.123800 18601 ts_itest-base.cc:117] 3 tablet servers
I20260812 08:03:16.123813 18601 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:03:16.123824 18601 ts_itest-base.cc:119] --------------
2026-08-12T08:03:16Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:03:16Z Disabled control of system clock
I20260812 08:03:16.134788 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.42.126:36583
--webserver_interface=127.18.42.126
--webserver_port=0
--builtin_ntp_servers=127.18.42.84:42005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.42.126:36583 with env {}
W20260812 08:03:16.245086 28295 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:16.245375 28295 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:16.245455 28295 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:16.248924 28295 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:03:16.249035 28295 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:16.249076 28295 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:03:16.249117 28295 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:03:16.253129 28295 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:42005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.42.126:36583
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.42.126:36583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.18.42.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.28295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:16.254374 28295 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:16.255579 28295 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:16.261266 28304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:16.261270 28302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:16.261798 28306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:16.262229 28295 server_base.cc:1061] running on GCE node
I20260812 08:03:16.262827 28295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:16.263862 28295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:16.265039 28295 hybrid_clock.cc:648] HybridClock initialized: now 1786521796265018 us; error 36 us; skew 500 ppm
I20260812 08:03:16.267102 28295 webserver.cc:533] Webserver started at http://127.18.42.126:40499/ using document root <none> and password file <none>
I20260812 08:03:16.267726 28295 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:16.267786 28295 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:16.268013 28295 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:16.269744 28295 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "84099356ba8d4ca7b503d9eb1059c1f0"
format_stamp: "Formatted at 2026-08-12 08:03:16 on dist-test-slave-zr1t"
I20260812 08:03:16.270272 28295 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "84099356ba8d4ca7b503d9eb1059c1f0"
format_stamp: "Formatted at 2026-08-12 08:03:16 on dist-test-slave-zr1t"
I20260812 08:03:16.273952 28295 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.001s
I20260812 08:03:16.276226 28324 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:16.277310 28295 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:16.277431 28295 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
uuid: "84099356ba8d4ca7b503d9eb1059c1f0"
format_stamp: "Formatted at 2026-08-12 08:03:16 on dist-test-slave-zr1t"
I20260812 08:03:16.277578 28295 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:16.289453 28295 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:16.290239 28295 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:16.290410 28295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:16.298718 28295 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.126:36583
I20260812 08:03:16.298760 28414 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.126:36583 every 8 connection(s)
I20260812 08:03:16.299870 28295 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:03:16.301223 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 28295
I20260812 08:03:16.301317 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:03:16.303346 28416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:16.309319 28416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84099356ba8d4ca7b503d9eb1059c1f0: Bootstrap starting.
I20260812 08:03:16.311652 28416 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84099356ba8d4ca7b503d9eb1059c1f0: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:16.312394 28416 log.cc:826] T 00000000000000000000000000000000 P 84099356ba8d4ca7b503d9eb1059c1f0: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:16.314419 28416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84099356ba8d4ca7b503d9eb1059c1f0: No bootstrap required, opened a new log
I20260812 08:03:16.317358 28416 raft_consensus.cc:359] T 00000000000000000000000000000000 P 84099356ba8d4ca7b503d9eb1059c1f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84099356ba8d4ca7b503d9eb1059c1f0" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 36583 } }
I20260812 08:03:16.317617 28416 raft_consensus.cc:385] T 00000000000000000000000000000000 P 84099356ba8d4ca7b503d9eb1059c1f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:16.317674 28416 raft_consensus.cc:740] T 00000000000000000000000000000000 P 84099356ba8d4ca7b503d9eb1059c1f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84099356ba8d4ca7b503d9eb1059c1f0, State: Initialized, Role: FOLLOWER
I20260812 08:03:16.318113 28416 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84099356ba8d4ca7b503d9eb1059c1f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84099356ba8d4ca7b503d9eb1059c1f0" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 36583 } }
I20260812 08:03:16.318244 28416 raft_consensus.cc:399] T 00000000000000000000000000000000 P 84099356ba8d4ca7b503d9eb1059c1f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:03:16.318301 28416 raft_consensus.cc:493] T 00000000000000000000000000000000 P 84099356ba8d4ca7b503d9eb1059c1f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:03:16.318439 28416 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 84099356ba8d4ca7b503d9eb1059c1f0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:16.319500 28416 raft_consensus.cc:515] T 00000000000000000000000000000000 P 84099356ba8d4ca7b503d9eb1059c1f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84099356ba8d4ca7b503d9eb1059c1f0" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 36583 } }
I20260812 08:03:16.319832 28416 leader_election.cc:304] T 00000000000000000000000000000000 P 84099356ba8d4ca7b503d9eb1059c1f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84099356ba8d4ca7b503d9eb1059c1f0; no voters: 
I20260812 08:03:16.320112 28416 leader_election.cc:290] T 00000000000000000000000000000000 P 84099356ba8d4ca7b503d9eb1059c1f0 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:03:16.320245 28425 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 84099356ba8d4ca7b503d9eb1059c1f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:16.320497 28425 raft_consensus.cc:697] T 00000000000000000000000000000000 P 84099356ba8d4ca7b503d9eb1059c1f0 [term 1 LEADER]: Becoming Leader. State: Replica: 84099356ba8d4ca7b503d9eb1059c1f0, State: Running, Role: LEADER
I20260812 08:03:16.320835 28425 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84099356ba8d4ca7b503d9eb1059c1f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84099356ba8d4ca7b503d9eb1059c1f0" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 36583 } }
I20260812 08:03:16.321385 28416 sys_catalog.cc:565] T 00000000000000000000000000000000 P 84099356ba8d4ca7b503d9eb1059c1f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:16.322500 28427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84099356ba8d4ca7b503d9eb1059c1f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84099356ba8d4ca7b503d9eb1059c1f0. Latest consensus state: current_term: 1 leader_uuid: "84099356ba8d4ca7b503d9eb1059c1f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84099356ba8d4ca7b503d9eb1059c1f0" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 36583 } } }
I20260812 08:03:16.322497 28426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84099356ba8d4ca7b503d9eb1059c1f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84099356ba8d4ca7b503d9eb1059c1f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84099356ba8d4ca7b503d9eb1059c1f0" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 36583 } } }
I20260812 08:03:16.322665 28427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84099356ba8d4ca7b503d9eb1059c1f0 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:16.322665 28426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84099356ba8d4ca7b503d9eb1059c1f0 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:16.323194 28439 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:03:16.326068 28439 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:03:16.331238 28439 catalog_manager.cc:1383] Generated new cluster ID: e80755962db2478f9a579cdcc52ed0f0
I20260812 08:03:16.331318 28439 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:03:16.337386 28439 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:03:16.338261 28439 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:03:16.343361 28439 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 84099356ba8d4ca7b503d9eb1059c1f0: Generated new TSK 0
I20260812 08:03:16.343945 28439 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:03:16.346903 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.65:0
--local_ip_for_outbound_sockets=127.18.42.65
--webserver_interface=127.18.42.65
--webserver_port=0
--tserver_master_addrs=127.18.42.126:36583
--builtin_ntp_servers=127.18.42.84:42005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_max_batch_size_bytes=50000 with env {}
W20260812 08:03:16.462004 28452 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:16.462210 28452 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:16.462302 28452 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:16.465932 28452 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:16.466048 28452 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.65
I20260812 08:03:16.470029 28452 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:42005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_max_batch_size_bytes=50000
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.18.42.65
--webserver_port=0
--tserver_master_addrs=127.18.42.126:36583
--never_fsync=true
--heap_profile_path=/tmp/kudu.28452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:16.471174 28452 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:16.472348 28452 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:16.478981 28470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:16.479068 28472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:16.479159 28465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:16.479921 28452 server_base.cc:1061] running on GCE node
I20260812 08:03:16.480319 28452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:16.480944 28452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:16.482101 28452 hybrid_clock.cc:648] HybridClock initialized: now 1786521796482080 us; error 32 us; skew 500 ppm
I20260812 08:03:16.484278 28452 webserver.cc:533] Webserver started at http://127.18.42.65:45129/ using document root <none> and password file <none>
I20260812 08:03:16.484975 28452 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:16.485049 28452 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:16.485252 28452 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:16.487303 28452 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "f3db342ac1a34617858477795c47468e"
format_stamp: "Formatted at 2026-08-12 08:03:16 on dist-test-slave-zr1t"
I20260812 08:03:16.487917 28452 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "f3db342ac1a34617858477795c47468e"
format_stamp: "Formatted at 2026-08-12 08:03:16 on dist-test-slave-zr1t"
I20260812 08:03:16.492182 28452 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.002s
I20260812 08:03:16.494890 28485 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:16.496043 28452 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:03:16.496191 28452 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "f3db342ac1a34617858477795c47468e"
format_stamp: "Formatted at 2026-08-12 08:03:16 on dist-test-slave-zr1t"
I20260812 08:03:16.496341 28452 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:16.507174 28452 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:16.507814 28452 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:16.508019 28452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:16.508625 28452 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:16.509770 28452 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:16.509845 28452 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:16.509939 28452 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:16.509996 28452 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:16.520429 28452 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.65:40187
I20260812 08:03:16.520463 28672 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.65:40187 every 8 connection(s)
I20260812 08:03:16.521548 28452 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:03:16.524986 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 28452
I20260812 08:03:16.525121 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:03:16.527556 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.66:0
--local_ip_for_outbound_sockets=127.18.42.66
--webserver_interface=127.18.42.66
--webserver_port=0
--tserver_master_addrs=127.18.42.126:36583
--builtin_ntp_servers=127.18.42.84:42005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_max_batch_size_bytes=50000 with env {}
I20260812 08:03:16.531908 28673 heartbeater.cc:344] Connected to a master server at 127.18.42.126:36583
I20260812 08:03:16.532227 28673 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:16.532920 28673 heartbeater.cc:507] Master 127.18.42.126:36583 requested a full tablet report, sending...
I20260812 08:03:16.534680 28355 ts_manager.cc:194] Registered new tserver with Master: f3db342ac1a34617858477795c47468e (127.18.42.65:40187)
I20260812 08:03:16.535917 28355 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.65:41943
W20260812 08:03:16.638703 28678 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:16.638932 28678 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:16.639016 28678 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:16.642495 28678 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:16.642616 28678 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.66
I20260812 08:03:16.646709 28678 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:42005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_max_batch_size_bytes=50000
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.18.42.66
--webserver_port=0
--tserver_master_addrs=127.18.42.126:36583
--never_fsync=true
--heap_profile_path=/tmp/kudu.28678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:16.647892 28678 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:16.649109 28678 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:16.656229 28685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:16.656212 28684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:16.656212 28688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:16.656790 28678 server_base.cc:1061] running on GCE node
I20260812 08:03:16.657229 28678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:16.657861 28678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:16.659077 28678 hybrid_clock.cc:648] HybridClock initialized: now 1786521796659061 us; error 36 us; skew 500 ppm
I20260812 08:03:16.661301 28678 webserver.cc:533] Webserver started at http://127.18.42.66:42041/ using document root <none> and password file <none>
I20260812 08:03:16.662032 28678 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:16.662156 28678 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:16.662446 28678 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:16.664232 28678 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "bcb6e372d3b548a1b1bc95753ce79516"
format_stamp: "Formatted at 2026-08-12 08:03:16 on dist-test-slave-zr1t"
I20260812 08:03:16.664773 28678 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "bcb6e372d3b548a1b1bc95753ce79516"
format_stamp: "Formatted at 2026-08-12 08:03:16 on dist-test-slave-zr1t"
I20260812 08:03:16.668651 28678 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260812 08:03:16.671288 28700 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:16.672587 28678 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:03:16.672746 28678 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "bcb6e372d3b548a1b1bc95753ce79516"
format_stamp: "Formatted at 2026-08-12 08:03:16 on dist-test-slave-zr1t"
I20260812 08:03:16.672866 28678 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:16.694195 28678 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:16.694963 28678 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:16.695186 28678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:16.695871 28678 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:16.696928 28678 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:16.696977 28678 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:16.697044 28678 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:16.697084 28678 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:16.707615 28678 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.66:37781
I20260812 08:03:16.707857 28888 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.66:37781 every 8 connection(s)
I20260812 08:03:16.708734 28678 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:03:16.716506 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 28678
I20260812 08:03:16.716703 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:03:16.718943 28889 heartbeater.cc:344] Connected to a master server at 127.18.42.126:36583
I20260812 08:03:16.719242 28889 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:16.719672 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.67:0
--local_ip_for_outbound_sockets=127.18.42.67
--webserver_interface=127.18.42.67
--webserver_port=0
--tserver_master_addrs=127.18.42.126:36583
--builtin_ntp_servers=127.18.42.84:42005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_max_batch_size_bytes=50000 with env {}
I20260812 08:03:16.719866 28889 heartbeater.cc:507] Master 127.18.42.126:36583 requested a full tablet report, sending...
I20260812 08:03:16.721098 28355 ts_manager.cc:194] Registered new tserver with Master: bcb6e372d3b548a1b1bc95753ce79516 (127.18.42.66:37781)
I20260812 08:03:16.722036 28355 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.66:56365
W20260812 08:03:16.834929 28896 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:16.835222 28896 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:16.835301 28896 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:16.838887 28896 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:16.839078 28896 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.67
I20260812 08:03:16.843240 28896 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:42005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_max_batch_size_bytes=50000
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.18.42.67
--webserver_port=0
--tserver_master_addrs=127.18.42.126:36583
--never_fsync=true
--heap_profile_path=/tmp/kudu.28896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:16.844480 28896 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:16.845723 28896 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:16.852578 28906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:16.852581 28902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:16.852581 28903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:16.852648 28896 server_base.cc:1061] running on GCE node
I20260812 08:03:16.853585 28896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:16.854200 28896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:16.855403 28896 hybrid_clock.cc:648] HybridClock initialized: now 1786521796855384 us; error 38 us; skew 500 ppm
I20260812 08:03:16.857497 28896 webserver.cc:533] Webserver started at http://127.18.42.67:35691/ using document root <none> and password file <none>
I20260812 08:03:16.858218 28896 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:16.858314 28896 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:16.858536 28896 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:16.860266 28896 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "43e48d5b5b544be4b8eb7b35f80aab4b"
format_stamp: "Formatted at 2026-08-12 08:03:16 on dist-test-slave-zr1t"
I20260812 08:03:16.860849 28896 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "43e48d5b5b544be4b8eb7b35f80aab4b"
format_stamp: "Formatted at 2026-08-12 08:03:16 on dist-test-slave-zr1t"
I20260812 08:03:16.865087 28896 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.003s
I20260812 08:03:16.867748 28913 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:16.868832 28896 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260812 08:03:16.868983 28896 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "43e48d5b5b544be4b8eb7b35f80aab4b"
format_stamp: "Formatted at 2026-08-12 08:03:16 on dist-test-slave-zr1t"
I20260812 08:03:16.869095 28896 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:16.910254 28896 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:16.911198 28896 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:16.911435 28896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:16.912087 28896 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:16.913196 28896 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:16.913272 28896 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:16.913341 28896 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:16.913371 28896 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:16.925238 28896 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.67:41601
I20260812 08:03:16.925336 29098 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.67:41601 every 8 connection(s)
I20260812 08:03:16.926421 28896 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:03:16.928340 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 28896
I20260812 08:03:16.928491 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:03:16.936998 29099 heartbeater.cc:344] Connected to a master server at 127.18.42.126:36583
I20260812 08:03:16.937453 29099 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:16.938162 29099 heartbeater.cc:507] Master 127.18.42.126:36583 requested a full tablet report, sending...
I20260812 08:03:16.939455 28355 ts_manager.cc:194] Registered new tserver with Master: 43e48d5b5b544be4b8eb7b35f80aab4b (127.18.42.67:41601)
I20260812 08:03:16.940083 28355 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.67:51835
I20260812 08:03:16.943914 18601 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:03:16.954576 28355 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56054:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
W20260812 08:03:16.957146 28355 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 08:03:16.975176 28566 tablet_service.cc:1511] Processing CreateTablet for tablet 03038146670b4cb48c4fd4d21e7d2620 (DEFAULT_TABLE table=TestTable [id=28f02e33575840afa0f6c019713d0276]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:16.975476 28783 tablet_service.cc:1511] Processing CreateTablet for tablet 03038146670b4cb48c4fd4d21e7d2620 (DEFAULT_TABLE table=TestTable [id=28f02e33575840afa0f6c019713d0276]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:16.975476 28993 tablet_service.cc:1511] Processing CreateTablet for tablet 03038146670b4cb48c4fd4d21e7d2620 (DEFAULT_TABLE table=TestTable [id=28f02e33575840afa0f6c019713d0276]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:16.976241 28566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03038146670b4cb48c4fd4d21e7d2620. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:16.976444 28993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03038146670b4cb48c4fd4d21e7d2620. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:16.976444 28783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03038146670b4cb48c4fd4d21e7d2620. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:16.981909 29128 tablet_bootstrap.cc:492] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e: Bootstrap starting.
I20260812 08:03:16.983321 29131 tablet_bootstrap.cc:492] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b: Bootstrap starting.
I20260812 08:03:16.984808 29128 tablet_bootstrap.cc:654] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:16.985242 29131 tablet_bootstrap.cc:654] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:16.985720 29128 log.cc:826] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:16.985906 29129 tablet_bootstrap.cc:492] T 03038146670b4cb48c4fd4d21e7d2620 P bcb6e372d3b548a1b1bc95753ce79516: Bootstrap starting.
I20260812 08:03:16.986135 29131 log.cc:826] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:16.987543 29128 tablet_bootstrap.cc:492] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e: No bootstrap required, opened a new log
I20260812 08:03:16.987788 29128 ts_tablet_manager.cc:1403] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e: Time spent bootstrapping tablet: real 0.006s	user 0.004s	sys 0.000s
I20260812 08:03:16.988029 29131 tablet_bootstrap.cc:492] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b: No bootstrap required, opened a new log
I20260812 08:03:16.988209 29129 tablet_bootstrap.cc:654] T 03038146670b4cb48c4fd4d21e7d2620 P bcb6e372d3b548a1b1bc95753ce79516: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:16.988240 29131 ts_tablet_manager.cc:1403] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b: Time spent bootstrapping tablet: real 0.005s	user 0.000s	sys 0.003s
I20260812 08:03:16.989094 29129 log.cc:826] T 03038146670b4cb48c4fd4d21e7d2620 P bcb6e372d3b548a1b1bc95753ce79516: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:16.990974 29129 tablet_bootstrap.cc:492] T 03038146670b4cb48c4fd4d21e7d2620 P bcb6e372d3b548a1b1bc95753ce79516: No bootstrap required, opened a new log
I20260812 08:03:16.990868 29128 raft_consensus.cc:359] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3db342ac1a34617858477795c47468e" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40187 } } peers { permanent_uuid: "43e48d5b5b544be4b8eb7b35f80aab4b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41601 } } peers { permanent_uuid: "bcb6e372d3b548a1b1bc95753ce79516" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 37781 } }
I20260812 08:03:16.991082 29128 raft_consensus.cc:385] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:16.991168 29128 raft_consensus.cc:740] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3db342ac1a34617858477795c47468e, State: Initialized, Role: FOLLOWER
I20260812 08:03:16.991173 29129 ts_tablet_manager.cc:1403] T 03038146670b4cb48c4fd4d21e7d2620 P bcb6e372d3b548a1b1bc95753ce79516: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.000s
I20260812 08:03:16.991353 29131 raft_consensus.cc:359] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3db342ac1a34617858477795c47468e" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40187 } } peers { permanent_uuid: "43e48d5b5b544be4b8eb7b35f80aab4b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41601 } } peers { permanent_uuid: "bcb6e372d3b548a1b1bc95753ce79516" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 37781 } }
I20260812 08:03:16.991557 29131 raft_consensus.cc:385] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:16.991626 29131 raft_consensus.cc:740] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43e48d5b5b544be4b8eb7b35f80aab4b, State: Initialized, Role: FOLLOWER
I20260812 08:03:16.991650 29128 consensus_queue.cc:260] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3db342ac1a34617858477795c47468e" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40187 } } peers { permanent_uuid: "43e48d5b5b544be4b8eb7b35f80aab4b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41601 } } peers { permanent_uuid: "bcb6e372d3b548a1b1bc95753ce79516" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 37781 } }
I20260812 08:03:16.992126 29131 consensus_queue.cc:260] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3db342ac1a34617858477795c47468e" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40187 } } peers { permanent_uuid: "43e48d5b5b544be4b8eb7b35f80aab4b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41601 } } peers { permanent_uuid: "bcb6e372d3b548a1b1bc95753ce79516" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 37781 } }
I20260812 08:03:16.992370 28673 heartbeater.cc:499] Master 127.18.42.126:36583 was elected leader, sending a full tablet report...
I20260812 08:03:16.992652 29128 ts_tablet_manager.cc:1434] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260812 08:03:16.992995 29131 ts_tablet_manager.cc:1434] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.005s
I20260812 08:03:16.993007 29099 heartbeater.cc:499] Master 127.18.42.126:36583 was elected leader, sending a full tablet report...
I20260812 08:03:16.994201 29140 raft_consensus.cc:493] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:16.994575 29140 raft_consensus.cc:515] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3db342ac1a34617858477795c47468e" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40187 } } peers { permanent_uuid: "43e48d5b5b544be4b8eb7b35f80aab4b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41601 } } peers { permanent_uuid: "bcb6e372d3b548a1b1bc95753ce79516" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 37781 } }
I20260812 08:03:16.995148 29129 raft_consensus.cc:359] T 03038146670b4cb48c4fd4d21e7d2620 P bcb6e372d3b548a1b1bc95753ce79516 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3db342ac1a34617858477795c47468e" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40187 } } peers { permanent_uuid: "43e48d5b5b544be4b8eb7b35f80aab4b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41601 } } peers { permanent_uuid: "bcb6e372d3b548a1b1bc95753ce79516" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 37781 } }
I20260812 08:03:16.995398 29129 raft_consensus.cc:385] T 03038146670b4cb48c4fd4d21e7d2620 P bcb6e372d3b548a1b1bc95753ce79516 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:16.995502 29129 raft_consensus.cc:740] T 03038146670b4cb48c4fd4d21e7d2620 P bcb6e372d3b548a1b1bc95753ce79516 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb6e372d3b548a1b1bc95753ce79516, State: Initialized, Role: FOLLOWER
I20260812 08:03:16.995985 29129 consensus_queue.cc:260] T 03038146670b4cb48c4fd4d21e7d2620 P bcb6e372d3b548a1b1bc95753ce79516 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3db342ac1a34617858477795c47468e" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40187 } } peers { permanent_uuid: "43e48d5b5b544be4b8eb7b35f80aab4b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41601 } } peers { permanent_uuid: "bcb6e372d3b548a1b1bc95753ce79516" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 37781 } }
I20260812 08:03:16.996657 28889 heartbeater.cc:499] Master 127.18.42.126:36583 was elected leader, sending a full tablet report...
I20260812 08:03:16.997977 29129 ts_tablet_manager.cc:1434] T 03038146670b4cb48c4fd4d21e7d2620 P bcb6e372d3b548a1b1bc95753ce79516: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.004s
I20260812 08:03:16.998564 29140 leader_election.cc:290] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3db342ac1a34617858477795c47468e (127.18.42.65:40187), bcb6e372d3b548a1b1bc95753ce79516 (127.18.42.66:37781)
I20260812 08:03:17.001253 28601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03038146670b4cb48c4fd4d21e7d2620" candidate_uuid: "43e48d5b5b544be4b8eb7b35f80aab4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3db342ac1a34617858477795c47468e" is_pre_election: true
I20260812 08:03:17.001653 28601 raft_consensus.cc:2468] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43e48d5b5b544be4b8eb7b35f80aab4b in term 0.
I20260812 08:03:17.002252 28921 leader_election.cc:304] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43e48d5b5b544be4b8eb7b35f80aab4b, f3db342ac1a34617858477795c47468e; no voters: 
I20260812 08:03:17.002526 29140 raft_consensus.cc:2804] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:03:17.002625 29140 raft_consensus.cc:493] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:03:17.002693 29140 raft_consensus.cc:3060] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:17.003540 28815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03038146670b4cb48c4fd4d21e7d2620" candidate_uuid: "43e48d5b5b544be4b8eb7b35f80aab4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb6e372d3b548a1b1bc95753ce79516" is_pre_election: true
I20260812 08:03:17.003826 28815 raft_consensus.cc:2468] T 03038146670b4cb48c4fd4d21e7d2620 P bcb6e372d3b548a1b1bc95753ce79516 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43e48d5b5b544be4b8eb7b35f80aab4b in term 0.
I20260812 08:03:17.003821 29140 raft_consensus.cc:515] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3db342ac1a34617858477795c47468e" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40187 } } peers { permanent_uuid: "43e48d5b5b544be4b8eb7b35f80aab4b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41601 } } peers { permanent_uuid: "bcb6e372d3b548a1b1bc95753ce79516" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 37781 } }
I20260812 08:03:17.004266 29140 leader_election.cc:290] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b [CANDIDATE]: Term 1 election: Requested vote from peers f3db342ac1a34617858477795c47468e (127.18.42.65:40187), bcb6e372d3b548a1b1bc95753ce79516 (127.18.42.66:37781)
I20260812 08:03:17.004601 28601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03038146670b4cb48c4fd4d21e7d2620" candidate_uuid: "43e48d5b5b544be4b8eb7b35f80aab4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3db342ac1a34617858477795c47468e"
I20260812 08:03:17.004734 28601 raft_consensus.cc:3060] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:17.004771 28815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03038146670b4cb48c4fd4d21e7d2620" candidate_uuid: "43e48d5b5b544be4b8eb7b35f80aab4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb6e372d3b548a1b1bc95753ce79516"
I20260812 08:03:17.004891 28815 raft_consensus.cc:3060] T 03038146670b4cb48c4fd4d21e7d2620 P bcb6e372d3b548a1b1bc95753ce79516 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:17.006340 28815 raft_consensus.cc:2468] T 03038146670b4cb48c4fd4d21e7d2620 P bcb6e372d3b548a1b1bc95753ce79516 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43e48d5b5b544be4b8eb7b35f80aab4b in term 1.
I20260812 08:03:17.006461 28601 raft_consensus.cc:2468] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43e48d5b5b544be4b8eb7b35f80aab4b in term 1.
I20260812 08:03:17.006726 28917 leader_election.cc:304] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43e48d5b5b544be4b8eb7b35f80aab4b, bcb6e372d3b548a1b1bc95753ce79516; no voters: 
I20260812 08:03:17.006944 29140 raft_consensus.cc:2804] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:17.007241 29140 raft_consensus.cc:697] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b [term 1 LEADER]: Becoming Leader. State: Replica: 43e48d5b5b544be4b8eb7b35f80aab4b, State: Running, Role: LEADER
I20260812 08:03:17.007578 29140 consensus_queue.cc:237] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3db342ac1a34617858477795c47468e" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40187 } } peers { permanent_uuid: "43e48d5b5b544be4b8eb7b35f80aab4b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41601 } } peers { permanent_uuid: "bcb6e372d3b548a1b1bc95753ce79516" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 37781 } }
I20260812 08:03:17.010991 28352 catalog_manager.cc:5719] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 43e48d5b5b544be4b8eb7b35f80aab4b (127.18.42.67). New cstate: current_term: 1 leader_uuid: "43e48d5b5b544be4b8eb7b35f80aab4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3db342ac1a34617858477795c47468e" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43e48d5b5b544be4b8eb7b35f80aab4b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcb6e372d3b548a1b1bc95753ce79516" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 37781 } health_report { overall_health: UNKNOWN } } }
I20260812 08:03:17.019742 18601 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:03:17.021144 18601 ts_itest-base.cc:246] Waiting for 1 tablets on tserver f3db342ac1a34617858477795c47468e to finish bootstrapping
W20260812 08:03:17.022785 28675 tablet.cc:2406] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:03:17.027818 18601 ts_itest-base.cc:246] Waiting for 1 tablets on tserver bcb6e372d3b548a1b1bc95753ce79516 to finish bootstrapping
I20260812 08:03:17.033654 18601 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 43e48d5b5b544be4b8eb7b35f80aab4b to finish bootstrapping
W20260812 08:03:17.049149 28879 generic_service.cc:139] {username='slave'} at 127.0.0.1:42950 forcing change of non-runtime-safe flag follower_reject_update_consensus_requests
I20260812 08:03:17.049316 28879 generic_service.cc:175] {username='slave'} at 127.0.0.1:42950 changed flags via RPC: follower_reject_update_consensus_requests from 'false' to 'true'
I20260812 08:03:17.050593 29091 generic_service.cc:175] {username='slave'} at 127.0.0.1:59416 changed flags via RPC: log_inject_latency_ms_mean from '100' to '6000'
I20260812 08:03:17.051802 29091 generic_service.cc:175] {username='slave'} at 127.0.0.1:59416 changed flags via RPC: log_inject_latency from 'false' to 'true'
W20260812 08:03:17.064122 29146 log.cc:510] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b: Injecting 5918ms of latency in SegmentAllocator::Sync()
I20260812 08:03:17.065291 28601 raft_consensus.cc:1275] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e [term 1 FOLLOWER]: Refusing update from remote peer 43e48d5b5b544be4b8eb7b35f80aab4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260812 08:03:17.065650 28917 consensus_peers.cc:597] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b -> Peer bcb6e372d3b548a1b1bc95753ce79516 (127.18.42.66:37781): Couldn't send request to peer bcb6e372d3b548a1b1bc95753ce79516. Error code: UNKNOWN_ERROR (1). Status: Illegal state: Rejected: --follower_reject_update_consensus_requests is set to true.. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:03:17.066042 29140 consensus_queue.cc:1048] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3db342ac1a34617858477795c47468e" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 08:03:17.156011 29107 meta_cache.cc:302] tablet 03038146670b4cb48c4fd4d21e7d2620: replica 43e48d5b5b544be4b8eb7b35f80aab4b (127.18.42.67:41601) has failed: Timed out: Write RPC to 127.18.42.67:41601 timed out after 0.099s (SENT)
W20260812 08:03:17.156270 29107 batcher.cc:441] Timed out: Failed to write batch of 1 ops to tablet 03038146670b4cb48c4fd4d21e7d2620 after 1 attempt(s): Failed to write to server: 43e48d5b5b544be4b8eb7b35f80aab4b (127.18.42.67:41601): Write RPC to 127.18.42.67:41601 timed out after 0.099s (SENT)
W20260812 08:03:17.209981 28892 tablet.cc:2406] T 03038146670b4cb48c4fd4d21e7d2620 P bcb6e372d3b548a1b1bc95753ce79516: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:03:18.073379 28663 generic_service.cc:139] {username='slave'} at 127.0.0.1:40176 forcing change of non-runtime-safe flag follower_reject_update_consensus_requests
I20260812 08:03:18.073607 28663 generic_service.cc:175] {username='slave'} at 127.0.0.1:40176 changed flags via RPC: follower_reject_update_consensus_requests from 'false' to 'true'
W20260812 08:03:18.079645 28921 consensus_peers.cc:597] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b -> Peer f3db342ac1a34617858477795c47468e (127.18.42.65:40187): Couldn't send request to peer f3db342ac1a34617858477795c47468e. Error code: UNKNOWN_ERROR (1). Status: Illegal state: Rejected: --follower_reject_update_consensus_requests is set to true.. This is attempt 1: this message will repeat every 5th retry.
W20260812 08:03:18.176620 29107 meta_cache.cc:302] tablet 03038146670b4cb48c4fd4d21e7d2620: replica 43e48d5b5b544be4b8eb7b35f80aab4b (127.18.42.67:41601) has failed: Timed out: Write RPC to 127.18.42.67:41601 timed out after 0.099s (SENT)
W20260812 08:03:18.176757 29107 batcher.cc:441] Timed out: Failed to write batch of 1 ops to tablet 03038146670b4cb48c4fd4d21e7d2620 after 1 attempt(s): Failed to write to server: 43e48d5b5b544be4b8eb7b35f80aab4b (127.18.42.67:41601): Write RPC to 127.18.42.67:41601 timed out after 0.099s (SENT)
I20260812 08:03:18.731779 29160 raft_consensus.cc:493] T 03038146670b4cb48c4fd4d21e7d2620 P bcb6e372d3b548a1b1bc95753ce79516 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:18.731997 29160 raft_consensus.cc:515] T 03038146670b4cb48c4fd4d21e7d2620 P bcb6e372d3b548a1b1bc95753ce79516 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3db342ac1a34617858477795c47468e" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40187 } } peers { permanent_uuid: "43e48d5b5b544be4b8eb7b35f80aab4b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41601 } } peers { permanent_uuid: "bcb6e372d3b548a1b1bc95753ce79516" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 37781 } }
I20260812 08:03:18.733177 29160 leader_election.cc:290] T 03038146670b4cb48c4fd4d21e7d2620 P bcb6e372d3b548a1b1bc95753ce79516 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3db342ac1a34617858477795c47468e (127.18.42.65:40187), 43e48d5b5b544be4b8eb7b35f80aab4b (127.18.42.67:41601)
I20260812 08:03:18.736711 28601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03038146670b4cb48c4fd4d21e7d2620" candidate_uuid: "bcb6e372d3b548a1b1bc95753ce79516" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3db342ac1a34617858477795c47468e" is_pre_election: true
I20260812 08:03:19.476168 29168 raft_consensus.cc:493] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43e48d5b5b544be4b8eb7b35f80aab4b)
I20260812 08:03:19.476415 29168 raft_consensus.cc:515] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3db342ac1a34617858477795c47468e" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40187 } } peers { permanent_uuid: "43e48d5b5b544be4b8eb7b35f80aab4b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41601 } } peers { permanent_uuid: "bcb6e372d3b548a1b1bc95753ce79516" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 37781 } }
I20260812 08:03:19.477722 29168 leader_election.cc:290] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43e48d5b5b544be4b8eb7b35f80aab4b (127.18.42.67:41601), bcb6e372d3b548a1b1bc95753ce79516 (127.18.42.66:37781)
I20260812 08:03:19.481842 28815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03038146670b4cb48c4fd4d21e7d2620" candidate_uuid: "f3db342ac1a34617858477795c47468e" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: false dest_uuid: "bcb6e372d3b548a1b1bc95753ce79516" is_pre_election: true
I20260812 08:03:19.481995 28815 raft_consensus.cc:2468] T 03038146670b4cb48c4fd4d21e7d2620 P bcb6e372d3b548a1b1bc95753ce79516 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3db342ac1a34617858477795c47468e in term 1.
I20260812 08:03:19.482456 28491 leader_election.cc:304] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcb6e372d3b548a1b1bc95753ce79516, f3db342ac1a34617858477795c47468e; no voters: 
I20260812 08:03:19.482704 29168 raft_consensus.cc:2804] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 08:03:19.482760 29168 raft_consensus.cc:493] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43e48d5b5b544be4b8eb7b35f80aab4b)
I20260812 08:03:19.482789 29168 raft_consensus.cc:3060] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:03:19.483616 29168 raft_consensus.cc:515] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3db342ac1a34617858477795c47468e" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40187 } } peers { permanent_uuid: "43e48d5b5b544be4b8eb7b35f80aab4b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41601 } } peers { permanent_uuid: "bcb6e372d3b548a1b1bc95753ce79516" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 37781 } }
I20260812 08:03:19.484010 29168 leader_election.cc:290] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e [CANDIDATE]: Term 2 election: Requested vote from peers 43e48d5b5b544be4b8eb7b35f80aab4b (127.18.42.67:41601), bcb6e372d3b548a1b1bc95753ce79516 (127.18.42.66:37781)
I20260812 08:03:19.484421 28815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03038146670b4cb48c4fd4d21e7d2620" candidate_uuid: "f3db342ac1a34617858477795c47468e" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: false dest_uuid: "bcb6e372d3b548a1b1bc95753ce79516"
I20260812 08:03:19.484536 28815 raft_consensus.cc:3060] T 03038146670b4cb48c4fd4d21e7d2620 P bcb6e372d3b548a1b1bc95753ce79516 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:03:19.485318 28815 raft_consensus.cc:2468] T 03038146670b4cb48c4fd4d21e7d2620 P bcb6e372d3b548a1b1bc95753ce79516 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3db342ac1a34617858477795c47468e in term 2.
I20260812 08:03:19.485719 28491 leader_election.cc:304] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcb6e372d3b548a1b1bc95753ce79516, f3db342ac1a34617858477795c47468e; no voters: 
I20260812 08:03:19.485932 29168 raft_consensus.cc:2804] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:03:19.486186 29168 raft_consensus.cc:697] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e [term 2 LEADER]: Becoming Leader. State: Replica: f3db342ac1a34617858477795c47468e, State: Running, Role: LEADER
I20260812 08:03:19.486783 29168 consensus_queue.cc:237] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.11, Last appended by leader: 11, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3db342ac1a34617858477795c47468e" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40187 } } peers { permanent_uuid: "43e48d5b5b544be4b8eb7b35f80aab4b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41601 } } peers { permanent_uuid: "bcb6e372d3b548a1b1bc95753ce79516" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 37781 } }
I20260812 08:03:19.489311 28352 catalog_manager.cc:5719] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e reported cstate change: term changed from 1 to 2, leader changed from 43e48d5b5b544be4b8eb7b35f80aab4b (127.18.42.67) to f3db342ac1a34617858477795c47468e (127.18.42.65). New cstate: current_term: 2 leader_uuid: "f3db342ac1a34617858477795c47468e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3db342ac1a34617858477795c47468e" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43e48d5b5b544be4b8eb7b35f80aab4b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb6e372d3b548a1b1bc95753ce79516" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 37781 } health_report { overall_health: UNKNOWN } } }
I20260812 08:03:19.495599 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 28678
W20260812 08:03:19.533131 28491 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.42.66:37781: connect: Connection refused (error 111)
W20260812 08:03:19.534152 28491 consensus_peers.cc:597] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e -> Peer bcb6e372d3b548a1b1bc95753ce79516 (127.18.42.66:37781): Couldn't send request to peer bcb6e372d3b548a1b1bc95753ce79516. Status: Network error: Client connection negotiation failed: client connection to 127.18.42.66:37781: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260812 08:03:19.623689 29197 meta_cache.cc:302] tablet 03038146670b4cb48c4fd4d21e7d2620: replica f3db342ac1a34617858477795c47468e (127.18.42.65:40187) has failed: Timed out: Write RPC to 127.18.42.65:40187 timed out after 0.099s (SENT)
W20260812 08:03:19.623816 29197 batcher.cc:441] Timed out: Failed to write batch of 1 ops to tablet 03038146670b4cb48c4fd4d21e7d2620 after 1 attempt(s): Failed to write to server: f3db342ac1a34617858477795c47468e (127.18.42.65:40187): Write RPC to 127.18.42.65:40187 timed out after 0.099s (SENT)
W20260812 08:03:19.627071 29210 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 08:03:19.625104 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.18.42.66:59731 (local address 127.18.42.67:41601)
0812 08:03:19.625875 (+   771us) server_negotiation.cc:207] Beginning negotiation
0812 08:03:19.625882 (+     7us) server_negotiation.cc:400] Waiting for connection header
0812 08:03:19.625903 (+    21us) server_negotiation.cc:408] Connection header received
0812 08:03:19.625967 (+    64us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 08:03:19.625970 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 08:03:19.626040 (+    70us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 08:03:19.626222 (+   182us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.42.66:59731: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":85,"threads_started":1}
W20260812 08:03:19.627934 28917 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.42.66:37781: connect: Connection refused (error 111)
I20260812 08:03:19.629204 18601 cluster_itest_util.cc:258] Not converged past 13 yet: 1.12 2.13
I20260812 08:03:19.630766 29025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03038146670b4cb48c4fd4d21e7d2620" candidate_uuid: "f3db342ac1a34617858477795c47468e" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: false dest_uuid: "43e48d5b5b544be4b8eb7b35f80aab4b"
I20260812 08:03:19.630766 29028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03038146670b4cb48c4fd4d21e7d2620" candidate_uuid: "f3db342ac1a34617858477795c47468e" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: false dest_uuid: "43e48d5b5b544be4b8eb7b35f80aab4b" is_pre_election: true
I20260812 08:03:19.631088 29024 raft_consensus.cc:3055] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b [term 1 LEADER]: Stepping down as leader of term 1
I20260812 08:03:19.631162 29024 raft_consensus.cc:740] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 43e48d5b5b544be4b8eb7b35f80aab4b, State: Running, Role: LEADER
I20260812 08:03:19.631335 29024 consensus_queue.cc:260] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.12, Last appended by leader: 12, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3db342ac1a34617858477795c47468e" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40187 } } peers { permanent_uuid: "43e48d5b5b544be4b8eb7b35f80aab4b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41601 } } peers { permanent_uuid: "bcb6e372d3b548a1b1bc95753ce79516" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 37781 } }
I20260812 08:03:19.631644 29024 raft_consensus.cc:3060] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:03:19.632863 29024 raft_consensus.cc:1275] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b [term 2 FOLLOWER]: Refusing update from remote peer f3db342ac1a34617858477795c47468e: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 13. (index mismatch)
I20260812 08:03:19.633870 29168 consensus_queue.cc:1048] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e [LEADER]: Connected to new peer: Peer: permanent_uuid: "43e48d5b5b544be4b8eb7b35f80aab4b" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:19.634029 29168 consensus_queue.cc:1243] T 03038146670b4cb48c4fd4d21e7d2620 P f3db342ac1a34617858477795c47468e [LEADER]: Peer 43e48d5b5b544be4b8eb7b35f80aab4b log is divergent from this leader: its last log entry 1.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260812 08:03:19.635048 29024 raft_consensus.cc:1217] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.5]   Dedup: 1.5->[]
I20260812 08:03:19.636729 29024 raft_consensus.cc:1217] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.9]   Dedup: 1.9->[]
I20260812 08:03:19.638430 29024 raft_consensus.cc:1217] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.9->[1.10-2.13]   Dedup: 1.11->[2.12-2.13]
I20260812 08:03:19.638513 29024 pending_rounds.cc:85] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b: Aborting all ops after (but not including) 11
I20260812 08:03:19.638572 29024 pending_rounds.cc:107] T 03038146670b4cb48c4fd4d21e7d2620 P 43e48d5b5b544be4b8eb7b35f80aab4b: Aborting uncommitted WRITE_OP operation due to leader change: 1.12
W20260812 08:03:19.638880 29024 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59434: Aborted: Op aborted by new leader
W20260812 08:03:19.639211 29024 rpcz_store.cc:267] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59434 (ReqId={client: 6f775316028047b1bb816481f36be510, seq_no=10, attempt_no=0}) took 1561 ms (client timeout 98 ms). Trace:
W20260812 08:03:19.639272 29024 rpcz_store.cc:269] 0812 08:03:18.077456 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:03:18.077587 (+   131us) service_pool.cc:224] Handling call
0812 08:03:19.639200 (+1561613us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:03:18.078090 (+     0us) write_op.cc:183] PREPARE: starting on tablet 03038146670b4cb48c4fd4d21e7d2620
0812 08:03:18.078183 (+    93us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:03:18.078187 (+     4us) write_op.cc:435] Acquired schema lock
0812 08:03:18.078188 (+     1us) tablet.cc:662] Decoding operations
0812 08:03:18.078244 (+    56us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:03:18.078252 (+     8us) write_op.cc:641] Partition lock acquired for write op
0812 08:03:18.078254 (+     2us) tablet.cc:685] Acquiring locks for 1 operations
0812 08:03:18.078287 (+    33us) tablet.cc:701] Row locks acquired
0812 08:03:18.078288 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:03:18.078324 (+    36us) write_op.cc:270] Start()
0812 08:03:18.078351 (+    27us) write_op.cc:276] Timestamp: P: 1786521798078287 usec, L: 0
0812 08:03:18.078352 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:03:18.078501 (+   149us) log.cc:844] Serialized 10331 byte log entry
0812 08:03:19.638767 (+1560266us) op_driver.cc:464] REPLICATION: finished
0812 08:03:19.638783 (+    16us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 08:03:19.638790 (+     7us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:03:19.638851 (+    61us) write_op.cc:454] Released schema lock
0812 08:03:19.638860 (+     9us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":1,"prepare.queue_time_us":74,"prepare.run_cpu_time_us":499,"prepare.run_wall_time_us":500,"raft.queue_time_us":121,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":189,"replication_time_us":1560285}]]}
I20260812 08:03:19.732719 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 28452
I20260812 08:03:19.751734 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 28896
I20260812 08:03:19.770105 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 28295
2026-08-12T08:03:19Z chronyd exiting
[       OK ] RaftConsensusITest.TestCommitIndexFarBehindAfterLeaderElection (3678 ms)
[ RUN      ] RaftConsensusITest.TestUpdateConsensusErrorNonePrepared
I20260812 08:03:19.802654 18601 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:03:19.802704 18601 ts_itest-base.cc:116] --------------
I20260812 08:03:19.802717 18601 ts_itest-base.cc:117] 3 tablet servers
I20260812 08:03:19.802731 18601 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:03:19.802778 18601 ts_itest-base.cc:119] --------------
2026-08-12T08:03:19Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:03:19Z Disabled control of system clock
I20260812 08:03:19.812809 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.42.126:45273
--webserver_interface=127.18.42.126
--webserver_port=0
--builtin_ntp_servers=127.18.42.84:42457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.42.126:45273
--catalog_manager_wait_for_new_tablets_to_elect_leader=false with env {}
W20260812 08:03:19.924098 29230 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:19.924386 29230 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:19.924463 29230 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:19.927903 29230 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:03:19.927991 29230 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:19.928030 29230 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:03:19.928072 29230 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:03:19.932065 29230 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:42457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
--catalog_manager_wait_for_new_tablets_to_elect_leader=false
--ipki_ca_key_size=768
--master_addresses=127.18.42.126:45273
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.42.126:45273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.18.42.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.29230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:19.933117 29230 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:19.934305 29230 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:19.939797 29240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:19.939983 29230 server_base.cc:1061] running on GCE node
W20260812 08:03:19.939796 29239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:19.939867 29243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:19.940573 29230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:19.941496 29230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:19.942754 29230 hybrid_clock.cc:648] HybridClock initialized: now 1786521799942739 us; error 36 us; skew 500 ppm
I20260812 08:03:19.944584 29230 webserver.cc:533] Webserver started at http://127.18.42.126:38209/ using document root <none> and password file <none>
I20260812 08:03:19.945176 29230 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:19.945264 29230 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:19.945499 29230 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:19.947105 29230 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "df1fe133c74c4cdfa79f715f4f96853b"
format_stamp: "Formatted at 2026-08-12 08:03:19 on dist-test-slave-zr1t"
I20260812 08:03:19.947629 29230 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "df1fe133c74c4cdfa79f715f4f96853b"
format_stamp: "Formatted at 2026-08-12 08:03:19 on dist-test-slave-zr1t"
I20260812 08:03:19.950995 29230 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.001s	sys 0.004s
I20260812 08:03:19.953194 29252 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:19.954332 29230 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 08:03:19.954520 29230 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
uuid: "df1fe133c74c4cdfa79f715f4f96853b"
format_stamp: "Formatted at 2026-08-12 08:03:19 on dist-test-slave-zr1t"
I20260812 08:03:19.954631 29230 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:19.967804 29230 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:19.968501 29230 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:19.968741 29230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:19.976500 29230 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.126:45273
I20260812 08:03:19.976519 29330 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.126:45273 every 8 connection(s)
I20260812 08:03:19.977594 29230 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:03:19.979060 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 29230
I20260812 08:03:19.979161 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:03:19.980787 29332 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:19.985754 29332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df1fe133c74c4cdfa79f715f4f96853b: Bootstrap starting.
I20260812 08:03:19.988139 29332 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df1fe133c74c4cdfa79f715f4f96853b: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:19.988961 29332 log.cc:826] T 00000000000000000000000000000000 P df1fe133c74c4cdfa79f715f4f96853b: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:19.990542 29332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df1fe133c74c4cdfa79f715f4f96853b: No bootstrap required, opened a new log
I20260812 08:03:19.993223 29332 raft_consensus.cc:359] T 00000000000000000000000000000000 P df1fe133c74c4cdfa79f715f4f96853b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1fe133c74c4cdfa79f715f4f96853b" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 45273 } }
I20260812 08:03:19.993392 29332 raft_consensus.cc:385] T 00000000000000000000000000000000 P df1fe133c74c4cdfa79f715f4f96853b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:19.993435 29332 raft_consensus.cc:740] T 00000000000000000000000000000000 P df1fe133c74c4cdfa79f715f4f96853b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df1fe133c74c4cdfa79f715f4f96853b, State: Initialized, Role: FOLLOWER
I20260812 08:03:19.993836 29332 consensus_queue.cc:260] T 00000000000000000000000000000000 P df1fe133c74c4cdfa79f715f4f96853b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1fe133c74c4cdfa79f715f4f96853b" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 45273 } }
I20260812 08:03:19.993949 29332 raft_consensus.cc:399] T 00000000000000000000000000000000 P df1fe133c74c4cdfa79f715f4f96853b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:03:19.993991 29332 raft_consensus.cc:493] T 00000000000000000000000000000000 P df1fe133c74c4cdfa79f715f4f96853b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:03:19.994071 29332 raft_consensus.cc:3060] T 00000000000000000000000000000000 P df1fe133c74c4cdfa79f715f4f96853b [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:19.994801 29332 raft_consensus.cc:515] T 00000000000000000000000000000000 P df1fe133c74c4cdfa79f715f4f96853b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1fe133c74c4cdfa79f715f4f96853b" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 45273 } }
I20260812 08:03:19.995080 29332 leader_election.cc:304] T 00000000000000000000000000000000 P df1fe133c74c4cdfa79f715f4f96853b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df1fe133c74c4cdfa79f715f4f96853b; no voters: 
I20260812 08:03:19.995308 29332 leader_election.cc:290] T 00000000000000000000000000000000 P df1fe133c74c4cdfa79f715f4f96853b [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:03:19.995477 29344 raft_consensus.cc:2804] T 00000000000000000000000000000000 P df1fe133c74c4cdfa79f715f4f96853b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:19.995712 29344 raft_consensus.cc:697] T 00000000000000000000000000000000 P df1fe133c74c4cdfa79f715f4f96853b [term 1 LEADER]: Becoming Leader. State: Replica: df1fe133c74c4cdfa79f715f4f96853b, State: Running, Role: LEADER
I20260812 08:03:19.996031 29344 consensus_queue.cc:237] T 00000000000000000000000000000000 P df1fe133c74c4cdfa79f715f4f96853b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1fe133c74c4cdfa79f715f4f96853b" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 45273 } }
I20260812 08:03:19.996444 29332 sys_catalog.cc:565] T 00000000000000000000000000000000 P df1fe133c74c4cdfa79f715f4f96853b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:19.997753 29346 sys_catalog.cc:455] T 00000000000000000000000000000000 P df1fe133c74c4cdfa79f715f4f96853b [sys.catalog]: SysCatalogTable state changed. Reason: New leader df1fe133c74c4cdfa79f715f4f96853b. Latest consensus state: current_term: 1 leader_uuid: "df1fe133c74c4cdfa79f715f4f96853b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1fe133c74c4cdfa79f715f4f96853b" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 45273 } } }
I20260812 08:03:19.997773 29345 sys_catalog.cc:455] T 00000000000000000000000000000000 P df1fe133c74c4cdfa79f715f4f96853b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df1fe133c74c4cdfa79f715f4f96853b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1fe133c74c4cdfa79f715f4f96853b" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 45273 } } }
I20260812 08:03:19.997911 29345 sys_catalog.cc:458] T 00000000000000000000000000000000 P df1fe133c74c4cdfa79f715f4f96853b [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:19.997913 29346 sys_catalog.cc:458] T 00000000000000000000000000000000 P df1fe133c74c4cdfa79f715f4f96853b [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:19.999151 29357 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:03:20.001780 29357 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:03:20.006759 29357 catalog_manager.cc:1383] Generated new cluster ID: 13d0084b7f5f4d9991d6388c18d3cb5c
I20260812 08:03:20.006831 29357 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:03:20.035501 29357 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:03:20.036389 29357 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:03:20.042860 29357 catalog_manager.cc:6092] T 00000000000000000000000000000000 P df1fe133c74c4cdfa79f715f4f96853b: Generated new TSK 0
I20260812 08:03:20.043496 29357 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:03:20.054071 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.65:0
--local_ip_for_outbound_sockets=127.18.42.65
--webserver_interface=127.18.42.65
--webserver_port=0
--tserver_master_addrs=127.18.42.126:45273
--builtin_ntp_servers=127.18.42.84:42457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false with env {}
W20260812 08:03:20.161840 29372 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 08:03:20.162051 29372 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:20.162073 29372 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:20.162155 29372 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:20.165452 29372 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:20.165642 29372 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.65
I20260812 08:03:20.169339 29372 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:42457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.18.42.65
--webserver_port=0
--tserver_master_addrs=127.18.42.126:45273
--never_fsync=true
--heap_profile_path=/tmp/kudu.29372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:20.170440 29372 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:20.171525 29372 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:20.177686 29385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:20.177700 29384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:20.177719 29389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:20.177999 29372 server_base.cc:1061] running on GCE node
I20260812 08:03:20.178435 29372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:20.178982 29372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:20.180171 29372 hybrid_clock.cc:648] HybridClock initialized: now 1786521800180151 us; error 33 us; skew 500 ppm
I20260812 08:03:20.181994 29372 webserver.cc:533] Webserver started at http://127.18.42.65:43677/ using document root <none> and password file <none>
I20260812 08:03:20.182564 29372 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:20.182622 29372 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:20.182824 29372 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:20.184340 29372 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "ac0370544d054247b7071d9feebe3868"
format_stamp: "Formatted at 2026-08-12 08:03:20 on dist-test-slave-zr1t"
I20260812 08:03:20.184825 29372 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "ac0370544d054247b7071d9feebe3868"
format_stamp: "Formatted at 2026-08-12 08:03:20 on dist-test-slave-zr1t"
I20260812 08:03:20.188206 29372 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:03:20.190357 29396 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:20.191314 29372 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260812 08:03:20.191447 29372 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "ac0370544d054247b7071d9feebe3868"
format_stamp: "Formatted at 2026-08-12 08:03:20 on dist-test-slave-zr1t"
I20260812 08:03:20.191550 29372 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:20.201283 29372 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:20.201928 29372 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:20.202116 29372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:20.202731 29372 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:20.203685 29372 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:20.203734 29372 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:20.203771 29372 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:20.203824 29372 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:20.213471 29372 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.65:33935
I20260812 08:03:20.213506 29586 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.65:33935 every 8 connection(s)
I20260812 08:03:20.214527 29372 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:03:20.221052 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 29372
I20260812 08:03:20.221146 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:03:20.223460 29588 heartbeater.cc:344] Connected to a master server at 127.18.42.126:45273
I20260812 08:03:20.223660 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.66:0
--local_ip_for_outbound_sockets=127.18.42.66
--webserver_interface=127.18.42.66
--webserver_port=0
--tserver_master_addrs=127.18.42.126:45273
--builtin_ntp_servers=127.18.42.84:42457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false with env {}
I20260812 08:03:20.223711 29588 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:20.224251 29588 heartbeater.cc:507] Master 127.18.42.126:45273 requested a full tablet report, sending...
I20260812 08:03:20.225863 29275 ts_manager.cc:194] Registered new tserver with Master: ac0370544d054247b7071d9feebe3868 (127.18.42.65:33935)
I20260812 08:03:20.227010 29275 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.65:55739
W20260812 08:03:20.331246 29593 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 08:03:20.331521 29593 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:20.331591 29593 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:20.331671 29593 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:20.335059 29593 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:20.335220 29593 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.66
I20260812 08:03:20.338987 29593 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:42457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.18.42.66
--webserver_port=0
--tserver_master_addrs=127.18.42.126:45273
--never_fsync=true
--heap_profile_path=/tmp/kudu.29593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:20.340164 29593 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:20.341305 29593 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:20.347626 29600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:20.347661 29598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:20.347661 29602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:20.348007 29593 server_base.cc:1061] running on GCE node
I20260812 08:03:20.348500 29593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:20.349035 29593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:20.350215 29593 hybrid_clock.cc:648] HybridClock initialized: now 1786521800350203 us; error 33 us; skew 500 ppm
I20260812 08:03:20.352023 29593 webserver.cc:533] Webserver started at http://127.18.42.66:42919/ using document root <none> and password file <none>
I20260812 08:03:20.352624 29593 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:20.352718 29593 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:20.352949 29593 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:20.354584 29593 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "9aa5c3ff7e274b52ab4f4fb060488b60"
format_stamp: "Formatted at 2026-08-12 08:03:20 on dist-test-slave-zr1t"
I20260812 08:03:20.355086 29593 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "9aa5c3ff7e274b52ab4f4fb060488b60"
format_stamp: "Formatted at 2026-08-12 08:03:20 on dist-test-slave-zr1t"
I20260812 08:03:20.358527 29593 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:03:20.360888 29611 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:20.361903 29593 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 08:03:20.362040 29593 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "9aa5c3ff7e274b52ab4f4fb060488b60"
format_stamp: "Formatted at 2026-08-12 08:03:20 on dist-test-slave-zr1t"
I20260812 08:03:20.362145 29593 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:20.374089 29593 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:20.374679 29593 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:20.374892 29593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:20.375438 29593 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:20.376492 29593 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:20.376554 29593 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:20.376605 29593 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:20.376633 29593 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:20.386204 29593 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.66:35493
I20260812 08:03:20.386231 29794 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.66:35493 every 8 connection(s)
I20260812 08:03:20.387123 29593 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:03:20.390655 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 29593
I20260812 08:03:20.390740 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:03:20.392920 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.67:0
--local_ip_for_outbound_sockets=127.18.42.67
--webserver_interface=127.18.42.67
--webserver_port=0
--tserver_master_addrs=127.18.42.126:45273
--builtin_ntp_servers=127.18.42.84:42457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false with env {}
I20260812 08:03:20.396457 29795 heartbeater.cc:344] Connected to a master server at 127.18.42.126:45273
I20260812 08:03:20.396744 29795 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:20.397321 29795 heartbeater.cc:507] Master 127.18.42.126:45273 requested a full tablet report, sending...
I20260812 08:03:20.398523 29275 ts_manager.cc:194] Registered new tserver with Master: 9aa5c3ff7e274b52ab4f4fb060488b60 (127.18.42.66:35493)
I20260812 08:03:20.399356 29275 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.66:60665
W20260812 08:03:20.500659 29802 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 08:03:20.500870 29802 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:20.500892 29802 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:20.500932 29802 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:20.504321 29802 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:20.504429 29802 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.67
I20260812 08:03:20.508096 29802 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:42457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.18.42.67
--webserver_port=0
--tserver_master_addrs=127.18.42.126:45273
--never_fsync=true
--heap_profile_path=/tmp/kudu.29802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:20.509330 29802 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:20.510517 29802 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:20.516820 29810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:20.516820 29813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:20.516921 29809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:20.517107 29802 server_base.cc:1061] running on GCE node
I20260812 08:03:20.517458 29802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:20.518009 29802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:20.519192 29802 hybrid_clock.cc:648] HybridClock initialized: now 1786521800519179 us; error 34 us; skew 500 ppm
I20260812 08:03:20.520939 29802 webserver.cc:533] Webserver started at http://127.18.42.67:35883/ using document root <none> and password file <none>
I20260812 08:03:20.521510 29802 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:20.521647 29802 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:20.521865 29802 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:20.523394 29802 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "6a00b56218ad4f139a1f65f084f653bd"
format_stamp: "Formatted at 2026-08-12 08:03:20 on dist-test-slave-zr1t"
I20260812 08:03:20.523875 29802 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "6a00b56218ad4f139a1f65f084f653bd"
format_stamp: "Formatted at 2026-08-12 08:03:20 on dist-test-slave-zr1t"
I20260812 08:03:20.527223 29802 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:03:20.529364 29822 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:20.530475 29802 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:20.530629 29802 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "6a00b56218ad4f139a1f65f084f653bd"
format_stamp: "Formatted at 2026-08-12 08:03:20 on dist-test-slave-zr1t"
I20260812 08:03:20.530745 29802 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:20.541615 29802 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:20.542284 29802 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:20.542474 29802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:20.543084 29802 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:20.544083 29802 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:20.544158 29802 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:20.544222 29802 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:20.544275 29802 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:20.554087 29802 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.67:36357
I20260812 08:03:20.554126 30002 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.67:36357 every 8 connection(s)
I20260812 08:03:20.555037 29802 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:03:20.560343 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 29802
I20260812 08:03:20.560428 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestUpdateConsensusErrorNonePrepared.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:03:20.563884 30005 heartbeater.cc:344] Connected to a master server at 127.18.42.126:45273
I20260812 08:03:20.564133 30005 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:20.564615 30005 heartbeater.cc:507] Master 127.18.42.126:45273 requested a full tablet report, sending...
I20260812 08:03:20.565707 29275 ts_manager.cc:194] Registered new tserver with Master: 6a00b56218ad4f139a1f65f084f653bd (127.18.42.67:36357)
I20260812 08:03:20.566298 29275 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.67:44189
I20260812 08:03:20.574813 18601 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:03:20.583712 29275 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44596:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
W20260812 08:03:20.586237 29275 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 08:03:20.603029 29473 tablet_service.cc:1511] Processing CreateTablet for tablet 207b4bd448af427d87219e16bb80b7ff (DEFAULT_TABLE table=TestTable [id=bb22f214060c4092a630e6a8f58d5a29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:20.603503 29682 tablet_service.cc:1511] Processing CreateTablet for tablet 207b4bd448af427d87219e16bb80b7ff (DEFAULT_TABLE table=TestTable [id=bb22f214060c4092a630e6a8f58d5a29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:20.604075 29473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 207b4bd448af427d87219e16bb80b7ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:20.603907 29895 tablet_service.cc:1511] Processing CreateTablet for tablet 207b4bd448af427d87219e16bb80b7ff (DEFAULT_TABLE table=TestTable [id=bb22f214060c4092a630e6a8f58d5a29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:20.604436 29682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 207b4bd448af427d87219e16bb80b7ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:20.604745 29895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 207b4bd448af427d87219e16bb80b7ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:20.609014 30033 tablet_bootstrap.cc:492] T 207b4bd448af427d87219e16bb80b7ff P ac0370544d054247b7071d9feebe3868: Bootstrap starting.
I20260812 08:03:20.610191 30036 tablet_bootstrap.cc:492] T 207b4bd448af427d87219e16bb80b7ff P 9aa5c3ff7e274b52ab4f4fb060488b60: Bootstrap starting.
I20260812 08:03:20.610816 30034 tablet_bootstrap.cc:492] T 207b4bd448af427d87219e16bb80b7ff P 6a00b56218ad4f139a1f65f084f653bd: Bootstrap starting.
I20260812 08:03:20.612082 30033 tablet_bootstrap.cc:654] T 207b4bd448af427d87219e16bb80b7ff P ac0370544d054247b7071d9feebe3868: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:20.612191 30036 tablet_bootstrap.cc:654] T 207b4bd448af427d87219e16bb80b7ff P 9aa5c3ff7e274b52ab4f4fb060488b60: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:20.612571 30034 tablet_bootstrap.cc:654] T 207b4bd448af427d87219e16bb80b7ff P 6a00b56218ad4f139a1f65f084f653bd: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:20.613006 30033 log.cc:826] T 207b4bd448af427d87219e16bb80b7ff P ac0370544d054247b7071d9feebe3868: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:20.613006 30036 log.cc:826] T 207b4bd448af427d87219e16bb80b7ff P 9aa5c3ff7e274b52ab4f4fb060488b60: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:20.613296 30034 log.cc:826] T 207b4bd448af427d87219e16bb80b7ff P 6a00b56218ad4f139a1f65f084f653bd: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:20.614657 30033 tablet_bootstrap.cc:492] T 207b4bd448af427d87219e16bb80b7ff P ac0370544d054247b7071d9feebe3868: No bootstrap required, opened a new log
I20260812 08:03:20.614660 30036 tablet_bootstrap.cc:492] T 207b4bd448af427d87219e16bb80b7ff P 9aa5c3ff7e274b52ab4f4fb060488b60: No bootstrap required, opened a new log
I20260812 08:03:20.614869 30033 ts_tablet_manager.cc:1403] T 207b4bd448af427d87219e16bb80b7ff P ac0370544d054247b7071d9feebe3868: Time spent bootstrapping tablet: real 0.006s	user 0.004s	sys 0.000s
I20260812 08:03:20.614889 30036 ts_tablet_manager.cc:1403] T 207b4bd448af427d87219e16bb80b7ff P 9aa5c3ff7e274b52ab4f4fb060488b60: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:03:20.616039 30034 tablet_bootstrap.cc:492] T 207b4bd448af427d87219e16bb80b7ff P 6a00b56218ad4f139a1f65f084f653bd: No bootstrap required, opened a new log
I20260812 08:03:20.616267 30034 ts_tablet_manager.cc:1403] T 207b4bd448af427d87219e16bb80b7ff P 6a00b56218ad4f139a1f65f084f653bd: Time spent bootstrapping tablet: real 0.006s	user 0.004s	sys 0.000s
I20260812 08:03:20.617584 30036 raft_consensus.cc:359] T 207b4bd448af427d87219e16bb80b7ff P 9aa5c3ff7e274b52ab4f4fb060488b60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0370544d054247b7071d9feebe3868" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 33935 } } peers { permanent_uuid: "9aa5c3ff7e274b52ab4f4fb060488b60" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 35493 } } peers { permanent_uuid: "6a00b56218ad4f139a1f65f084f653bd" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 36357 } }
I20260812 08:03:20.617799 30036 raft_consensus.cc:740] T 207b4bd448af427d87219e16bb80b7ff P 9aa5c3ff7e274b52ab4f4fb060488b60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa5c3ff7e274b52ab4f4fb060488b60, State: Initialized, Role: FOLLOWER
I20260812 08:03:20.618026 30033 raft_consensus.cc:359] T 207b4bd448af427d87219e16bb80b7ff P ac0370544d054247b7071d9feebe3868 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0370544d054247b7071d9feebe3868" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 33935 } } peers { permanent_uuid: "9aa5c3ff7e274b52ab4f4fb060488b60" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 35493 } } peers { permanent_uuid: "6a00b56218ad4f139a1f65f084f653bd" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 36357 } }
I20260812 08:03:20.618120 30036 consensus_queue.cc:260] T 207b4bd448af427d87219e16bb80b7ff P 9aa5c3ff7e274b52ab4f4fb060488b60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0370544d054247b7071d9feebe3868" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 33935 } } peers { permanent_uuid: "9aa5c3ff7e274b52ab4f4fb060488b60" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 35493 } } peers { permanent_uuid: "6a00b56218ad4f139a1f65f084f653bd" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 36357 } }
I20260812 08:03:20.618227 30033 raft_consensus.cc:740] T 207b4bd448af427d87219e16bb80b7ff P ac0370544d054247b7071d9feebe3868 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac0370544d054247b7071d9feebe3868, State: Initialized, Role: FOLLOWER
I20260812 08:03:20.618556 30033 consensus_queue.cc:260] T 207b4bd448af427d87219e16bb80b7ff P ac0370544d054247b7071d9feebe3868 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0370544d054247b7071d9feebe3868" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 33935 } } peers { permanent_uuid: "9aa5c3ff7e274b52ab4f4fb060488b60" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 35493 } } peers { permanent_uuid: "6a00b56218ad4f139a1f65f084f653bd" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 36357 } }
I20260812 08:03:20.618759 29795 heartbeater.cc:499] Master 127.18.42.126:45273 was elected leader, sending a full tablet report...
I20260812 08:03:20.618983 30036 ts_tablet_manager.cc:1434] T 207b4bd448af427d87219e16bb80b7ff P 9aa5c3ff7e274b52ab4f4fb060488b60: Time spent starting tablet: real 0.004s	user 0.000s	sys 0.004s
I20260812 08:03:20.619279 30034 raft_consensus.cc:359] T 207b4bd448af427d87219e16bb80b7ff P 6a00b56218ad4f139a1f65f084f653bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0370544d054247b7071d9feebe3868" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 33935 } } peers { permanent_uuid: "9aa5c3ff7e274b52ab4f4fb060488b60" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 35493 } } peers { permanent_uuid: "6a00b56218ad4f139a1f65f084f653bd" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 36357 } }
I20260812 08:03:20.619434 30033 ts_tablet_manager.cc:1434] T 207b4bd448af427d87219e16bb80b7ff P ac0370544d054247b7071d9feebe3868: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:03:20.619465 30034 raft_consensus.cc:740] T 207b4bd448af427d87219e16bb80b7ff P 6a00b56218ad4f139a1f65f084f653bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a00b56218ad4f139a1f65f084f653bd, State: Initialized, Role: FOLLOWER
I20260812 08:03:20.619496 29588 heartbeater.cc:499] Master 127.18.42.126:45273 was elected leader, sending a full tablet report...
I20260812 08:03:20.619717 30034 consensus_queue.cc:260] T 207b4bd448af427d87219e16bb80b7ff P 6a00b56218ad4f139a1f65f084f653bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0370544d054247b7071d9feebe3868" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 33935 } } peers { permanent_uuid: "9aa5c3ff7e274b52ab4f4fb060488b60" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 35493 } } peers { permanent_uuid: "6a00b56218ad4f139a1f65f084f653bd" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 36357 } }
I20260812 08:03:20.620301 30005 heartbeater.cc:499] Master 127.18.42.126:45273 was elected leader, sending a full tablet report...
I20260812 08:03:20.620498 30034 ts_tablet_manager.cc:1434] T 207b4bd448af427d87219e16bb80b7ff P 6a00b56218ad4f139a1f65f084f653bd: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260812 08:03:20.627727 18601 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:03:20.629201 18601 ts_itest-base.cc:246] Waiting for 1 tablets on tserver ac0370544d054247b7071d9feebe3868 to finish bootstrapping
I20260812 08:03:20.635673 18601 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 9aa5c3ff7e274b52ab4f4fb060488b60 to finish bootstrapping
W20260812 08:03:20.638016 29796 tablet.cc:2406] T 207b4bd448af427d87219e16bb80b7ff P 9aa5c3ff7e274b52ab4f4fb060488b60: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:03:20.641330 18601 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 6a00b56218ad4f139a1f65f084f653bd to finish bootstrapping
I20260812 08:03:20.647188 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 29802
I20260812 08:03:20.669507 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 29372
W20260812 08:03:20.688043 29784 generic_service.cc:139] {username='slave'} at 127.0.0.1:41352 forcing change of non-runtime-safe flag follower_fail_all_prepare
I20260812 08:03:20.688172 29784 generic_service.cc:175] {username='slave'} at 127.0.0.1:41352 changed flags via RPC: follower_fail_all_prepare from 'false' to 'true'
W20260812 08:03:20.697080 29712 raft_consensus.cc:1579] T 207b4bd448af427d87219e16bb80b7ff P 9aa5c3ff7e274b52ab4f4fb060488b60 [term 0 FOLLOWER]: Could not prepare op 'term: 0 index: 1' and following 9 ops. Status for this op: Illegal state: Rejected: --follower_fail_all_prepare is set to true.
I20260812 08:03:20.697398 29712 raft_consensus.cc:1595] T 207b4bd448af427d87219e16bb80b7ff P 9aa5c3ff7e274b52ab4f4fb060488b60 [term 0 FOLLOWER]: Rejecting Update request from peer ac0370544d054247b7071d9feebe3868 for term 0. Could not prepare a single op due to: Illegal state: Rejected: --follower_fail_all_prepare is set to true.
I20260812 08:03:20.698935 29272 catalog_manager.cc:5719] T 207b4bd448af427d87219e16bb80b7ff P 9aa5c3ff7e274b52ab4f4fb060488b60 reported cstate change: leader changed from <none> to ac0370544d054247b7071d9feebe3868 (127.18.42.65). New cstate: current_term: 0 leader_uuid: "ac0370544d054247b7071d9feebe3868" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0370544d054247b7071d9feebe3868" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 33935 } } peers { permanent_uuid: "9aa5c3ff7e274b52ab4f4fb060488b60" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 35493 } } peers { permanent_uuid: "6a00b56218ad4f139a1f65f084f653bd" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 36357 } } }
I20260812 08:03:20.698285 18601 raft_consensus-itest.cc:2696] responder_uuid: "9aa5c3ff7e274b52ab4f4fb060488b60" responder_term: 0 status { last_received { term: 0 index: 0 } last_committed_idx: 0 error { code: CANNOT_PREPARE status { code: ILLEGAL_STATE message: "Rejecting Update request from peer ac0370544d054247b7071d9feebe3868 for term 0. Could not prepare a single op due to: Illegal state: Rejected: --follower_fail_all_prepare is set to true." } } last_received_current_leader { term: 0 index: 0 } }
I20260812 08:03:20.699738 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 29593
W20260812 08:03:20.718937 29256 connection.cc:573] server connection from 127.18.42.66:60665 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 08:03:20.719394 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 29230
2026-08-12T08:03:20Z chronyd exiting
[       OK ] RaftConsensusITest.TestUpdateConsensusErrorNonePrepared (948 ms)
[ RUN      ] RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC
I20260812 08:03:20.751155 18601 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:03:20.751223 18601 ts_itest-base.cc:116] --------------
I20260812 08:03:20.751242 18601 ts_itest-base.cc:117] 3 tablet servers
I20260812 08:03:20.751259 18601 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:03:20.751286 18601 ts_itest-base.cc:119] --------------
2026-08-12T08:03:20Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:03:20Z Disabled control of system clock
I20260812 08:03:20.761040 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.42.126:45411
--webserver_interface=127.18.42.126
--webserver_port=0
--builtin_ntp_servers=127.18.42.84:35393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.42.126:45411
--catalog_manager_evict_excess_replicas=false with env {}
W20260812 08:03:20.866117 30064 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:20.866379 30064 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:20.866453 30064 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:20.869771 30064 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:03:20.869851 30064 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:20.869887 30064 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:03:20.869926 30064 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:03:20.873695 30064 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:35393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
--catalog_manager_evict_excess_replicas=false
--ipki_ca_key_size=768
--master_addresses=127.18.42.126:45411
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.42.126:45411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.18.42.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.30064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:20.874754 30064 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:20.875869 30064 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:20.881175 30073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:20.881177 30074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:20.881297 30077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:20.881577 30064 server_base.cc:1061] running on GCE node
I20260812 08:03:20.882164 30064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:20.883070 30064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:20.884238 30064 hybrid_clock.cc:648] HybridClock initialized: now 1786521800884206 us; error 41 us; skew 500 ppm
I20260812 08:03:20.886113 30064 webserver.cc:533] Webserver started at http://127.18.42.126:44475/ using document root <none> and password file <none>
I20260812 08:03:20.886700 30064 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:20.886756 30064 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:20.886998 30064 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:20.888491 30064 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "0e3ac934adbd4e08807872d3eadbf56b"
format_stamp: "Formatted at 2026-08-12 08:03:20 on dist-test-slave-zr1t"
I20260812 08:03:20.888995 30064 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "0e3ac934adbd4e08807872d3eadbf56b"
format_stamp: "Formatted at 2026-08-12 08:03:20 on dist-test-slave-zr1t"
I20260812 08:03:20.892241 30064 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:03:20.894455 30089 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:20.895450 30064 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:20.895570 30064 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
uuid: "0e3ac934adbd4e08807872d3eadbf56b"
format_stamp: "Formatted at 2026-08-12 08:03:20 on dist-test-slave-zr1t"
I20260812 08:03:20.895669 30064 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:20.924460 30064 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:20.925316 30064 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:20.925557 30064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:20.933207 30182 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.126:45411 every 8 connection(s)
I20260812 08:03:20.933213 30064 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.126:45411
I20260812 08:03:20.934358 30064 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:03:20.936937 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 30064
I20260812 08:03:20.937069 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:03:20.937270 30184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:20.943022 30184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e3ac934adbd4e08807872d3eadbf56b: Bootstrap starting.
I20260812 08:03:20.945390 30184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e3ac934adbd4e08807872d3eadbf56b: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:20.946146 30184 log.cc:826] T 00000000000000000000000000000000 P 0e3ac934adbd4e08807872d3eadbf56b: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:20.947726 30184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e3ac934adbd4e08807872d3eadbf56b: No bootstrap required, opened a new log
I20260812 08:03:20.950204 30184 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0e3ac934adbd4e08807872d3eadbf56b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3ac934adbd4e08807872d3eadbf56b" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 45411 } }
I20260812 08:03:20.950428 30184 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0e3ac934adbd4e08807872d3eadbf56b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:20.950512 30184 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0e3ac934adbd4e08807872d3eadbf56b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e3ac934adbd4e08807872d3eadbf56b, State: Initialized, Role: FOLLOWER
I20260812 08:03:20.950968 30184 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e3ac934adbd4e08807872d3eadbf56b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3ac934adbd4e08807872d3eadbf56b" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 45411 } }
I20260812 08:03:20.951115 30184 raft_consensus.cc:399] T 00000000000000000000000000000000 P 0e3ac934adbd4e08807872d3eadbf56b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:03:20.951200 30184 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0e3ac934adbd4e08807872d3eadbf56b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:03:20.951303 30184 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0e3ac934adbd4e08807872d3eadbf56b [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:20.952139 30184 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0e3ac934adbd4e08807872d3eadbf56b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3ac934adbd4e08807872d3eadbf56b" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 45411 } }
I20260812 08:03:20.952483 30184 leader_election.cc:304] T 00000000000000000000000000000000 P 0e3ac934adbd4e08807872d3eadbf56b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e3ac934adbd4e08807872d3eadbf56b; no voters: 
I20260812 08:03:20.952797 30184 leader_election.cc:290] T 00000000000000000000000000000000 P 0e3ac934adbd4e08807872d3eadbf56b [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:03:20.952965 30190 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0e3ac934adbd4e08807872d3eadbf56b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:20.953199 30190 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0e3ac934adbd4e08807872d3eadbf56b [term 1 LEADER]: Becoming Leader. State: Replica: 0e3ac934adbd4e08807872d3eadbf56b, State: Running, Role: LEADER
I20260812 08:03:20.953483 30190 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e3ac934adbd4e08807872d3eadbf56b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3ac934adbd4e08807872d3eadbf56b" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 45411 } }
I20260812 08:03:20.953969 30184 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0e3ac934adbd4e08807872d3eadbf56b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:20.955184 30192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e3ac934adbd4e08807872d3eadbf56b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e3ac934adbd4e08807872d3eadbf56b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3ac934adbd4e08807872d3eadbf56b" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 45411 } } }
I20260812 08:03:20.955386 30192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e3ac934adbd4e08807872d3eadbf56b [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:20.955761 30193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e3ac934adbd4e08807872d3eadbf56b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e3ac934adbd4e08807872d3eadbf56b. Latest consensus state: current_term: 1 leader_uuid: "0e3ac934adbd4e08807872d3eadbf56b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3ac934adbd4e08807872d3eadbf56b" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 45411 } } }
I20260812 08:03:20.955899 30193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e3ac934adbd4e08807872d3eadbf56b [sys.catalog]: This master's current role is: LEADER
W20260812 08:03:20.958715 30217 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 0e3ac934adbd4e08807872d3eadbf56b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 08:03:20.958781 30217 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 08:03:20.958861 30206 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:03:20.959622 30206 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:03:20.964669 30206 catalog_manager.cc:1383] Generated new cluster ID: 0291433b86fd478eb00db095f501dd0a
I20260812 08:03:20.964753 30206 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:03:20.981074 30206 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:03:20.982124 30206 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:03:21.007282 30206 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 0e3ac934adbd4e08807872d3eadbf56b: Generated new TSK 0
I20260812 08:03:21.008035 30206 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:03:21.022286 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.65:0
--local_ip_for_outbound_sockets=127.18.42.65
--webserver_interface=127.18.42.65
--webserver_port=0
--tserver_master_addrs=127.18.42.126:45411
--builtin_ntp_servers=127.18.42.84:35393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--evict_failed_followers=false
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20260812 08:03:21.133687 30221 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:03:21.133999 30221 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:21.134056 30221 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:21.134137 30221 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:21.137403 30221 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:03:21.137486 30221 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:03:21.137562 30221 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:21.137676 30221 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:03:21.137729 30221 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.65
I20260812 08:03:21.141605 30221 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:35393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--evict_failed_followers=false
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.18.42.65
--webserver_port=0
--tserver_master_addrs=127.18.42.126:45411
--never_fsync=true
--heap_profile_path=/tmp/kudu.30221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:21.142705 30221 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:21.143817 30221 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:21.149978 30235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:21.149987 30236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:21.149978 30239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:21.150310 30221 server_base.cc:1061] running on GCE node
I20260812 08:03:21.150720 30221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:21.151229 30221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:21.152387 30221 hybrid_clock.cc:648] HybridClock initialized: now 1786521801152370 us; error 30 us; skew 500 ppm
I20260812 08:03:21.154296 30221 webserver.cc:533] Webserver started at http://127.18.42.65:34573/ using document root <none> and password file <none>
I20260812 08:03:21.154974 30221 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:21.155043 30221 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:21.155339 30221 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:21.157004 30221 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "5df61636ab3d4017a6832b15387e505c"
format_stamp: "Formatted at 2026-08-12 08:03:21 on dist-test-slave-zr1t"
I20260812 08:03:21.157581 30221 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "5df61636ab3d4017a6832b15387e505c"
format_stamp: "Formatted at 2026-08-12 08:03:21 on dist-test-slave-zr1t"
I20260812 08:03:21.161000 30221 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.000s
I20260812 08:03:21.163328 30250 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:21.164273 30221 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:21.164427 30221 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "5df61636ab3d4017a6832b15387e505c"
format_stamp: "Formatted at 2026-08-12 08:03:21 on dist-test-slave-zr1t"
I20260812 08:03:21.164526 30221 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:21.188513 30221 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:21.189369 30221 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:21.189684 30221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:21.190303 30221 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:21.191359 30221 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:21.191411 30221 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:21.191461 30221 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:21.191490 30221 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:21.201453 30221 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.65:40389
I20260812 08:03:21.201488 30451 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.65:40389 every 8 connection(s)
I20260812 08:03:21.202455 30221 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:03:21.209759 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 30221
I20260812 08:03:21.209916 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:03:21.212510 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.66:0
--local_ip_for_outbound_sockets=127.18.42.66
--webserver_interface=127.18.42.66
--webserver_port=0
--tserver_master_addrs=127.18.42.126:45411
--builtin_ntp_servers=127.18.42.84:35393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--evict_failed_followers=false
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20260812 08:03:21.214581 30452 heartbeater.cc:344] Connected to a master server at 127.18.42.126:45411
I20260812 08:03:21.214845 30452 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:21.215524 30452 heartbeater.cc:507] Master 127.18.42.126:45411 requested a full tablet report, sending...
I20260812 08:03:21.217253 30119 ts_manager.cc:194] Registered new tserver with Master: 5df61636ab3d4017a6832b15387e505c (127.18.42.65:40389)
I20260812 08:03:21.219017 30119 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.65:56285
W20260812 08:03:21.329423 30456 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:03:21.329669 30456 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:21.329720 30456 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:21.329826 30456 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:21.333048 30456 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:03:21.333097 30456 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:03:21.333134 30456 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:21.333277 30456 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:03:21.333312 30456 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.66
I20260812 08:03:21.337112 30456 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:35393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--evict_failed_followers=false
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.18.42.66
--webserver_port=0
--tserver_master_addrs=127.18.42.126:45411
--never_fsync=true
--heap_profile_path=/tmp/kudu.30456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:21.338124 30456 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:21.339193 30456 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:21.345304 30473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:21.345311 30472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:21.345311 30478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:21.345695 30456 server_base.cc:1061] running on GCE node
I20260812 08:03:21.346084 30456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:21.346657 30456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:21.347853 30456 hybrid_clock.cc:648] HybridClock initialized: now 1786521801347840 us; error 26 us; skew 500 ppm
I20260812 08:03:21.349697 30456 webserver.cc:533] Webserver started at http://127.18.42.66:40951/ using document root <none> and password file <none>
I20260812 08:03:21.350263 30456 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:21.350349 30456 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:21.350567 30456 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:21.352164 30456 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "ba1ebe016f0742348997dbde07ec3c0f"
format_stamp: "Formatted at 2026-08-12 08:03:21 on dist-test-slave-zr1t"
I20260812 08:03:21.352663 30456 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "ba1ebe016f0742348997dbde07ec3c0f"
format_stamp: "Formatted at 2026-08-12 08:03:21 on dist-test-slave-zr1t"
I20260812 08:03:21.356254 30456 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.000s	sys 0.006s
I20260812 08:03:21.358477 30494 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:21.359474 30456 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.001s
I20260812 08:03:21.359606 30456 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "ba1ebe016f0742348997dbde07ec3c0f"
format_stamp: "Formatted at 2026-08-12 08:03:21 on dist-test-slave-zr1t"
I20260812 08:03:21.359706 30456 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:21.376672 30456 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:21.377317 30456 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:21.377561 30456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:21.378146 30456 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:21.379156 30456 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:21.379230 30456 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:21.379300 30456 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:21.379347 30456 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:21.388888 30456 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.66:45243
I20260812 08:03:21.388929 30686 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.66:45243 every 8 connection(s)
I20260812 08:03:21.390012 30456 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:03:21.397032 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 30456
I20260812 08:03:21.397130 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:03:21.399042 30688 heartbeater.cc:344] Connected to a master server at 127.18.42.126:45411
I20260812 08:03:21.399300 30688 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:21.399593 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.67:0
--local_ip_for_outbound_sockets=127.18.42.67
--webserver_interface=127.18.42.67
--webserver_port=0
--tserver_master_addrs=127.18.42.126:45411
--builtin_ntp_servers=127.18.42.84:35393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--evict_failed_followers=false
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20260812 08:03:21.399917 30688 heartbeater.cc:507] Master 127.18.42.126:45411 requested a full tablet report, sending...
I20260812 08:03:21.401165 30118 ts_manager.cc:194] Registered new tserver with Master: ba1ebe016f0742348997dbde07ec3c0f (127.18.42.66:45243)
I20260812 08:03:21.401824 30118 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.66:47281
W20260812 08:03:21.506006 30691 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:03:21.506212 30691 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:21.506259 30691 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:21.506356 30691 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:21.509608 30691 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:03:21.509656 30691 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:03:21.509717 30691 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:21.509830 30691 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:03:21.509871 30691 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.67
I20260812 08:03:21.513558 30691 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:35393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--evict_failed_followers=false
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.18.42.67
--webserver_port=0
--tserver_master_addrs=127.18.42.126:45411
--never_fsync=true
--heap_profile_path=/tmp/kudu.30691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:21.514515 30691 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:21.515643 30691 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:21.521796 30699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:21.521811 30698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:21.521809 30705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:21.522150 30691 server_base.cc:1061] running on GCE node
I20260812 08:03:21.522527 30691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:21.523079 30691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:21.524261 30691 hybrid_clock.cc:648] HybridClock initialized: now 1786521801524246 us; error 30 us; skew 500 ppm
I20260812 08:03:21.525959 30691 webserver.cc:533] Webserver started at http://127.18.42.67:39413/ using document root <none> and password file <none>
I20260812 08:03:21.526512 30691 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:21.526597 30691 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:21.526813 30691 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:21.528364 30691 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "630413e403d5457c82dc9025104ddd07"
format_stamp: "Formatted at 2026-08-12 08:03:21 on dist-test-slave-zr1t"
I20260812 08:03:21.528857 30691 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "630413e403d5457c82dc9025104ddd07"
format_stamp: "Formatted at 2026-08-12 08:03:21 on dist-test-slave-zr1t"
I20260812 08:03:21.532174 30691 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:03:21.536252 30714 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:21.537412 30691 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20260812 08:03:21.537621 30691 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "630413e403d5457c82dc9025104ddd07"
format_stamp: "Formatted at 2026-08-12 08:03:21 on dist-test-slave-zr1t"
I20260812 08:03:21.537737 30691 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:21.566242 30691 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:21.566931 30691 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:21.567131 30691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:21.567747 30691 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:21.568882 30691 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:21.568957 30691 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:21.569036 30691 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:21.569070 30691 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:21.579030 30691 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.67:41293
I20260812 08:03:21.579054 30917 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.67:41293 every 8 connection(s)
I20260812 08:03:21.579947 30691 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:03:21.586898 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 30691
I20260812 08:03:21.587020 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:03:21.588791 30919 heartbeater.cc:344] Connected to a master server at 127.18.42.126:45411
I20260812 08:03:21.589059 30919 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:21.589656 30919 heartbeater.cc:507] Master 127.18.42.126:45411 requested a full tablet report, sending...
I20260812 08:03:21.590754 30118 ts_manager.cc:194] Registered new tserver with Master: 630413e403d5457c82dc9025104ddd07 (127.18.42.67:41293)
I20260812 08:03:21.591302 30118 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.67:58915
I20260812 08:03:21.601773 18601 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:03:21.610574 30118 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48604:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
W20260812 08:03:21.612965 30118 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 08:03:21.629210 30569 tablet_service.cc:1511] Processing CreateTablet for tablet f7b78a0c7c054379bbc1732feaf8bac2 (DEFAULT_TABLE table=TestTable [id=8aa273786b3f4d208c0623f75d8e891a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:21.629212 30334 tablet_service.cc:1511] Processing CreateTablet for tablet f7b78a0c7c054379bbc1732feaf8bac2 (DEFAULT_TABLE table=TestTable [id=8aa273786b3f4d208c0623f75d8e891a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:21.629129 30804 tablet_service.cc:1511] Processing CreateTablet for tablet f7b78a0c7c054379bbc1732feaf8bac2 (DEFAULT_TABLE table=TestTable [id=8aa273786b3f4d208c0623f75d8e891a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:21.630195 30804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7b78a0c7c054379bbc1732feaf8bac2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:21.630192 30569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7b78a0c7c054379bbc1732feaf8bac2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:21.630192 30334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7b78a0c7c054379bbc1732feaf8bac2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:21.635012 30944 tablet_bootstrap.cc:492] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07: Bootstrap starting.
I20260812 08:03:21.635555 30946 tablet_bootstrap.cc:492] T f7b78a0c7c054379bbc1732feaf8bac2 P ba1ebe016f0742348997dbde07ec3c0f: Bootstrap starting.
I20260812 08:03:21.635972 30947 tablet_bootstrap.cc:492] T f7b78a0c7c054379bbc1732feaf8bac2 P 5df61636ab3d4017a6832b15387e505c: Bootstrap starting.
I20260812 08:03:21.638106 30947 tablet_bootstrap.cc:654] T f7b78a0c7c054379bbc1732feaf8bac2 P 5df61636ab3d4017a6832b15387e505c: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:21.638502 30944 tablet_bootstrap.cc:654] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:21.639073 30947 log.cc:826] T f7b78a0c7c054379bbc1732feaf8bac2 P 5df61636ab3d4017a6832b15387e505c: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:21.639357 30944 log.cc:826] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:21.639611 30946 tablet_bootstrap.cc:654] T f7b78a0c7c054379bbc1732feaf8bac2 P ba1ebe016f0742348997dbde07ec3c0f: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:21.640396 30946 log.cc:826] T f7b78a0c7c054379bbc1732feaf8bac2 P ba1ebe016f0742348997dbde07ec3c0f: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:21.640758 30947 tablet_bootstrap.cc:492] T f7b78a0c7c054379bbc1732feaf8bac2 P 5df61636ab3d4017a6832b15387e505c: No bootstrap required, opened a new log
I20260812 08:03:21.641011 30947 ts_tablet_manager.cc:1403] T f7b78a0c7c054379bbc1732feaf8bac2 P 5df61636ab3d4017a6832b15387e505c: Time spent bootstrapping tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 08:03:21.641029 30944 tablet_bootstrap.cc:492] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07: No bootstrap required, opened a new log
I20260812 08:03:21.641237 30944 ts_tablet_manager.cc:1403] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07: Time spent bootstrapping tablet: real 0.006s	user 0.005s	sys 0.000s
I20260812 08:03:21.642047 30946 tablet_bootstrap.cc:492] T f7b78a0c7c054379bbc1732feaf8bac2 P ba1ebe016f0742348997dbde07ec3c0f: No bootstrap required, opened a new log
I20260812 08:03:21.642225 30946 ts_tablet_manager.cc:1403] T f7b78a0c7c054379bbc1732feaf8bac2 P ba1ebe016f0742348997dbde07ec3c0f: Time spent bootstrapping tablet: real 0.007s	user 0.003s	sys 0.001s
I20260812 08:03:21.643774 30947 raft_consensus.cc:359] T f7b78a0c7c054379bbc1732feaf8bac2 P 5df61636ab3d4017a6832b15387e505c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "630413e403d5457c82dc9025104ddd07" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41293 } } peers { permanent_uuid: "5df61636ab3d4017a6832b15387e505c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40389 } } peers { permanent_uuid: "ba1ebe016f0742348997dbde07ec3c0f" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45243 } }
I20260812 08:03:21.644055 30947 raft_consensus.cc:385] T f7b78a0c7c054379bbc1732feaf8bac2 P 5df61636ab3d4017a6832b15387e505c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:21.644183 30947 raft_consensus.cc:740] T f7b78a0c7c054379bbc1732feaf8bac2 P 5df61636ab3d4017a6832b15387e505c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5df61636ab3d4017a6832b15387e505c, State: Initialized, Role: FOLLOWER
I20260812 08:03:21.644301 30944 raft_consensus.cc:359] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "630413e403d5457c82dc9025104ddd07" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41293 } } peers { permanent_uuid: "5df61636ab3d4017a6832b15387e505c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40389 } } peers { permanent_uuid: "ba1ebe016f0742348997dbde07ec3c0f" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45243 } }
I20260812 08:03:21.644552 30944 raft_consensus.cc:385] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:21.644621 30944 raft_consensus.cc:740] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 630413e403d5457c82dc9025104ddd07, State: Initialized, Role: FOLLOWER
I20260812 08:03:21.644765 30947 consensus_queue.cc:260] T f7b78a0c7c054379bbc1732feaf8bac2 P 5df61636ab3d4017a6832b15387e505c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "630413e403d5457c82dc9025104ddd07" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41293 } } peers { permanent_uuid: "5df61636ab3d4017a6832b15387e505c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40389 } } peers { permanent_uuid: "ba1ebe016f0742348997dbde07ec3c0f" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45243 } }
I20260812 08:03:21.645026 30944 consensus_queue.cc:260] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "630413e403d5457c82dc9025104ddd07" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41293 } } peers { permanent_uuid: "5df61636ab3d4017a6832b15387e505c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40389 } } peers { permanent_uuid: "ba1ebe016f0742348997dbde07ec3c0f" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45243 } }
I20260812 08:03:21.645136 30946 raft_consensus.cc:359] T f7b78a0c7c054379bbc1732feaf8bac2 P ba1ebe016f0742348997dbde07ec3c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "630413e403d5457c82dc9025104ddd07" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41293 } } peers { permanent_uuid: "5df61636ab3d4017a6832b15387e505c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40389 } } peers { permanent_uuid: "ba1ebe016f0742348997dbde07ec3c0f" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45243 } }
I20260812 08:03:21.645479 30452 heartbeater.cc:499] Master 127.18.42.126:45411 was elected leader, sending a full tablet report...
I20260812 08:03:21.645746 30947 ts_tablet_manager.cc:1434] T f7b78a0c7c054379bbc1732feaf8bac2 P 5df61636ab3d4017a6832b15387e505c: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:03:21.645740 30919 heartbeater.cc:499] Master 127.18.42.126:45411 was elected leader, sending a full tablet report...
I20260812 08:03:21.645996 30944 ts_tablet_manager.cc:1434] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:03:21.646505 30946 raft_consensus.cc:385] T f7b78a0c7c054379bbc1732feaf8bac2 P ba1ebe016f0742348997dbde07ec3c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:21.646642 30946 raft_consensus.cc:740] T f7b78a0c7c054379bbc1732feaf8bac2 P ba1ebe016f0742348997dbde07ec3c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba1ebe016f0742348997dbde07ec3c0f, State: Initialized, Role: FOLLOWER
I20260812 08:03:21.647043 30946 consensus_queue.cc:260] T f7b78a0c7c054379bbc1732feaf8bac2 P ba1ebe016f0742348997dbde07ec3c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "630413e403d5457c82dc9025104ddd07" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41293 } } peers { permanent_uuid: "5df61636ab3d4017a6832b15387e505c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40389 } } peers { permanent_uuid: "ba1ebe016f0742348997dbde07ec3c0f" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45243 } }
I20260812 08:03:21.647677 30688 heartbeater.cc:499] Master 127.18.42.126:45411 was elected leader, sending a full tablet report...
I20260812 08:03:21.647876 30946 ts_tablet_manager.cc:1434] T f7b78a0c7c054379bbc1732feaf8bac2 P ba1ebe016f0742348997dbde07ec3c0f: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.005s
I20260812 08:03:21.659281 30954 raft_consensus.cc:493] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:21.659464 30954 raft_consensus.cc:515] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "630413e403d5457c82dc9025104ddd07" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41293 } } peers { permanent_uuid: "5df61636ab3d4017a6832b15387e505c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40389 } } peers { permanent_uuid: "ba1ebe016f0742348997dbde07ec3c0f" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45243 } }
I20260812 08:03:21.660599 30954 leader_election.cc:290] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5df61636ab3d4017a6832b15387e505c (127.18.42.65:40389), ba1ebe016f0742348997dbde07ec3c0f (127.18.42.66:45243)
I20260812 08:03:21.665719 30369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b78a0c7c054379bbc1732feaf8bac2" candidate_uuid: "630413e403d5457c82dc9025104ddd07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5df61636ab3d4017a6832b15387e505c" is_pre_election: true
I20260812 08:03:21.665999 30369 raft_consensus.cc:2468] T f7b78a0c7c054379bbc1732feaf8bac2 P 5df61636ab3d4017a6832b15387e505c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 630413e403d5457c82dc9025104ddd07 in term 0.
I20260812 08:03:21.666023 30608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b78a0c7c054379bbc1732feaf8bac2" candidate_uuid: "630413e403d5457c82dc9025104ddd07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba1ebe016f0742348997dbde07ec3c0f" is_pre_election: true
I20260812 08:03:21.666293 30608 raft_consensus.cc:2468] T f7b78a0c7c054379bbc1732feaf8bac2 P ba1ebe016f0742348997dbde07ec3c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 630413e403d5457c82dc9025104ddd07 in term 0.
I20260812 08:03:21.666504 30718 leader_election.cc:304] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5df61636ab3d4017a6832b15387e505c, 630413e403d5457c82dc9025104ddd07; no voters: 
I20260812 08:03:21.666754 30954 raft_consensus.cc:2804] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:03:21.666823 30954 raft_consensus.cc:493] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:03:21.666882 30954 raft_consensus.cc:3060] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:21.667769 30954 raft_consensus.cc:515] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "630413e403d5457c82dc9025104ddd07" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41293 } } peers { permanent_uuid: "5df61636ab3d4017a6832b15387e505c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40389 } } peers { permanent_uuid: "ba1ebe016f0742348997dbde07ec3c0f" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45243 } }
I20260812 08:03:21.668195 30954 leader_election.cc:290] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07 [CANDIDATE]: Term 1 election: Requested vote from peers 5df61636ab3d4017a6832b15387e505c (127.18.42.65:40389), ba1ebe016f0742348997dbde07ec3c0f (127.18.42.66:45243)
I20260812 08:03:21.668576 30608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b78a0c7c054379bbc1732feaf8bac2" candidate_uuid: "630413e403d5457c82dc9025104ddd07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba1ebe016f0742348997dbde07ec3c0f"
I20260812 08:03:21.668672 30369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b78a0c7c054379bbc1732feaf8bac2" candidate_uuid: "630413e403d5457c82dc9025104ddd07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5df61636ab3d4017a6832b15387e505c"
I20260812 08:03:21.668720 30608 raft_consensus.cc:3060] T f7b78a0c7c054379bbc1732feaf8bac2 P ba1ebe016f0742348997dbde07ec3c0f [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:21.668860 30369 raft_consensus.cc:3060] T f7b78a0c7c054379bbc1732feaf8bac2 P 5df61636ab3d4017a6832b15387e505c [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:21.669942 30608 raft_consensus.cc:2468] T f7b78a0c7c054379bbc1732feaf8bac2 P ba1ebe016f0742348997dbde07ec3c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 630413e403d5457c82dc9025104ddd07 in term 1.
I20260812 08:03:21.670317 30719 leader_election.cc:304] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 630413e403d5457c82dc9025104ddd07, ba1ebe016f0742348997dbde07ec3c0f; no voters: 
I20260812 08:03:21.670514 30954 raft_consensus.cc:2804] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:21.670607 30369 raft_consensus.cc:2468] T f7b78a0c7c054379bbc1732feaf8bac2 P 5df61636ab3d4017a6832b15387e505c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 630413e403d5457c82dc9025104ddd07 in term 1.
I20260812 08:03:21.670763 30954 raft_consensus.cc:697] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07 [term 1 LEADER]: Becoming Leader. State: Replica: 630413e403d5457c82dc9025104ddd07, State: Running, Role: LEADER
I20260812 08:03:21.671087 30954 consensus_queue.cc:237] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "630413e403d5457c82dc9025104ddd07" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41293 } } peers { permanent_uuid: "5df61636ab3d4017a6832b15387e505c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40389 } } peers { permanent_uuid: "ba1ebe016f0742348997dbde07ec3c0f" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45243 } }
I20260812 08:03:21.674085 30118 catalog_manager.cc:5719] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 630413e403d5457c82dc9025104ddd07 (127.18.42.67). New cstate: current_term: 1 leader_uuid: "630413e403d5457c82dc9025104ddd07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "630413e403d5457c82dc9025104ddd07" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5df61636ab3d4017a6832b15387e505c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba1ebe016f0742348997dbde07ec3c0f" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45243 } health_report { overall_health: UNKNOWN } } }
W20260812 08:03:21.680780 30920 tablet.cc:2406] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:03:21.685791 18601 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:03:21.687227 18601 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 5df61636ab3d4017a6832b15387e505c to finish bootstrapping
W20260812 08:03:21.691401 30689 tablet.cc:2406] T f7b78a0c7c054379bbc1732feaf8bac2 P ba1ebe016f0742348997dbde07ec3c0f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:03:21.692603 18601 ts_itest-base.cc:246] Waiting for 1 tablets on tserver ba1ebe016f0742348997dbde07ec3c0f to finish bootstrapping
I20260812 08:03:21.697041 18601 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 630413e403d5457c82dc9025104ddd07 to finish bootstrapping
W20260812 08:03:21.704166 30453 tablet.cc:2406] T f7b78a0c7c054379bbc1732feaf8bac2 P 5df61636ab3d4017a6832b15387e505c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:03:21.712594 18601 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 1.1 0.0
I20260812 08:03:21.815965 18601 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 1.1 0.0
I20260812 08:03:22.019294 18601 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 1.1 0.0
I20260812 08:03:22.110980 30954 consensus_queue.cc:1048] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5df61636ab3d4017a6832b15387e505c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:22.117333 30966 consensus_queue.cc:1048] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba1ebe016f0742348997dbde07ec3c0f" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:22.347203 18601 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
I20260812 08:03:22.347604 18601 raft_consensus-itest-base.cc:268] Waiting until we've written at least 4MB...
I20260812 08:03:22.407400 30453 maintenance_manager.cc:419] P 5df61636ab3d4017a6832b15387e505c: Scheduling FlushMRSOp(f7b78a0c7c054379bbc1732feaf8bac2): 1840852 bytes log retention, and flush 2998537 bytes memory
I20260812 08:03:22.458077 18601 raft_consensus-itest-base.cc:274] Waiting for log GC on 630413e403d5457c82dc9025104ddd07
I20260812 08:03:22.459231 30267 maintenance_manager.cc:643] P 5df61636ab3d4017a6832b15387e505c: FlushMRSOp(f7b78a0c7c054379bbc1732feaf8bac2) complete. Timing: real 0.051s	user 0.038s	sys 0.004s Metrics: {"bytes_written":135869,"cfile_init":1,"compiler_manager_pool.queue_time_us":1449,"drs_written":1,"lbm_read_time_us":88,"lbm_reads_lt_1ms":4,"lbm_write_time_us":940,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":17,"thread_start_us":285,"threads_started":2}
I20260812 08:03:22.460343 30453 maintenance_manager.cc:419] P 5df61636ab3d4017a6832b15387e505c: Scheduling LogGCOp(f7b78a0c7c054379bbc1732feaf8bac2): free 1840852 bytes of WAL
I20260812 08:03:22.460606 30267 log_reader.cc:385] T f7b78a0c7c054379bbc1732feaf8bac2: removed 2 log segments from log reader
I20260812 08:03:22.460698 30267 log.cc:1079] T f7b78a0c7c054379bbc1732feaf8bac2 P 5df61636ab3d4017a6832b15387e505c: Deleting log segment in path: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal/wals/f7b78a0c7c054379bbc1732feaf8bac2/wal-000000001 (ops 1-8)
I20260812 08:03:22.460786 30267 log.cc:1079] T f7b78a0c7c054379bbc1732feaf8bac2 P 5df61636ab3d4017a6832b15387e505c: Deleting log segment in path: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal/wals/f7b78a0c7c054379bbc1732feaf8bac2/wal-000000002 (ops 9-15)
I20260812 08:03:22.461447 30267 maintenance_manager.cc:643] P 5df61636ab3d4017a6832b15387e505c: LogGCOp(f7b78a0c7c054379bbc1732feaf8bac2) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 08:03:22.461793 30453 maintenance_manager.cc:419] P 5df61636ab3d4017a6832b15387e505c: Scheduling FlushMRSOp(f7b78a0c7c054379bbc1732feaf8bac2): 2629693 bytes log retention, and flush 2998537 bytes memory
I20260812 08:03:22.483881 30267 maintenance_manager.cc:643] P 5df61636ab3d4017a6832b15387e505c: FlushMRSOp(f7b78a0c7c054379bbc1732feaf8bac2) complete. Timing: real 0.022s	user 0.018s	sys 0.002s Metrics: {"bytes_written":135869,"cfile_init":1,"dirs.queue_time_us":73,"dirs.run_cpu_time_us":241,"dirs.run_wall_time_us":1513,"drs_written":1,"lbm_read_time_us":41,"lbm_reads_lt_1ms":4,"lbm_write_time_us":752,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":19}
I20260812 08:03:22.484889 30453 maintenance_manager.cc:419] P 5df61636ab3d4017a6832b15387e505c: Scheduling LogGCOp(f7b78a0c7c054379bbc1732feaf8bac2): free 788953 bytes of WAL
I20260812 08:03:22.485070 30920 maintenance_manager.cc:419] P 630413e403d5457c82dc9025104ddd07: Scheduling FlushMRSOp(f7b78a0c7c054379bbc1732feaf8bac2): 4602202 bytes log retention, and flush 6119689 bytes memory
I20260812 08:03:22.485160 30267 log_reader.cc:385] T f7b78a0c7c054379bbc1732feaf8bac2: removed 1 log segments from log reader
I20260812 08:03:22.485237 30267 log.cc:1079] T f7b78a0c7c054379bbc1732feaf8bac2 P 5df61636ab3d4017a6832b15387e505c: Deleting log segment in path: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal/wals/f7b78a0c7c054379bbc1732feaf8bac2/wal-000000003 (ops 16-21)
I20260812 08:03:22.485702 30267 maintenance_manager.cc:643] P 5df61636ab3d4017a6832b15387e505c: LogGCOp(f7b78a0c7c054379bbc1732feaf8bac2) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 08:03:22.486047 30453 maintenance_manager.cc:419] P 5df61636ab3d4017a6832b15387e505c: Scheduling CompactRowSetsOp(f7b78a0c7c054379bbc1732feaf8bac2): perf score=0.974356
I20260812 08:03:22.547322 30267 maintenance_manager.cc:643] P 5df61636ab3d4017a6832b15387e505c: CompactRowSetsOp(f7b78a0c7c054379bbc1732feaf8bac2) complete. Timing: real 0.061s	user 0.057s	sys 0.003s Metrics: {"bytes_written":136001,"cfile_cache_hit":10,"cfile_cache_hit_bytes":262392,"cfile_cache_miss":26,"cfile_cache_miss_bytes":264418,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":1101,"dirs.run_cpu_time_us":414,"dirs.run_wall_time_us":2205,"drs_written":1,"lbm_read_time_us":1316,"lbm_reads_lt_1ms":54,"lbm_write_time_us":845,"lbm_writes_lt_1ms":23,"mutex_wait_us":288,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":36,"thread_start_us":410,"threads_started":5}
I20260812 08:03:22.548179 30453 maintenance_manager.cc:419] P 5df61636ab3d4017a6832b15387e505c: Scheduling UndoDeltaBlockGCOp(f7b78a0c7c054379bbc1732feaf8bac2): 536 bytes on disk
I20260812 08:03:22.549072 30267 maintenance_manager.cc:643] P 5df61636ab3d4017a6832b15387e505c: UndoDeltaBlockGCOp(f7b78a0c7c054379bbc1732feaf8bac2) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":61,"lbm_reads_lt_1ms":4}
I20260812 08:03:22.552625 30724 maintenance_manager.cc:643] P 630413e403d5457c82dc9025104ddd07: FlushMRSOp(f7b78a0c7c054379bbc1732feaf8bac2) complete. Timing: real 0.067s	user 0.064s	sys 0.000s Metrics: {"bytes_written":136001,"cfile_init":1,"compiler_manager_pool.queue_time_us":288,"dirs.queue_time_us":1881,"dirs.run_cpu_time_us":188,"dirs.run_wall_time_us":986,"drs_written":1,"lbm_read_time_us":96,"lbm_reads_lt_1ms":4,"lbm_write_time_us":895,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":36,"thread_start_us":310,"threads_started":2}
I20260812 08:03:22.553844 30920 maintenance_manager.cc:419] P 630413e403d5457c82dc9025104ddd07: Scheduling LogGCOp(f7b78a0c7c054379bbc1732feaf8bac2): free 2761328 bytes of WAL
I20260812 08:03:22.554133 30724 log_reader.cc:385] T f7b78a0c7c054379bbc1732feaf8bac2: removed 3 log segments from log reader
I20260812 08:03:22.554219 30724 log.cc:1079] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07: Deleting log segment in path: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal/wals/f7b78a0c7c054379bbc1732feaf8bac2/wal-000000001 (ops 1-8)
I20260812 08:03:22.554306 30724 log.cc:1079] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07: Deleting log segment in path: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal/wals/f7b78a0c7c054379bbc1732feaf8bac2/wal-000000002 (ops 9-15)
I20260812 08:03:22.554360 30724 log.cc:1079] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07: Deleting log segment in path: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal/wals/f7b78a0c7c054379bbc1732feaf8bac2/wal-000000003 (ops 16-22)
I20260812 08:03:22.555516 30724 maintenance_manager.cc:643] P 630413e403d5457c82dc9025104ddd07: LogGCOp(f7b78a0c7c054379bbc1732feaf8bac2) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 08:03:22.555889 30920 maintenance_manager.cc:419] P 630413e403d5457c82dc9025104ddd07: Scheduling UndoDeltaBlockGCOp(f7b78a0c7c054379bbc1732feaf8bac2): 536 bytes on disk
I20260812 08:03:22.556488 30724 maintenance_manager.cc:643] P 630413e403d5457c82dc9025104ddd07: UndoDeltaBlockGCOp(f7b78a0c7c054379bbc1732feaf8bac2) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":55,"lbm_reads_lt_1ms":4}
I20260812 08:03:22.560797 18601 raft_consensus-itest-base.cc:279] Log GC complete on 630413e403d5457c82dc9025104ddd07
I20260812 08:03:24.562358 18601 raft_consensus-itest-base.cc:295] Servers converged with original term 1
I20260812 08:03:24.562479 18601 raft_consensus-itest-base.cc:300] Resuming ba1ebe016f0742348997dbde07ec3c0f
I20260812 08:03:24.563704 30955 raft_consensus.cc:493] T f7b78a0c7c054379bbc1732feaf8bac2 P ba1ebe016f0742348997dbde07ec3c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 630413e403d5457c82dc9025104ddd07)
I20260812 08:03:24.563875 30955 raft_consensus.cc:515] T f7b78a0c7c054379bbc1732feaf8bac2 P ba1ebe016f0742348997dbde07ec3c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "630413e403d5457c82dc9025104ddd07" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 41293 } } peers { permanent_uuid: "5df61636ab3d4017a6832b15387e505c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 40389 } } peers { permanent_uuid: "ba1ebe016f0742348997dbde07ec3c0f" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 45243 } }
I20260812 08:03:24.565258 30955 leader_election.cc:290] T f7b78a0c7c054379bbc1732feaf8bac2 P ba1ebe016f0742348997dbde07ec3c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 630413e403d5457c82dc9025104ddd07 (127.18.42.67:41293), 5df61636ab3d4017a6832b15387e505c (127.18.42.65:40389)
I20260812 08:03:24.570381 30836 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f7b78a0c7c054379bbc1732feaf8bac2"
dest_uuid: "630413e403d5457c82dc9025104ddd07"
mode: GRACEFUL
new_leader_uuid: "ba1ebe016f0742348997dbde07ec3c0f"
 from {username='slave'} at 127.0.0.1:33666
I20260812 08:03:24.570592 30836 raft_consensus.cc:606] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07 [term 1 LEADER]: Received request to transfer leadership to TS ba1ebe016f0742348997dbde07ec3c0f
I20260812 08:03:24.570648 31018 consensus_queue.cc:753] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07 [LEADER]: The logs necessary to catch up peer ba1ebe016f0742348997dbde07ec3c0f have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 3..31: Segment 1 which contained index 3 has been GCed)
I20260812 08:03:24.570725 31018 consensus_queue.cc:579] T f7b78a0c7c054379bbc1732feaf8bac2 P 630413e403d5457c82dc9025104ddd07 [LEADER]: The logs necessary to catch up peer ba1ebe016f0742348997dbde07ec3c0f have been garbage collected. The replica will never be able to catch up
W20260812 08:03:24.572801 30113 catalog_manager.cc:5397] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet f7b78a0c7c054379bbc1732feaf8bac2 with cas_config_opid_index -1: no extra replica candidate found for tablet f7b78a0c7c054379bbc1732feaf8bac2 (table TestTable [id=8aa273786b3f4d208c0623f75d8e891a]): Not found: could not select location for extra replica: not enough tablet servers to satisfy replica placement policy: the total number of registered tablet servers (3) does not allow for adding an extra replica; consider bringing up more to have at least 4 tablet servers up and running
I20260812 08:03:24.574208 30368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b78a0c7c054379bbc1732feaf8bac2" candidate_uuid: "ba1ebe016f0742348997dbde07ec3c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5df61636ab3d4017a6832b15387e505c" is_pre_election: true
W20260812 08:03:24.575137 30771 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33666: Service unavailable: leader transfer in progress
I20260812 08:03:24.576025 30836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b78a0c7c054379bbc1732feaf8bac2" candidate_uuid: "ba1ebe016f0742348997dbde07ec3c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "630413e403d5457c82dc9025104ddd07" is_pre_election: true
I20260812 08:03:24.576697 30497 leader_election.cc:304] T f7b78a0c7c054379bbc1732feaf8bac2 P ba1ebe016f0742348997dbde07ec3c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba1ebe016f0742348997dbde07ec3c0f; no voters: 5df61636ab3d4017a6832b15387e505c, 630413e403d5457c82dc9025104ddd07
I20260812 08:03:24.577010 30955 raft_consensus.cc:2749] T f7b78a0c7c054379bbc1732feaf8bac2 P ba1ebe016f0742348997dbde07ec3c0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 08:03:24.579849 30771 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33666: Service unavailable: leader transfer in progress
W20260812 08:03:24.585246 30771 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33666: Service unavailable: leader transfer in progress
W20260812 08:03:24.592898 30771 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33666: Service unavailable: leader transfer in progress
W20260812 08:03:24.605409 30771 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33666: Service unavailable: leader transfer in progress
W20260812 08:03:24.625948 30771 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33666: Service unavailable: leader transfer in progress
W20260812 08:03:24.662441 30771 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33666: Service unavailable: leader transfer in progress
W20260812 08:03:24.730643 30771 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33666: Service unavailable: leader transfer in progress
W20260812 08:03:24.862928 30771 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33666: Service unavailable: leader transfer in progress
W20260812 08:03:25.123297 30771 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33666: Service unavailable: leader transfer in progress
W20260812 08:03:25.639756 30771 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33666: Service unavailable: leader transfer in progress
I20260812 08:03:26.648715 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 30221
I20260812 08:03:26.670626 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 30456
I20260812 08:03:26.687752 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 30691
I20260812 08:03:26.708384 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 30064
2026-08-12T08:03:26Z chronyd exiting
[       OK ] RaftConsensusITest.TestLeaderTransferWhenFollowerFallsBehindLeaderGC (5991 ms)
[----------] 6 tests from RaftConsensusITest (29138 ms total)

[----------] 1 test from RaftConsensusParamReplicationModesITest
[ RUN      ] RaftConsensusParamReplicationModesITest.Test_KUDU_1735/1
I20260812 08:03:26.743088 18601 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:03:26.743144 18601 ts_itest-base.cc:116] --------------
I20260812 08:03:26.743155 18601 ts_itest-base.cc:117] 3 tablet servers
I20260812 08:03:26.743166 18601 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:03:26.743176 18601 ts_itest-base.cc:119] --------------
2026-08-12T08:03:26Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:03:26Z Disabled control of system clock
I20260812 08:03:26.753234 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.42.126:45841
--webserver_interface=127.18.42.126
--webserver_port=0
--builtin_ntp_servers=127.18.42.84:42841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.42.126:45841
--catalog_manager_wait_for_new_tablets_to_elect_leader=false
--raft_prepare_replacement_before_eviction=true with env {}
W20260812 08:03:26.860033 31058 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:26.860307 31058 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:26.860460 31058 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:26.863853 31058 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:03:26.863955 31058 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:26.863994 31058 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:03:26.864035 31058 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:03:26.867888 31058 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:42841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
--catalog_manager_wait_for_new_tablets_to_elect_leader=false
--ipki_ca_key_size=768
--master_addresses=127.18.42.126:45841
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.42.126:45841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.18.42.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.31058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:26.869014 31058 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:26.870173 31058 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:26.875928 31068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:26.876019 31058 server_base.cc:1061] running on GCE node
W20260812 08:03:26.876020 31067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:26.876276 31070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:26.876793 31058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:26.877938 31058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:26.879120 31058 hybrid_clock.cc:648] HybridClock initialized: now 1786521806879133 us; error 58 us; skew 500 ppm
I20260812 08:03:26.881116 31058 webserver.cc:533] Webserver started at http://127.18.42.126:40949/ using document root <none> and password file <none>
I20260812 08:03:26.881806 31058 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:26.881893 31058 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:26.882115 31058 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:26.883698 31058 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "c9f8de27f1f84874ae01532707b61820"
format_stamp: "Formatted at 2026-08-12 08:03:26 on dist-test-slave-zr1t"
I20260812 08:03:26.884205 31058 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "c9f8de27f1f84874ae01532707b61820"
format_stamp: "Formatted at 2026-08-12 08:03:26 on dist-test-slave-zr1t"
I20260812 08:03:26.887537 31058 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:03:26.889690 31080 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:26.890616 31058 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:26.890750 31058 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
uuid: "c9f8de27f1f84874ae01532707b61820"
format_stamp: "Formatted at 2026-08-12 08:03:26 on dist-test-slave-zr1t"
I20260812 08:03:26.890846 31058 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:26.927026 31058 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:26.927794 31058 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:26.928005 31058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:26.935786 31058 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.126:45841
I20260812 08:03:26.935784 31163 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.126:45841 every 8 connection(s)
I20260812 08:03:26.936841 31058 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:03:26.939393 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 31058
I20260812 08:03:26.939538 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:03:26.939862 31164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:26.945873 31164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9f8de27f1f84874ae01532707b61820: Bootstrap starting.
I20260812 08:03:26.949397 31164 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9f8de27f1f84874ae01532707b61820: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:26.950242 31164 log.cc:826] T 00000000000000000000000000000000 P c9f8de27f1f84874ae01532707b61820: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:26.952394 31164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9f8de27f1f84874ae01532707b61820: No bootstrap required, opened a new log
I20260812 08:03:26.955446 31164 raft_consensus.cc:359] T 00000000000000000000000000000000 P c9f8de27f1f84874ae01532707b61820 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f8de27f1f84874ae01532707b61820" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 45841 } }
I20260812 08:03:26.955670 31164 raft_consensus.cc:385] T 00000000000000000000000000000000 P c9f8de27f1f84874ae01532707b61820 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:26.955756 31164 raft_consensus.cc:740] T 00000000000000000000000000000000 P c9f8de27f1f84874ae01532707b61820 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9f8de27f1f84874ae01532707b61820, State: Initialized, Role: FOLLOWER
I20260812 08:03:26.956164 31164 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9f8de27f1f84874ae01532707b61820 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f8de27f1f84874ae01532707b61820" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 45841 } }
I20260812 08:03:26.956310 31164 raft_consensus.cc:399] T 00000000000000000000000000000000 P c9f8de27f1f84874ae01532707b61820 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:03:26.956400 31164 raft_consensus.cc:493] T 00000000000000000000000000000000 P c9f8de27f1f84874ae01532707b61820 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:03:26.956535 31164 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c9f8de27f1f84874ae01532707b61820 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:26.957502 31164 raft_consensus.cc:515] T 00000000000000000000000000000000 P c9f8de27f1f84874ae01532707b61820 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f8de27f1f84874ae01532707b61820" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 45841 } }
I20260812 08:03:26.957895 31164 leader_election.cc:304] T 00000000000000000000000000000000 P c9f8de27f1f84874ae01532707b61820 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9f8de27f1f84874ae01532707b61820; no voters: 
I20260812 08:03:26.958251 31164 leader_election.cc:290] T 00000000000000000000000000000000 P c9f8de27f1f84874ae01532707b61820 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:03:26.958396 31172 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c9f8de27f1f84874ae01532707b61820 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:26.958642 31172 raft_consensus.cc:697] T 00000000000000000000000000000000 P c9f8de27f1f84874ae01532707b61820 [term 1 LEADER]: Becoming Leader. State: Replica: c9f8de27f1f84874ae01532707b61820, State: Running, Role: LEADER
I20260812 08:03:26.959028 31172 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9f8de27f1f84874ae01532707b61820 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f8de27f1f84874ae01532707b61820" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 45841 } }
I20260812 08:03:26.959482 31164 sys_catalog.cc:565] T 00000000000000000000000000000000 P c9f8de27f1f84874ae01532707b61820 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:26.960738 31174 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9f8de27f1f84874ae01532707b61820 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9f8de27f1f84874ae01532707b61820. Latest consensus state: current_term: 1 leader_uuid: "c9f8de27f1f84874ae01532707b61820" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f8de27f1f84874ae01532707b61820" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 45841 } } }
I20260812 08:03:26.960880 31174 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9f8de27f1f84874ae01532707b61820 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:26.961140 31173 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9f8de27f1f84874ae01532707b61820 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9f8de27f1f84874ae01532707b61820" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f8de27f1f84874ae01532707b61820" member_type: VOTER last_known_addr { host: "127.18.42.126" port: 45841 } } }
I20260812 08:03:26.961228 31173 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9f8de27f1f84874ae01532707b61820 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:26.961509 31183 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:03:26.964252 31183 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:03:26.970037 31183 catalog_manager.cc:1383] Generated new cluster ID: fec898f072a74c50a9b50522b63e5e2c
I20260812 08:03:26.970115 31183 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:03:26.990281 31183 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:03:26.991472 31183 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:03:26.999231 31183 catalog_manager.cc:6092] T 00000000000000000000000000000000 P c9f8de27f1f84874ae01532707b61820: Generated new TSK 0
I20260812 08:03:26.999917 31183 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:03:27.008347 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.65:0
--local_ip_for_outbound_sockets=127.18.42.65
--webserver_interface=127.18.42.65
--webserver_port=0
--tserver_master_addrs=127.18.42.126:45841
--builtin_ntp_servers=127.18.42.84:42841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--raft_prepare_replacement_before_eviction=true with env {}
W20260812 08:03:27.120203 31201 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 08:03:27.120491 31201 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:27.120561 31201 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:27.120642 31201 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:27.124123 31201 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:27.124300 31201 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.65
I20260812 08:03:27.128223 31201 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:42841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.18.42.65
--webserver_port=0
--tserver_master_addrs=127.18.42.126:45841
--never_fsync=true
--heap_profile_path=/tmp/kudu.31201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:27.129325 31201 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:27.130553 31201 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:27.136874 31210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:27.136957 31213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:27.136888 31208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:27.137229 31201 server_base.cc:1061] running on GCE node
I20260812 08:03:27.137647 31201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:27.138233 31201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:27.139395 31201 hybrid_clock.cc:648] HybridClock initialized: now 1786521807139368 us; error 48 us; skew 500 ppm
I20260812 08:03:27.141228 31201 webserver.cc:533] Webserver started at http://127.18.42.65:46729/ using document root <none> and password file <none>
I20260812 08:03:27.141935 31201 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:27.141999 31201 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:27.142208 31201 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:27.143853 31201 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "6ad3f890a7a943de85f5ae307f5d103c"
format_stamp: "Formatted at 2026-08-12 08:03:27 on dist-test-slave-zr1t"
I20260812 08:03:27.144368 31201 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "6ad3f890a7a943de85f5ae307f5d103c"
format_stamp: "Formatted at 2026-08-12 08:03:27 on dist-test-slave-zr1t"
I20260812 08:03:27.147781 31201 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:03:27.149937 31225 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:27.150928 31201 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:27.151063 31201 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "6ad3f890a7a943de85f5ae307f5d103c"
format_stamp: "Formatted at 2026-08-12 08:03:27 on dist-test-slave-zr1t"
I20260812 08:03:27.151165 31201 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:27.169436 31201 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:27.170131 31201 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:27.170344 31201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:27.170956 31201 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:27.172058 31201 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:27.172137 31201 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:27.172209 31201 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:27.172261 31201 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:27.182333 31201 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.65:45017
I20260812 08:03:27.182365 31413 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.65:45017 every 8 connection(s)
I20260812 08:03:27.183314 31201 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:03:27.185767 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 31201
I20260812 08:03:27.185870 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:03:27.188333 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.66:0
--local_ip_for_outbound_sockets=127.18.42.66
--webserver_interface=127.18.42.66
--webserver_port=0
--tserver_master_addrs=127.18.42.126:45841
--builtin_ntp_servers=127.18.42.84:42841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--raft_prepare_replacement_before_eviction=true with env {}
I20260812 08:03:27.192827 31414 heartbeater.cc:344] Connected to a master server at 127.18.42.126:45841
I20260812 08:03:27.193080 31414 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:27.193681 31414 heartbeater.cc:507] Master 127.18.42.126:45841 requested a full tablet report, sending...
I20260812 08:03:27.195207 31106 ts_manager.cc:194] Registered new tserver with Master: 6ad3f890a7a943de85f5ae307f5d103c (127.18.42.65:45017)
I20260812 08:03:27.196424 31106 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.65:56697
W20260812 08:03:27.306018 31418 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 08:03:27.306380 31418 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:27.306437 31418 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:27.306504 31418 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:27.310257 31418 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:27.310425 31418 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.66
I20260812 08:03:27.314539 31418 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:42841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.18.42.66
--webserver_port=0
--tserver_master_addrs=127.18.42.126:45841
--never_fsync=true
--heap_profile_path=/tmp/kudu.31418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:27.315626 31418 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:27.316782 31418 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:27.323182 31427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:27.323510 31418 server_base.cc:1061] running on GCE node
W20260812 08:03:27.323167 31425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:27.323173 31429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:27.323977 31418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:27.324476 31418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:27.325580 31418 hybrid_clock.cc:648] HybridClock initialized: now 1786521807325564 us; error 33 us; skew 500 ppm
I20260812 08:03:27.327311 31418 webserver.cc:533] Webserver started at http://127.18.42.66:43527/ using document root <none> and password file <none>
I20260812 08:03:27.327883 31418 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:27.327939 31418 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:27.328172 31418 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:27.329770 31418 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "90dda07aa7b141dc9303dc8203667fc3"
format_stamp: "Formatted at 2026-08-12 08:03:27 on dist-test-slave-zr1t"
I20260812 08:03:27.330252 31418 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "90dda07aa7b141dc9303dc8203667fc3"
format_stamp: "Formatted at 2026-08-12 08:03:27 on dist-test-slave-zr1t"
I20260812 08:03:27.333613 31418 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:03:27.335893 31438 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:27.336941 31418 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:27.337081 31418 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "90dda07aa7b141dc9303dc8203667fc3"
format_stamp: "Formatted at 2026-08-12 08:03:27 on dist-test-slave-zr1t"
I20260812 08:03:27.337185 31418 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:27.360791 31418 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:27.361600 31418 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:27.361804 31418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:27.362479 31418 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:27.363511 31418 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:27.363591 31418 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:27.363658 31418 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:27.363713 31418 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:27.373103 31418 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.66:35775
I20260812 08:03:27.373167 31641 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.66:35775 every 8 connection(s)
I20260812 08:03:27.374117 31418 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:03:27.376531 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 31418
I20260812 08:03:27.376636 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:03:27.379021 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.67:0
--local_ip_for_outbound_sockets=127.18.42.67
--webserver_interface=127.18.42.67
--webserver_port=0
--tserver_master_addrs=127.18.42.126:45841
--builtin_ntp_servers=127.18.42.84:42841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--raft_prepare_replacement_before_eviction=true with env {}
I20260812 08:03:27.383745 31642 heartbeater.cc:344] Connected to a master server at 127.18.42.126:45841
I20260812 08:03:27.383988 31642 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:27.384495 31642 heartbeater.cc:507] Master 127.18.42.126:45841 requested a full tablet report, sending...
I20260812 08:03:27.385685 31106 ts_manager.cc:194] Registered new tserver with Master: 90dda07aa7b141dc9303dc8203667fc3 (127.18.42.66:35775)
I20260812 08:03:27.386497 31106 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.66:40133
W20260812 08:03:27.488368 31653 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 08:03:27.488584 31653 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:27.488606 31653 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:27.488647 31653 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:27.491941 31653 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:27.492045 31653 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.67
I20260812 08:03:27.495795 31653 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:42841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.18.42.67
--webserver_port=0
--tserver_master_addrs=127.18.42.126:45841
--never_fsync=true
--heap_profile_path=/tmp/kudu.31653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:27.496830 31653 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:27.497908 31653 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:27.504614 31664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:27.504688 31668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:27.504657 31666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:27.504987 31653 server_base.cc:1061] running on GCE node
I20260812 08:03:27.505333 31653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:27.505872 31653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:27.507028 31653 hybrid_clock.cc:648] HybridClock initialized: now 1786521807507005 us; error 34 us; skew 500 ppm
I20260812 08:03:27.508673 31653 webserver.cc:533] Webserver started at http://127.18.42.67:41223/ using document root <none> and password file <none>
I20260812 08:03:27.509232 31653 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:27.509289 31653 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:27.509558 31653 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:27.511082 31653 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "1fb57828e6764314aaf7408ad4ef66b2"
format_stamp: "Formatted at 2026-08-12 08:03:27 on dist-test-slave-zr1t"
I20260812 08:03:27.511570 31653 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "1fb57828e6764314aaf7408ad4ef66b2"
format_stamp: "Formatted at 2026-08-12 08:03:27 on dist-test-slave-zr1t"
I20260812 08:03:27.514936 31653 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.003s
I20260812 08:03:27.517484 31674 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:27.518534 31653 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 08:03:27.518671 31653 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "1fb57828e6764314aaf7408ad4ef66b2"
format_stamp: "Formatted at 2026-08-12 08:03:27 on dist-test-slave-zr1t"
I20260812 08:03:27.518776 31653 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:27.541282 31653 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:27.542111 31653 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:27.542326 31653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:27.542929 31653 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:27.544065 31653 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:27.544137 31653 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:27.544209 31653 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:27.544252 31653 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:27.555703 31653 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.67:37839
I20260812 08:03:27.555814 31867 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.67:37839 every 8 connection(s)
I20260812 08:03:27.556732 31653 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:03:27.558320 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 31653
I20260812 08:03:27.558426 18601 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:03:27.566083 31868 heartbeater.cc:344] Connected to a master server at 127.18.42.126:45841
I20260812 08:03:27.566354 31868 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:27.566859 31868 heartbeater.cc:507] Master 127.18.42.126:45841 requested a full tablet report, sending...
I20260812 08:03:27.567942 31106 ts_manager.cc:194] Registered new tserver with Master: 1fb57828e6764314aaf7408ad4ef66b2 (127.18.42.67:37839)
I20260812 08:03:27.568451 31106 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.67:45071
I20260812 08:03:27.572893 18601 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:03:27.581707 31106 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55524:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
W20260812 08:03:27.584110 31106 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 08:03:27.600711 31535 tablet_service.cc:1511] Processing CreateTablet for tablet 5ca6ae5d0b454aa4a46607ee93c1c186 (DEFAULT_TABLE table=TestTable [id=aedb7b3b9ce14dad8fed8c21ebece2c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:27.600808 31756 tablet_service.cc:1511] Processing CreateTablet for tablet 5ca6ae5d0b454aa4a46607ee93c1c186 (DEFAULT_TABLE table=TestTable [id=aedb7b3b9ce14dad8fed8c21ebece2c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:27.600805 31305 tablet_service.cc:1511] Processing CreateTablet for tablet 5ca6ae5d0b454aa4a46607ee93c1c186 (DEFAULT_TABLE table=TestTable [id=aedb7b3b9ce14dad8fed8c21ebece2c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:27.601734 31535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ca6ae5d0b454aa4a46607ee93c1c186. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:27.601770 31305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ca6ae5d0b454aa4a46607ee93c1c186. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:27.601770 31756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ca6ae5d0b454aa4a46607ee93c1c186. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:27.606753 31892 tablet_bootstrap.cc:492] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: Bootstrap starting.
I20260812 08:03:27.607786 31894 tablet_bootstrap.cc:492] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c: Bootstrap starting.
I20260812 08:03:27.607786 31895 tablet_bootstrap.cc:492] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2: Bootstrap starting.
I20260812 08:03:27.609470 31892 tablet_bootstrap.cc:654] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:27.610014 31894 tablet_bootstrap.cc:654] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:27.610025 31895 tablet_bootstrap.cc:654] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:27.610316 31892 log.cc:826] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:27.610838 31894 log.cc:826] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:27.610838 31895 log.cc:826] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:27.612087 31892 tablet_bootstrap.cc:492] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: No bootstrap required, opened a new log
I20260812 08:03:27.612284 31892 ts_tablet_manager.cc:1403] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: Time spent bootstrapping tablet: real 0.006s	user 0.004s	sys 0.000s
I20260812 08:03:27.612387 31894 tablet_bootstrap.cc:492] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c: No bootstrap required, opened a new log
I20260812 08:03:27.612475 31895 tablet_bootstrap.cc:492] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2: No bootstrap required, opened a new log
I20260812 08:03:27.612578 31894 ts_tablet_manager.cc:1403] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:03:27.612654 31895 ts_tablet_manager.cc:1403] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:03:27.615247 31894 raft_consensus.cc:359] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90dda07aa7b141dc9303dc8203667fc3" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 35775 } } peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } }
I20260812 08:03:27.615456 31894 raft_consensus.cc:740] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad3f890a7a943de85f5ae307f5d103c, State: Initialized, Role: FOLLOWER
I20260812 08:03:27.615761 31894 consensus_queue.cc:260] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90dda07aa7b141dc9303dc8203667fc3" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 35775 } } peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } }
I20260812 08:03:27.615689 31892 raft_consensus.cc:359] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90dda07aa7b141dc9303dc8203667fc3" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 35775 } } peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } }
I20260812 08:03:27.615689 31895 raft_consensus.cc:359] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90dda07aa7b141dc9303dc8203667fc3" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 35775 } } peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } }
I20260812 08:03:27.615897 31895 raft_consensus.cc:740] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fb57828e6764314aaf7408ad4ef66b2, State: Initialized, Role: FOLLOWER
I20260812 08:03:27.615906 31892 raft_consensus.cc:740] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90dda07aa7b141dc9303dc8203667fc3, State: Initialized, Role: FOLLOWER
I20260812 08:03:27.616204 31895 consensus_queue.cc:260] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90dda07aa7b141dc9303dc8203667fc3" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 35775 } } peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } }
I20260812 08:03:27.616204 31892 consensus_queue.cc:260] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90dda07aa7b141dc9303dc8203667fc3" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 35775 } } peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } }
I20260812 08:03:27.616554 31414 heartbeater.cc:499] Master 127.18.42.126:45841 was elected leader, sending a full tablet report...
I20260812 08:03:27.616734 31894 ts_tablet_manager.cc:1434] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:03:27.616885 31868 heartbeater.cc:499] Master 127.18.42.126:45841 was elected leader, sending a full tablet report...
I20260812 08:03:27.617075 31895 ts_tablet_manager.cc:1434] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:03:27.617587 31892 ts_tablet_manager.cc:1434] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260812 08:03:27.618093 31642 heartbeater.cc:499] Master 127.18.42.126:45841 was elected leader, sending a full tablet report...
W20260812 08:03:27.625018 31650 tablet.cc:2406] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:03:27.625257 18601 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:03:27.626631 18601 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 6ad3f890a7a943de85f5ae307f5d103c to finish bootstrapping
I20260812 08:03:27.632545 18601 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 90dda07aa7b141dc9303dc8203667fc3 to finish bootstrapping
I20260812 08:03:27.638082 18601 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 1fb57828e6764314aaf7408ad4ef66b2 to finish bootstrapping
I20260812 08:03:27.646512 31338 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ca6ae5d0b454aa4a46607ee93c1c186"
dest_uuid: "6ad3f890a7a943de85f5ae307f5d103c"
 from {username='slave'} at 127.0.0.1:48708
I20260812 08:03:27.646754 31338 raft_consensus.cc:493] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 08:03:27.646883 31338 raft_consensus.cc:3060] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:27.647857 31338 raft_consensus.cc:515] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90dda07aa7b141dc9303dc8203667fc3" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 35775 } } peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } }
I20260812 08:03:27.648942 31338 leader_election.cc:290] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [CANDIDATE]: Term 1 election: Requested vote from peers 90dda07aa7b141dc9303dc8203667fc3 (127.18.42.66:35775), 1fb57828e6764314aaf7408ad4ef66b2 (127.18.42.67:37839)
I20260812 08:03:27.653009 31566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ca6ae5d0b454aa4a46607ee93c1c186" candidate_uuid: "6ad3f890a7a943de85f5ae307f5d103c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "90dda07aa7b141dc9303dc8203667fc3"
I20260812 08:03:27.653240 31566 raft_consensus.cc:3060] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:27.654042 31791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ca6ae5d0b454aa4a46607ee93c1c186" candidate_uuid: "6ad3f890a7a943de85f5ae307f5d103c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1fb57828e6764314aaf7408ad4ef66b2"
I20260812 08:03:27.654207 31791 raft_consensus.cc:3060] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:27.654445 31566 raft_consensus.cc:2468] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad3f890a7a943de85f5ae307f5d103c in term 1.
I20260812 08:03:27.655149 31229 leader_election.cc:304] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad3f890a7a943de85f5ae307f5d103c, 90dda07aa7b141dc9303dc8203667fc3; no voters: 
I20260812 08:03:27.655198 31791 raft_consensus.cc:2468] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad3f890a7a943de85f5ae307f5d103c in term 1.
I20260812 08:03:27.655418 31902 raft_consensus.cc:2804] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:27.655658 31902 raft_consensus.cc:697] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [term 1 LEADER]: Becoming Leader. State: Replica: 6ad3f890a7a943de85f5ae307f5d103c, State: Running, Role: LEADER
I20260812 08:03:27.655984 31902 consensus_queue.cc:237] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90dda07aa7b141dc9303dc8203667fc3" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 35775 } } peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } }
I20260812 08:03:27.659243 31106 catalog_manager.cc:5719] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ad3f890a7a943de85f5ae307f5d103c (127.18.42.65). New cstate: current_term: 1 leader_uuid: "6ad3f890a7a943de85f5ae307f5d103c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90dda07aa7b141dc9303dc8203667fc3" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 35775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } health_report { overall_health: UNKNOWN } } }
W20260812 08:03:27.684465 31416 tablet.cc:2406] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:03:27.807680 31870 tablet.cc:2406] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:03:28.079392 31902 consensus_queue.cc:1048] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [LEADER]: Connected to new peer: Peer: permanent_uuid: "90dda07aa7b141dc9303dc8203667fc3" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 35775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:28.086565 31917 consensus_queue.cc:1048] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:28.100842 31628 generic_service.cc:175] {username='slave'} at 127.0.0.1:34858 changed flags via RPC: fault_crash_before_append_commit from '0' to '1.0'
I20260812 08:03:28.102002 31859 generic_service.cc:175] {username='slave'} at 127.0.0.1:43456 changed flags via RPC: fault_crash_before_append_commit from '0' to '1.0'
I20260812 08:03:28.103559 31338 consensus_queue.cc:237] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } }
I20260812 08:03:28.104760 31791 raft_consensus.cc:1275] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2 [term 1 FOLLOWER]: Refusing update from remote peer 6ad3f890a7a943de85f5ae307f5d103c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:03:28.105221 31917 consensus_queue.cc:1048] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:28.107128 31912 raft_consensus.cc:2955] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 90dda07aa7b141dc9303dc8203667fc3 (127.18.42.66) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } } }
I20260812 08:03:28.107679 31791 raft_consensus.cc:2955] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 90dda07aa7b141dc9303dc8203667fc3 (127.18.42.66) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } } }
W20260812 08:03:28.108938 31791 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260812 08:03:28.108989 31791 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260812 08:03:28.109014 31791 fault_injection.cc:59] injecting fault for kudu (pid 31653): FLAGS_fault_crash_before_append_commit (process will exit)
I20260812 08:03:28.119431 31104 catalog_manager.cc:5719] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c reported cstate change: config changed from index -1 to 2, VOTER 90dda07aa7b141dc9303dc8203667fc3 (127.18.42.66) evicted. New cstate: current_term: 1 leader_uuid: "6ad3f890a7a943de85f5ae307f5d103c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:28.122916 31535 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ca6ae5d0b454aa4a46607ee93c1c186 with delete_type TABLET_DATA_TOMBSTONED (TS 90dda07aa7b141dc9303dc8203667fc3 not found in new config with opid_index 2) from {username='slave'} at 127.0.0.1:34850
I20260812 08:03:28.123528 31925 tablet_replica.cc:333] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: stopping tablet replica
I20260812 08:03:28.123822 31925 raft_consensus.cc:2243] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 08:03:28.123871 31338 consensus_queue.cc:237] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } } peers { permanent_uuid: "90dda07aa7b141dc9303dc8203667fc3" member_type: NON_VOTER last_known_addr { host: "127.18.42.66" port: 35775 } attrs { promote: true } }
I20260812 08:03:28.124132 31925 raft_consensus.cc:2272] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 08:03:28.125331 31925 ts_tablet_manager.cc:1916] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260812 08:03:28.126220 31229 consensus_peers.cc:597] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c -> Peer 90dda07aa7b141dc9303dc8203667fc3 (127.18.42.66:35775): Couldn't send request to peer 90dda07aa7b141dc9303dc8203667fc3. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:03:28.127562 31925 ts_tablet_manager.cc:1929] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.1
I20260812 08:03:28.127684 31925 log.cc:1199] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: Deleting WAL directory at /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal/wals/5ca6ae5d0b454aa4a46607ee93c1c186
W20260812 08:03:28.127820 31229 connection.cc:573] client connection to 127.18.42.67:37839 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 08:03:28.128000 31229 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 08:03:28.128424 31084 catalog_manager.cc:5050] TS 90dda07aa7b141dc9303dc8203667fc3 (127.18.42.66:35775): tablet 5ca6ae5d0b454aa4a46607ee93c1c186 (table TestTable [id=aedb7b3b9ce14dad8fed8c21ebece2c1]) successfully deleted
W20260812 08:03:28.129179 31229 consensus_peers.cc:597] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c -> Peer 1fb57828e6764314aaf7408ad4ef66b2 (127.18.42.67:37839): Couldn't send request to peer 1fb57828e6764314aaf7408ad4ef66b2. Status: Network error: Client connection negotiation failed: client connection to 127.18.42.67:37839: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 08:03:28.544001 31927 ts_tablet_manager.cc:933] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: Initiating tablet copy from peer 6ad3f890a7a943de85f5ae307f5d103c (127.18.42.65:45017)
I20260812 08:03:28.544643 31927 tablet_copy_client.cc:323] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: tablet copy: Beginning tablet copy session from remote peer at address 127.18.42.65:45017
I20260812 08:03:28.549057 31374 tablet_copy_service.cc:140] P 6ad3f890a7a943de85f5ae307f5d103c: Received BeginTabletCopySession request for tablet 5ca6ae5d0b454aa4a46607ee93c1c186 from peer 90dda07aa7b141dc9303dc8203667fc3 ({username='slave'} at 127.18.42.66:41247)
I20260812 08:03:28.549202 31374 tablet_copy_service.cc:161] P 6ad3f890a7a943de85f5ae307f5d103c: Beginning new tablet copy session on tablet 5ca6ae5d0b454aa4a46607ee93c1c186 from peer 90dda07aa7b141dc9303dc8203667fc3 at {username='slave'} at 127.18.42.66:41247: session id = 90dda07aa7b141dc9303dc8203667fc3-5ca6ae5d0b454aa4a46607ee93c1c186
I20260812 08:03:28.550575 31374 tablet_copy_source_session.cc:215] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 08:03:28.552114 31927 ts_tablet_manager.cc:1916] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260812 08:03:28.554018 31927 ts_tablet_manager.cc:1929] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.1
I20260812 08:03:28.554239 31927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ca6ae5d0b454aa4a46607ee93c1c186. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:28.555320 31927 tablet_copy_client.cc:806] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: tablet copy: Starting download of 0 data blocks...
I20260812 08:03:28.555516 31927 tablet_copy_client.cc:670] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: tablet copy: Starting download of 1 WAL segments...
I20260812 08:03:28.557442 31927 tablet_copy_client.cc:538] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 08:03:28.558851 31927 tablet_bootstrap.cc:492] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: Bootstrap starting.
I20260812 08:03:28.563333 31927 tablet_bootstrap.cc:492] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260812 08:03:28.563611 31927 tablet_bootstrap.cc:492] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: Bootstrap complete.
I20260812 08:03:28.563742 31927 ts_tablet_manager.cc:1403] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:03:28.564235 31927 raft_consensus.cc:359] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } }
I20260812 08:03:28.564457 31927 raft_consensus.cc:740] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 90dda07aa7b141dc9303dc8203667fc3, State: Initialized, Role: LEARNER
I20260812 08:03:28.564548 31927 consensus_queue.cc:260] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } } peers { permanent_uuid: "90dda07aa7b141dc9303dc8203667fc3" member_type: NON_VOTER last_known_addr { host: "127.18.42.66" port: 35775 } attrs { promote: true } }
I20260812 08:03:28.564826 31927 ts_tablet_manager.cc:1434] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.000s
I20260812 08:03:28.565650 31374 tablet_copy_service.cc:342] P 6ad3f890a7a943de85f5ae307f5d103c: Request end of tablet copy session 90dda07aa7b141dc9303dc8203667fc3-5ca6ae5d0b454aa4a46607ee93c1c186 received from {username='slave'} at 127.18.42.66:41247
I20260812 08:03:28.565773 31374 tablet_copy_service.cc:434] P 6ad3f890a7a943de85f5ae307f5d103c: ending tablet copy session 90dda07aa7b141dc9303dc8203667fc3-5ca6ae5d0b454aa4a46607ee93c1c186 on tablet 5ca6ae5d0b454aa4a46607ee93c1c186 with peer 90dda07aa7b141dc9303dc8203667fc3
I20260812 08:03:28.948781 31566 raft_consensus.cc:1217] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.2->[1.3-1.3]   Dedup: 1.3->[]
I20260812 08:03:29.422417 31945 raft_consensus.cc:1027] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [term 1 LEADER]: attempt to promote peer 90dda07aa7b141dc9303dc8203667fc3: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260812 08:03:29.969081 31945 raft_consensus.cc:1027] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [term 1 LEADER]: attempt to promote peer 90dda07aa7b141dc9303dc8203667fc3: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260812 08:03:30.387060 31945 raft_consensus.cc:1027] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [term 1 LEADER]: attempt to promote peer 90dda07aa7b141dc9303dc8203667fc3: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
W20260812 08:03:30.726876 31229 consensus_peers.cc:597] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c -> Peer 1fb57828e6764314aaf7408ad4ef66b2 (127.18.42.67:37839): Couldn't send request to peer 1fb57828e6764314aaf7408ad4ef66b2. Status: Network error: Client connection negotiation failed: client connection to 127.18.42.67:37839: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260812 08:03:30.877815 31948 raft_consensus.cc:1027] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [term 1 LEADER]: attempt to promote peer 90dda07aa7b141dc9303dc8203667fc3: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260812 08:03:31.456503 31948 raft_consensus.cc:1027] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [term 1 LEADER]: attempt to promote peer 90dda07aa7b141dc9303dc8203667fc3: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260812 08:03:32.035960 31948 raft_consensus.cc:1027] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [term 1 LEADER]: attempt to promote peer 90dda07aa7b141dc9303dc8203667fc3: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260812 08:03:32.610103 31948 raft_consensus.cc:1027] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [term 1 LEADER]: attempt to promote peer 90dda07aa7b141dc9303dc8203667fc3: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260812 08:03:33.092470 31969 raft_consensus.cc:1027] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [term 1 LEADER]: attempt to promote peer 90dda07aa7b141dc9303dc8203667fc3: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
W20260812 08:03:33.165723 31229 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.42.67:37839: connect: Connection refused (error 111) [suppressed 10 similar messages]
W20260812 08:03:33.166563 31229 consensus_peers.cc:597] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c -> Peer 1fb57828e6764314aaf7408ad4ef66b2 (127.18.42.67:37839): Couldn't send request to peer 1fb57828e6764314aaf7408ad4ef66b2. Status: Network error: Client connection negotiation failed: client connection to 127.18.42.67:37839: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260812 08:03:33.219396 31106 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55524:
table { table_name: "TestTable" } modify_external_catalogs: true
I20260812 08:03:33.219583 31106 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55524:
table { table_name: "TestTable" } modify_external_catalogs: true
I20260812 08:03:33.222875 31106 catalog_manager.cc:6006] T 00000000000000000000000000000000 P c9f8de27f1f84874ae01532707b61820: Sending DeleteTablet for 2 replicas of tablet 5ca6ae5d0b454aa4a46607ee93c1c186
I20260812 08:03:33.223743 31305 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ca6ae5d0b454aa4a46607ee93c1c186 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 08:03:33 UTC) from {username='slave'} at 127.0.0.1:48700
W20260812 08:03:33.224149 31084 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.42.67:37839: connect: Connection refused (error 111)
I20260812 08:03:33.224238 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 31418
I20260812 08:03:33.224308 31976 tablet_replica.cc:333] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c: stopping tablet replica
I20260812 08:03:33.225067 31976 raft_consensus.cc:2243] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [term 1 LEADER]: Raft consensus shutting down.
I20260812 08:03:33.225368 31976 pending_rounds.cc:70] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c: Trying to abort 1 pending ops.
I20260812 08:03:33.225458 31976 pending_rounds.cc:77] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7317593326074433536 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "5ca6ae5d0b454aa4a46607ee93c1c186" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } } peers { permanent_uuid: "90dda07aa7b141dc9303dc8203667fc3" member_type: NON_VOTER last_known_addr { host: "127.18.42.66" port: 35775 } attrs { promote: true } } } }
I20260812 08:03:33.225626 31976 raft_consensus.cc:2917] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [term 1 LEADER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20260812 08:03:33.225693 31976 raft_consensus.cc:2889] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [term 1 LEADER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260812 08:03:33.225903 31976 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:48700 (request call id 1) took 5102 ms. Trace:
I20260812 08:03:33.225955 31976 rpcz_store.cc:276] 0812 08:03:28.123286 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:03:28.123434 (+   148us) service_pool.cc:224] Handling call
0812 08:03:28.124535 (+  1101us) log.cc:844] Serialized 369 byte log entry
0812 08:03:33.225888 (+5101353us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":854,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":227}
I20260812 08:03:33.226568 31976 raft_consensus.cc:2272] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 08:03:33.227480 31976 ts_tablet_manager.cc:1916] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 08:03:33.230221 31976 ts_tablet_manager.cc:1929] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260812 08:03:33.230307 31976 log.cc:1199] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c: Deleting WAL directory at /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-0/wal/wals/5ca6ae5d0b454aa4a46607ee93c1c186
I20260812 08:03:33.230612 31976 ts_tablet_manager.cc:1950] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 6ad3f890a7a943de85f5ae307f5d103c: Deleting consensus metadata
I20260812 08:03:33.235391 31084 catalog_manager.cc:5050] TS 6ad3f890a7a943de85f5ae307f5d103c (127.18.42.65:45017): tablet 5ca6ae5d0b454aa4a46607ee93c1c186 (table TestTable [id=aedb7b3b9ce14dad8fed8c21ebece2c1]) successfully deleted
W20260812 08:03:33.236086 31084 catalog_manager.cc:4777] TS 1fb57828e6764314aaf7408ad4ef66b2 (127.18.42.67:37839): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5ca6ae5d0b454aa4a46607ee93c1c186: Network error: Client connection negotiation failed: client connection to 127.18.42.67:37839: connect: Connection refused (error 111)
I20260812 08:03:33.243611 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.66:35775
--local_ip_for_outbound_sockets=127.18.42.66
--tserver_master_addrs=127.18.42.126:45841
--webserver_port=43527
--webserver_interface=127.18.42.66
--builtin_ntp_servers=127.18.42.84:42841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--raft_prepare_replacement_before_eviction=true with env {}
W20260812 08:03:33.354619 31978 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 08:03:33.354902 31978 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:33.354974 31978 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:33.355075 31978 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:33.358582 31978 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:33.358752 31978 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.66
I20260812 08:03:33.362771 31978 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:42841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.66:35775
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.18.42.66
--webserver_port=43527
--tserver_master_addrs=127.18.42.126:45841
--never_fsync=true
--heap_profile_path=/tmp/kudu.31978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:33.363936 31978 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:33.365127 31978 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:33.371765 31993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:33.371753 31991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:33.371757 31990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:33.372242 31978 server_base.cc:1061] running on GCE node
I20260812 08:03:33.372633 31978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:33.373257 31978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:33.374454 31978 hybrid_clock.cc:648] HybridClock initialized: now 1786521813374417 us; error 57 us; skew 500 ppm
I20260812 08:03:33.376474 31978 webserver.cc:533] Webserver started at http://127.18.42.66:43527/ using document root <none> and password file <none>
I20260812 08:03:33.377120 31978 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:33.377242 31978 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:33.380888 31978 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:33.383015 32003 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:33.384042 31978 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:03:33.384187 31978 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "90dda07aa7b141dc9303dc8203667fc3"
format_stamp: "Formatted at 2026-08-12 08:03:27 on dist-test-slave-zr1t"
I20260812 08:03:33.384601 31978 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:33.414508 31978 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:33.415279 31978 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:33.415517 31978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:33.416210 31978 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:33.417590 32015 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:03:33.421015 31978 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:33.421108 31978 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:03:33.421221 31978 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:33.422843 31978 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:33.422916 31978 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:03:33.423216 32015 tablet_bootstrap.cc:492] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: Bootstrap starting.
I20260812 08:03:33.435113 31978 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.66:35775
I20260812 08:03:33.435200 32185 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.66:35775 every 8 connection(s)
I20260812 08:03:33.436152 31978 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:03:33.440740 32015 log.cc:826] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:33.441425 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 31978
I20260812 08:03:33.441911 18601 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
/tmp/dist-test-taskGfvRiA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.42.67:37839
--local_ip_for_outbound_sockets=127.18.42.67
--tserver_master_addrs=127.18.42.126:45841
--webserver_port=41223
--webserver_interface=127.18.42.67
--builtin_ntp_servers=127.18.42.84:42841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--raft_prepare_replacement_before_eviction=true with env {}
I20260812 08:03:33.445924 32015 tablet_bootstrap.cc:492] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260812 08:03:33.446460 32015 tablet_bootstrap.cc:492] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: Bootstrap complete.
I20260812 08:03:33.448118 32015 ts_tablet_manager.cc:1403] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: Time spent bootstrapping tablet: real 0.025s	user 0.007s	sys 0.014s
I20260812 08:03:33.449946 32186 heartbeater.cc:344] Connected to a master server at 127.18.42.126:45841
I20260812 08:03:33.450193 32186 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:33.450729 32186 heartbeater.cc:507] Master 127.18.42.126:45841 requested a full tablet report, sending...
I20260812 08:03:33.452494 31106 ts_manager.cc:194] Re-registered known tserver with Master: 90dda07aa7b141dc9303dc8203667fc3 (127.18.42.66:35775)
I20260812 08:03:33.452291 32015 raft_consensus.cc:359] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } }
I20260812 08:03:33.453099 32015 raft_consensus.cc:740] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 90dda07aa7b141dc9303dc8203667fc3, State: Initialized, Role: LEARNER
I20260812 08:03:33.453608 32015 consensus_queue.cc:260] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } } peers { permanent_uuid: "90dda07aa7b141dc9303dc8203667fc3" member_type: NON_VOTER last_known_addr { host: "127.18.42.66" port: 35775 } attrs { promote: true } }
I20260812 08:03:33.453802 31106 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.66:33385
I20260812 08:03:33.454615 32015 ts_tablet_manager.cc:1434] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260812 08:03:33.455729 32186 heartbeater.cc:499] Master 127.18.42.126:45841 was elected leader, sending a full tablet report...
I20260812 08:03:33.460899 32087 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ca6ae5d0b454aa4a46607ee93c1c186 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 08:03:33 UTC) from {username='slave'} at 127.0.0.1:48678
I20260812 08:03:33.461617 32209 tablet_replica.cc:333] stopping tablet replica
I20260812 08:03:33.461875 32209 raft_consensus.cc:2243] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3 [term 1 LEARNER]: Raft consensus shutting down.
I20260812 08:03:33.462078 32209 pending_rounds.cc:70] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: Trying to abort 1 pending ops.
I20260812 08:03:33.462157 32209 pending_rounds.cc:77] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7317593326074433536 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "5ca6ae5d0b454aa4a46607ee93c1c186" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } } peers { permanent_uuid: "90dda07aa7b141dc9303dc8203667fc3" member_type: NON_VOTER last_known_addr { host: "127.18.42.66" port: 35775 } attrs { promote: true } } } }
I20260812 08:03:33.462343 32209 raft_consensus.cc:2917] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3 [term 1 LEARNER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20260812 08:03:33.462450 32209 raft_consensus.cc:2889] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3 [term 1 NON_PARTICIPANT]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260812 08:03:33.462522 32209 raft_consensus.cc:2272] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260812 08:03:33.463317 32209 ts_tablet_manager.cc:1916] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 08:03:33.466056 32209 ts_tablet_manager.cc:1929] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260812 08:03:33.466127 32209 log.cc:1199] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: Deleting WAL directory at /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-1/wal/wals/5ca6ae5d0b454aa4a46607ee93c1c186
I20260812 08:03:33.466392 32209 ts_tablet_manager.cc:1950] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 90dda07aa7b141dc9303dc8203667fc3: Deleting consensus metadata
I20260812 08:03:33.467659 31084 catalog_manager.cc:5050] TS 90dda07aa7b141dc9303dc8203667fc3 (127.18.42.66:35775): tablet 5ca6ae5d0b454aa4a46607ee93c1c186 (table TestTable [id=aedb7b3b9ce14dad8fed8c21ebece2c1]) successfully deleted
W20260812 08:03:33.553382 32204 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 08:03:33.553701 32204 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:33.553769 32204 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:33.553851 32204 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:33.557260 32204 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:33.557420 32204 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.42.67
I20260812 08:03:33.561254 32204 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.42.84:42841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.42.67:37839
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.18.42.67
--webserver_port=41223
--tserver_master_addrs=127.18.42.126:45841
--never_fsync=true
--heap_profile_path=/tmp/kudu.32204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.42.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:33.562484 32204 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:33.563596 32204 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:33.569790 32217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:33.569795 32220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:33.569795 32218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:33.569911 32204 server_base.cc:1061] running on GCE node
I20260812 08:03:33.570485 32204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:33.571085 32204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:33.572264 32204 hybrid_clock.cc:648] HybridClock initialized: now 1786521813572265 us; error 47 us; skew 500 ppm
I20260812 08:03:33.574020 32204 webserver.cc:533] Webserver started at http://127.18.42.67:41223/ using document root <none> and password file <none>
I20260812 08:03:33.574719 32204 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:33.574855 32204 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:33.578397 32204 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:03:33.580360 32233 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:33.581326 32204 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 08:03:33.581465 32204 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "1fb57828e6764314aaf7408ad4ef66b2"
format_stamp: "Formatted at 2026-08-12 08:03:27 on dist-test-slave-zr1t"
I20260812 08:03:33.581920 32204 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:33.612659 32204 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:33.613374 32204 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:33.613624 32204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:33.614266 32204 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:33.615515 32244 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:03:33.618796 32204 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:33.618881 32204 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:03:33.618983 32204 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:33.620492 32204 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:33.620563 32204 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:03:33.620958 32244 tablet_bootstrap.cc:492] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2: Bootstrap starting.
I20260812 08:03:33.632649 32204 rpc_server.cc:307] RPC server started. Bound to: 127.18.42.67:37839
I20260812 08:03:33.632714 32427 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.42.67:37839 every 8 connection(s)
I20260812 08:03:33.633723 32204 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:03:33.637143 32244 log.cc:826] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:33.639783 32244 tablet_bootstrap.cc:492] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260812 08:03:33.640219 32244 tablet_bootstrap.cc:492] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2: Bootstrap complete.
I20260812 08:03:33.640607 18601 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu as pid 32204
I20260812 08:03:33.641896 32244 ts_tablet_manager.cc:1403] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.007s
I20260812 08:03:33.642625 32430 heartbeater.cc:344] Connected to a master server at 127.18.42.126:45841
I20260812 08:03:33.642851 32430 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:33.643414 32430 heartbeater.cc:507] Master 127.18.42.126:45841 requested a full tablet report, sending...
I20260812 08:03:33.644953 32244 raft_consensus.cc:359] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } }
I20260812 08:03:33.645284 31106 ts_manager.cc:194] Re-registered known tserver with Master: 1fb57828e6764314aaf7408ad4ef66b2 (127.18.42.67:37839)
I20260812 08:03:33.645562 32244 raft_consensus.cc:886] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2 [term 1 FOLLOWER]: Ignoring setting pending config change with OpId 1.2 because the committed config has OpId index 2. The config change we are ignoring is: Old config: { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90dda07aa7b141dc9303dc8203667fc3" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 35775 } } peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } } }. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } } }
I20260812 08:03:33.645797 32244 raft_consensus.cc:740] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fb57828e6764314aaf7408ad4ef66b2, State: Initialized, Role: FOLLOWER
I20260812 08:03:33.646152 32244 consensus_queue.cc:260] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } }
I20260812 08:03:33.646268 31106 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.42.67:35167
I20260812 08:03:33.647109 32244 ts_tablet_manager.cc:1434] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:03:33.648296 32430 heartbeater.cc:499] Master 127.18.42.126:45841 was elected leader, sending a full tablet report...
I20260812 08:03:33.901237 32313 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ca6ae5d0b454aa4a46607ee93c1c186 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 08:03:33 UTC) from {username='slave'} at 127.0.0.1:40312
I20260812 08:03:33.902236 32441 tablet_replica.cc:333] stopping tablet replica
I20260812 08:03:33.902536 32441 raft_consensus.cc:2243] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 08:03:33.902746 32441 pending_rounds.cc:70] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2: Trying to abort 1 pending ops.
I20260812 08:03:33.902830 32441 pending_rounds.cc:77] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7317593325990813696 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "5ca6ae5d0b454aa4a46607ee93c1c186" old_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90dda07aa7b141dc9303dc8203667fc3" member_type: VOTER last_known_addr { host: "127.18.42.66" port: 35775 } } peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } } } new_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6ad3f890a7a943de85f5ae307f5d103c" member_type: VOTER last_known_addr { host: "127.18.42.65" port: 45017 } } peers { permanent_uuid: "1fb57828e6764314aaf7408ad4ef66b2" member_type: VOTER last_known_addr { host: "127.18.42.67" port: 37839 } } } }
I20260812 08:03:33.902974 32441 raft_consensus.cc:2925] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2 [term 1 FOLLOWER]: Skipping abort of non-pending config change with OpId 1.2: Aborted: Op aborted
I20260812 08:03:33.903016 32441 raft_consensus.cc:2889] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260812 08:03:33.903059 32441 raft_consensus.cc:2272] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 08:03:33.904052 32441 ts_tablet_manager.cc:1916] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 08:03:33.907126 32441 ts_tablet_manager.cc:1929] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 08:03:33.907208 32441 log.cc:1199] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2: Deleting WAL directory at /tmp/dist-test-taskGfvRiA/test-tmp/raft_consensus-itest.5.RaftConsensusParamReplicationModesITest.Test_KUDU_1735_1.1786521777590645-18601-0/raft_consensus-itest-cluster/ts-2/wal/wals/5ca6ae5d0b454aa4a46607ee93c1c186
I20260812 08:03:33.907470 32441 ts_tablet_manager.cc:1950] T 5ca6ae5d0b454aa4a46607ee93c1c186 P 1fb57828e6764314aaf7408ad4ef66b2: Deleting consensus metadata
I20260812 08:03:33.908665 31084 catalog_manager.cc:5050] TS 1fb57828e6764314aaf7408ad4ef66b2 (127.18.42.67:37839): tablet 5ca6ae5d0b454aa4a46607ee93c1c186 (table TestTable [id=aedb7b3b9ce14dad8fed8c21ebece2c1]) successfully deleted
I20260812 08:03:33.917318 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 31201
I20260812 08:03:33.928081 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 31978
I20260812 08:03:33.945777 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 32204
I20260812 08:03:33.963093 18601 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskGfvRiA/build/debug/bin/kudu with pid 31058
2026-08-12T08:03:33Z chronyd exiting
[       OK ] RaftConsensusParamReplicationModesITest.Test_KUDU_1735/1 (7255 ms)
[----------] 1 test from RaftConsensusParamReplicationModesITest (7255 ms total)

[----------] Global test environment tear-down
[==========] 7 tests from 2 test suites ran. (36393 ms total)
[  PASSED  ] 7 tests.
