[==========] Running 4 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 4 tests from ClientFailoverOnNegotiationTimeoutITest
[ RUN      ] ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer
2026-08-12T06:36:32Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:36:32Z Disabled control of system clock
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260812 06:36:32.286850  3846 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
/tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.193.190:43089
--webserver_interface=127.3.193.190
--webserver_port=0
--builtin_ntp_servers=127.3.193.148:39575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.193.190:43089
--raft_heartbeat_interval_ms=25
--leader_failure_exp_backoff_max_delta_ms=1000 with env {}
W20260812 06:36:32.394554  3859 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:36:32.394763  3859 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:36:32.394800  3859 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:36:32.397976  3859 flags.cc:432] Enabled experimental flag: --leader_failure_exp_backoff_max_delta_ms=1000
W20260812 06:36:32.398033  3859 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:36:32.398053  3859 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:36:32.398072  3859 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:36:32.398087  3859 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:36:32.401805  3859 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.193.148:39575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_exp_backoff_max_delta_ms=1000
--raft_heartbeat_interval_ms=25
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.193.190:43089
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.193.190:43089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.193.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.3859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:36:32.402973  3859 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:32.404175  3859 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:32.409824  3867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:32.409865  3873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:32.409998  3859 server_base.cc:1061] running on GCE node
W20260812 06:36:32.410022  3869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:32.410534  3859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:36:32.411484  3859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:36:32.412675  3859 hybrid_clock.cc:648] HybridClock initialized: now 1786516592412672 us; error 94 us; skew 500 ppm
I20260812 06:36:32.414424  3859 webserver.cc:533] Webserver started at http://127.3.193.190:43899/ using document root <none> and password file <none>
I20260812 06:36:32.414963  3859 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:32.415019  3859 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:32.415174  3859 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:32.416723  3859 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/master-0/data/instance:
uuid: "3539ebbca3264d46b5dc878ba5672cc2"
format_stamp: "Formatted at 2026-08-12 06:36:32 on dist-test-slave-mjjr"
I20260812 06:36:32.417173  3859 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/master-0/wal/instance:
uuid: "3539ebbca3264d46b5dc878ba5672cc2"
format_stamp: "Formatted at 2026-08-12 06:36:32 on dist-test-slave-mjjr"
I20260812 06:36:32.420588  3859 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 06:36:32.423059  3879 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:32.424124  3859 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 06:36:32.424257  3859 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/master-0/data,/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/master-0/wal
uuid: "3539ebbca3264d46b5dc878ba5672cc2"
format_stamp: "Formatted at 2026-08-12 06:36:32 on dist-test-slave-mjjr"
I20260812 06:36:32.424362  3859 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:32.438086  3859 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:32.438791  3859 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:32.439006  3859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:32.447062  3859 rpc_server.cc:307] RPC server started. Bound to: 127.3.193.190:43089
I20260812 06:36:32.447079  3961 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.193.190:43089 every 8 connection(s)
I20260812 06:36:32.448210  3859 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/master-0/data/info.pb
I20260812 06:36:32.451267  3962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:32.452891  3846 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu as pid 3859
I20260812 06:36:32.453070  3846 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/master-0/wal/instance
I20260812 06:36:32.456581  3962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3539ebbca3264d46b5dc878ba5672cc2: Bootstrap starting.
I20260812 06:36:32.459414  3962 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3539ebbca3264d46b5dc878ba5672cc2: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:32.460358  3962 log.cc:826] T 00000000000000000000000000000000 P 3539ebbca3264d46b5dc878ba5672cc2: Log is configured to *not* fsync() on all Append() calls
I20260812 06:36:32.462440  3962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3539ebbca3264d46b5dc878ba5672cc2: No bootstrap required, opened a new log
I20260812 06:36:32.465731  3962 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3539ebbca3264d46b5dc878ba5672cc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3539ebbca3264d46b5dc878ba5672cc2" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 43089 } }
I20260812 06:36:32.465956  3962 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3539ebbca3264d46b5dc878ba5672cc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:32.466022  3962 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3539ebbca3264d46b5dc878ba5672cc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3539ebbca3264d46b5dc878ba5672cc2, State: Initialized, Role: FOLLOWER
I20260812 06:36:32.466500  3962 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3539ebbca3264d46b5dc878ba5672cc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3539ebbca3264d46b5dc878ba5672cc2" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 43089 } }
I20260812 06:36:32.466645  3962 raft_consensus.cc:399] T 00000000000000000000000000000000 P 3539ebbca3264d46b5dc878ba5672cc2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:36:32.466741  3962 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3539ebbca3264d46b5dc878ba5672cc2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:36:32.466908  3962 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3539ebbca3264d46b5dc878ba5672cc2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:32.467983  3962 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3539ebbca3264d46b5dc878ba5672cc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3539ebbca3264d46b5dc878ba5672cc2" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 43089 } }
I20260812 06:36:32.468334  3962 leader_election.cc:304] T 00000000000000000000000000000000 P 3539ebbca3264d46b5dc878ba5672cc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3539ebbca3264d46b5dc878ba5672cc2; no voters: 
I20260812 06:36:32.468669  3962 leader_election.cc:290] T 00000000000000000000000000000000 P 3539ebbca3264d46b5dc878ba5672cc2 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 06:36:32.468770  3972 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3539ebbca3264d46b5dc878ba5672cc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:36:32.468961  3972 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3539ebbca3264d46b5dc878ba5672cc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3539ebbca3264d46b5dc878ba5672cc2, State: Running, Role: LEADER
I20260812 06:36:32.469417  3972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3539ebbca3264d46b5dc878ba5672cc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3539ebbca3264d46b5dc878ba5672cc2" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 43089 } }
I20260812 06:36:32.470005  3962 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3539ebbca3264d46b5dc878ba5672cc2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:36:32.470993  3973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3539ebbca3264d46b5dc878ba5672cc2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3539ebbca3264d46b5dc878ba5672cc2. Latest consensus state: current_term: 1 leader_uuid: "3539ebbca3264d46b5dc878ba5672cc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3539ebbca3264d46b5dc878ba5672cc2" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 43089 } } }
I20260812 06:36:32.471140  3973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3539ebbca3264d46b5dc878ba5672cc2 [sys.catalog]: This master's current role is: LEADER
I20260812 06:36:32.471747  3974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3539ebbca3264d46b5dc878ba5672cc2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3539ebbca3264d46b5dc878ba5672cc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3539ebbca3264d46b5dc878ba5672cc2" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 43089 } } }
I20260812 06:36:32.471861  3974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3539ebbca3264d46b5dc878ba5672cc2 [sys.catalog]: This master's current role is: LEADER
W20260812 06:36:32.474989  4001 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 3539ebbca3264d46b5dc878ba5672cc2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:36:32.475150  4001 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:36:32.475286  3991 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:36:32.476269  3991 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:36:32.481798  3991 catalog_manager.cc:1383] Generated new cluster ID: 3a5761717aa442a1b417a7e927d4c3b7
I20260812 06:36:32.481866  3991 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:36:32.501976  3991 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 06:36:32.503039  3991 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:36:32.509608  3991 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 3539ebbca3264d46b5dc878ba5672cc2: Generated new TSK 0
I20260812 06:36:32.510180  3991 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:36:32.520378  3846 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
/tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.193.129:0
--local_ip_for_outbound_sockets=127.3.193.129
--webserver_interface=127.3.193.129
--webserver_port=0
--tserver_master_addrs=127.3.193.190:43089
--builtin_ntp_servers=127.3.193.148:39575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=25
--leader_failure_exp_backoff_max_delta_ms=1000
--heartbeat_interval_ms=25
--enable_txn_system_client_init=true with env {}
W20260812 06:36:32.627987  4006 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:36:32.628211  4006 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:36:32.628261  4006 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:36:32.631844  4006 flags.cc:432] Enabled experimental flag: --leader_failure_exp_backoff_max_delta_ms=1000
W20260812 06:36:32.631913  4006 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:36:32.631987  4006 flags.cc:432] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:36:32.632098  4006 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.193.129
I20260812 06:36:32.636001  4006 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.193.148:39575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_exp_backoff_max_delta_ms=1000
--raft_heartbeat_interval_ms=25
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.193.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.193.129
--webserver_port=0
--enable_txn_system_client_init=true
--heartbeat_interval_ms=25
--tserver_master_addrs=127.3.193.190:43089
--never_fsync=true
--heap_profile_path=/tmp/kudu.4006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.193.129

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:36:32.637084  4006 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:32.638202  4006 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 06:36:32.640964  4006 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 12
W20260812 06:36:32.644977  4017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:32.644981  4018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:32.645172  4006 server_base.cc:1061] running on GCE node
W20260812 06:36:32.644982  4022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:32.645726  4006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:36:32.646322  4006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:36:32.647511  4006 hybrid_clock.cc:648] HybridClock initialized: now 1786516592647507 us; error 35 us; skew 500 ppm
I20260812 06:36:32.649420  4006 webserver.cc:533] Webserver started at http://127.3.193.129:42721/ using document root <none> and password file <none>
I20260812 06:36:32.650023  4006 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:32.650115  4006 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:32.650348  4006 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:32.651983  4006 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-0/data/instance:
uuid: "c8af5f280df94129ba27e02fff9356a3"
format_stamp: "Formatted at 2026-08-12 06:36:32 on dist-test-slave-mjjr"
I20260812 06:36:32.652480  4006 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-0/wal/instance:
uuid: "c8af5f280df94129ba27e02fff9356a3"
format_stamp: "Formatted at 2026-08-12 06:36:32 on dist-test-slave-mjjr"
I20260812 06:36:32.655844  4006 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 06:36:32.657960  4031 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:32.658990  4006 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:36:32.659142  4006 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-0/wal
uuid: "c8af5f280df94129ba27e02fff9356a3"
format_stamp: "Formatted at 2026-08-12 06:36:32 on dist-test-slave-mjjr"
I20260812 06:36:32.659255  4006 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:32.673194  4006 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:32.673872  4006 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:32.674090  4006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:32.675913  4006 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:36:32.675994  4006 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:32.676074  4006 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:36:32.676120  4006 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:32.689803  4006 rpc_server.cc:307] RPC server started. Bound to: 127.3.193.129:40455
I20260812 06:36:32.689898  4212 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.193.129:40455 every 8 connection(s)
I20260812 06:36:32.690843  4006 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-0/data/info.pb
I20260812 06:36:32.692885  4213 heartbeater.cc:344] Connected to a master server at 127.3.193.190:43089
I20260812 06:36:32.693086  4213 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:32.693570  4213 heartbeater.cc:507] Master 127.3.193.190:43089 requested a full tablet report, sending...
I20260812 06:36:32.695264  3906 ts_manager.cc:194] Registered new tserver with Master: c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:36:32.696645  3906 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.3.193.129:60325
I20260812 06:36:32.696918  3846 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu as pid 4006
I20260812 06:36:32.697048  3846 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-0/wal/instance
I20260812 06:36:32.702724  3846 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
/tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.193.130:0
--local_ip_for_outbound_sockets=127.3.193.130
--webserver_interface=127.3.193.130
--webserver_port=0
--tserver_master_addrs=127.3.193.190:43089
--builtin_ntp_servers=127.3.193.148:39575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=25
--leader_failure_exp_backoff_max_delta_ms=1000
--heartbeat_interval_ms=25
--enable_txn_system_client_init=true with env {}
I20260812 06:36:32.724007  4213 heartbeater.cc:499] Master 127.3.193.190:43089 was elected leader, sending a full tablet report...
W20260812 06:36:32.814949  4219 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:36:32.815228  4219 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:36:32.815310  4219 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:36:32.818832  4219 flags.cc:432] Enabled experimental flag: --leader_failure_exp_backoff_max_delta_ms=1000
W20260812 06:36:32.818924  4219 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:36:32.819016  4219 flags.cc:432] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:36:32.819105  4219 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.193.130
I20260812 06:36:32.822911  4219 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.193.148:39575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_exp_backoff_max_delta_ms=1000
--raft_heartbeat_interval_ms=25
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.193.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.193.130
--webserver_port=0
--enable_txn_system_client_init=true
--heartbeat_interval_ms=25
--tserver_master_addrs=127.3.193.190:43089
--never_fsync=true
--heap_profile_path=/tmp/kudu.4219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.193.130

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:36:32.824112  4219 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:32.825271  4219 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 06:36:32.828068  4219 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 12
W20260812 06:36:32.831883  4232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:32.831874  4228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:32.831907  4229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:32.832162  4219 server_base.cc:1061] running on GCE node
I20260812 06:36:32.832619  4219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:36:32.833207  4219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:36:32.834367  4219 hybrid_clock.cc:648] HybridClock initialized: now 1786516592834348 us; error 30 us; skew 500 ppm
I20260812 06:36:32.836189  4219 webserver.cc:533] Webserver started at http://127.3.193.130:39713/ using document root <none> and password file <none>
I20260812 06:36:32.836737  4219 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:32.836791  4219 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:32.836948  4219 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:32.838505  4219 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-1/data/instance:
uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
format_stamp: "Formatted at 2026-08-12 06:36:32 on dist-test-slave-mjjr"
I20260812 06:36:32.838953  4219 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-1/wal/instance:
uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
format_stamp: "Formatted at 2026-08-12 06:36:32 on dist-test-slave-mjjr"
I20260812 06:36:32.842348  4219 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 06:36:32.844674  4240 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:32.845647  4219 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 06:36:32.845780  4219 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-1/wal
uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
format_stamp: "Formatted at 2026-08-12 06:36:32 on dist-test-slave-mjjr"
I20260812 06:36:32.845882  4219 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:32.869669  4219 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:32.870448  4219 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:32.870671  4219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:32.872707  4219 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:36:32.872781  4219 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:32.872845  4219 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:36:32.872880  4219 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:32.885411  4219 rpc_server.cc:307] RPC server started. Bound to: 127.3.193.130:35779
I20260812 06:36:32.886541  4219 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-1/data/info.pb
I20260812 06:36:32.887506  4458 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.193.130:35779 every 8 connection(s)
I20260812 06:36:32.888123  4459 heartbeater.cc:344] Connected to a master server at 127.3.193.190:43089
I20260812 06:36:32.888288  4459 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:32.888737  4459 heartbeater.cc:507] Master 127.3.193.190:43089 requested a full tablet report, sending...
I20260812 06:36:32.889721  3906 ts_manager.cc:194] Registered new tserver with Master: df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:36:32.890023  3846 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu as pid 4219
I20260812 06:36:32.890120  3846 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-1/wal/instance
I20260812 06:36:32.890288  3906 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.3.193.130:34739
I20260812 06:36:32.895319  3846 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
/tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.193.131:0
--local_ip_for_outbound_sockets=127.3.193.131
--webserver_interface=127.3.193.131
--webserver_port=0
--tserver_master_addrs=127.3.193.190:43089
--builtin_ntp_servers=127.3.193.148:39575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=25
--leader_failure_exp_backoff_max_delta_ms=1000
--heartbeat_interval_ms=25
--enable_txn_system_client_init=true with env {}
I20260812 06:36:32.917327  4459 heartbeater.cc:499] Master 127.3.193.190:43089 was elected leader, sending a full tablet report...
W20260812 06:36:33.007764  4461 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:36:33.008025  4461 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:36:33.008107  4461 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:36:33.011634  4461 flags.cc:432] Enabled experimental flag: --leader_failure_exp_backoff_max_delta_ms=1000
W20260812 06:36:33.011785  4461 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:36:33.011864  4461 flags.cc:432] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:36:33.011965  4461 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.193.131
I20260812 06:36:33.015911  4461 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.193.148:39575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_exp_backoff_max_delta_ms=1000
--raft_heartbeat_interval_ms=25
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.193.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.193.131
--webserver_port=0
--enable_txn_system_client_init=true
--heartbeat_interval_ms=25
--tserver_master_addrs=127.3.193.190:43089
--never_fsync=true
--heap_profile_path=/tmp/kudu.4461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.193.131

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:36:33.017058  4461 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:33.018189  4461 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 06:36:33.021318  4461 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 12
W20260812 06:36:33.025133  4475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:33.025205  4478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:33.025147  4472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:33.026245  4461 server_base.cc:1061] running on GCE node
I20260812 06:36:33.026628  4461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:36:33.027154  4461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:36:33.028339  4461 hybrid_clock.cc:648] HybridClock initialized: now 1786516593028304 us; error 34 us; skew 500 ppm
I20260812 06:36:33.030350  4461 webserver.cc:533] Webserver started at http://127.3.193.131:33319/ using document root <none> and password file <none>
I20260812 06:36:33.030910  4461 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:33.030963  4461 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:33.031141  4461 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:33.032871  4461 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-2/data/instance:
uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
format_stamp: "Formatted at 2026-08-12 06:36:33 on dist-test-slave-mjjr"
I20260812 06:36:33.033309  4461 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-2/wal/instance:
uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
format_stamp: "Formatted at 2026-08-12 06:36:33 on dist-test-slave-mjjr"
I20260812 06:36:33.036691  4461 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.000s	sys 0.004s
I20260812 06:36:33.038746  4486 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:33.039920  4461 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:36:33.040055  4461 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-2/wal
uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
format_stamp: "Formatted at 2026-08-12 06:36:33 on dist-test-slave-mjjr"
I20260812 06:36:33.040158  4461 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:33.070587  4461 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:33.071463  4461 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:33.071732  4461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:33.073855  4461 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:36:33.073933  4461 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:33.074026  4461 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:36:33.074090  4461 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:33.087579  4461 rpc_server.cc:307] RPC server started. Bound to: 127.3.193.131:38503
I20260812 06:36:33.088048  4677 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.193.131:38503 every 8 connection(s)
I20260812 06:36:33.088650  4461 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-2/data/info.pb
I20260812 06:36:33.090579  4678 heartbeater.cc:344] Connected to a master server at 127.3.193.190:43089
I20260812 06:36:33.090762  4678 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:33.091295  4678 heartbeater.cc:507] Master 127.3.193.190:43089 requested a full tablet report, sending...
I20260812 06:36:33.092365  3906 ts_manager.cc:194] Registered new tserver with Master: d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:33.092535  3846 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu as pid 4461
I20260812 06:36:33.092644  3846 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1786516592259368-3846-0/minicluster-data/ts-2/wal/instance
I20260812 06:36:33.093058  3906 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.3.193.131:40563
I20260812 06:36:33.102044  3846 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 06:36:33.116737  3906 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36028:
name: "kudu1580"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "val"
    type: INT32
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  hash_schema {
    columns {
      name: "key"
    }
    num_buckets: 3
    seed: 0
  }
}
W20260812 06:36:33.119527  3906 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu1580 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:36:33.120757  4678 heartbeater.cc:499] Master 127.3.193.190:43089 was elected leader, sending a full tablet report...
I20260812 06:36:33.139904  4086 tablet_service.cc:1511] Processing CreateTablet for tablet 3cf285c378e04319a5acdcae46870291 (DEFAULT_TABLE table=kudu1580 [id=8fd5343ae0c94ccdb88b4142bfd83301]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260812 06:36:33.139696  4311 tablet_service.cc:1511] Processing CreateTablet for tablet 30fb1aacee684c4eacf9e756aa9927fc (DEFAULT_TABLE table=kudu1580 [id=8fd5343ae0c94ccdb88b4142bfd83301]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260812 06:36:33.139696  4313 tablet_service.cc:1511] Processing CreateTablet for tablet 3cf285c378e04319a5acdcae46870291 (DEFAULT_TABLE table=kudu1580 [id=8fd5343ae0c94ccdb88b4142bfd83301]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260812 06:36:33.140798  4313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cf285c378e04319a5acdcae46870291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:33.140798  4086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cf285c378e04319a5acdcae46870291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:33.141167  4311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30fb1aacee684c4eacf9e756aa9927fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:33.143913  4102 tablet_service.cc:1511] Processing CreateTablet for tablet 30fb1aacee684c4eacf9e756aa9927fc (DEFAULT_TABLE table=kudu1580 [id=8fd5343ae0c94ccdb88b4142bfd83301]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260812 06:36:33.143975  4103 tablet_service.cc:1511] Processing CreateTablet for tablet 27b60a3088e74447b13844f7b49da4ff (DEFAULT_TABLE table=kudu1580 [id=8fd5343ae0c94ccdb88b4142bfd83301]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260812 06:36:33.144213  4102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30fb1aacee684c4eacf9e756aa9927fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:33.144395  4314 tablet_service.cc:1511] Processing CreateTablet for tablet 27b60a3088e74447b13844f7b49da4ff (DEFAULT_TABLE table=kudu1580 [id=8fd5343ae0c94ccdb88b4142bfd83301]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260812 06:36:33.144652  4314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27b60a3088e74447b13844f7b49da4ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:33.146927  4103 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27b60a3088e74447b13844f7b49da4ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:33.148065  4700 tablet_bootstrap.cc:492] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85: Bootstrap starting.
I20260812 06:36:33.150401  4700 tablet_bootstrap.cc:654] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:33.150720  4564 tablet_service.cc:1511] Processing CreateTablet for tablet 27b60a3088e74447b13844f7b49da4ff (DEFAULT_TABLE table=kudu1580 [id=8fd5343ae0c94ccdb88b4142bfd83301]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260812 06:36:33.151655  4700 log.cc:826] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85: Log is configured to *not* fsync() on all Append() calls
I20260812 06:36:33.151888  4562 tablet_service.cc:1511] Processing CreateTablet for tablet 30fb1aacee684c4eacf9e756aa9927fc (DEFAULT_TABLE table=kudu1580 [id=8fd5343ae0c94ccdb88b4142bfd83301]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260812 06:36:33.152393  4563 tablet_service.cc:1511] Processing CreateTablet for tablet 3cf285c378e04319a5acdcae46870291 (DEFAULT_TABLE table=kudu1580 [id=8fd5343ae0c94ccdb88b4142bfd83301]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260812 06:36:33.152920  4564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27b60a3088e74447b13844f7b49da4ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:33.153586  4562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30fb1aacee684c4eacf9e756aa9927fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:33.154517  4708 tablet_bootstrap.cc:492] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3: Bootstrap starting.
I20260812 06:36:33.157366  4700 tablet_bootstrap.cc:492] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85: No bootstrap required, opened a new log
I20260812 06:36:33.157567  4700 ts_tablet_manager.cc:1403] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85: Time spent bootstrapping tablet: real 0.010s	user 0.004s	sys 0.000s
I20260812 06:36:33.157850  4563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cf285c378e04319a5acdcae46870291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:33.158720  4708 tablet_bootstrap.cc:654] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:33.161057  4709 tablet_bootstrap.cc:492] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2: Bootstrap starting.
I20260812 06:36:33.161356  4700 raft_consensus.cc:359] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:33.161621  4700 raft_consensus.cc:385] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:33.161692  4708 log.cc:826] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3: Log is configured to *not* fsync() on all Append() calls
I20260812 06:36:33.161748  4700 raft_consensus.cc:740] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df4e0103d6e942fb850d4f18b1cd2b85, State: Initialized, Role: FOLLOWER
I20260812 06:36:33.162348  4700 consensus_queue.cc:260] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:33.163355  4700 ts_tablet_manager.cc:1434] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260812 06:36:33.163690  4709 tablet_bootstrap.cc:654] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:33.163807  4708 tablet_bootstrap.cc:492] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3: No bootstrap required, opened a new log
I20260812 06:36:33.163797  4700 tablet_bootstrap.cc:492] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85: Bootstrap starting.
I20260812 06:36:33.164036  4708 ts_tablet_manager.cc:1403] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3: Time spent bootstrapping tablet: real 0.010s	user 0.006s	sys 0.000s
I20260812 06:36:33.164994  4709 log.cc:826] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2: Log is configured to *not* fsync() on all Append() calls
I20260812 06:36:33.165323  4700 tablet_bootstrap.cc:654] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:33.166702  4709 tablet_bootstrap.cc:492] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2: No bootstrap required, opened a new log
I20260812 06:36:33.166879  4709 ts_tablet_manager.cc:1403] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2: Time spent bootstrapping tablet: real 0.006s	user 0.005s	sys 0.000s
I20260812 06:36:33.167327  4708 raft_consensus.cc:359] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:33.167522  4708 raft_consensus.cc:385] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:33.167584  4708 raft_consensus.cc:740] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8af5f280df94129ba27e02fff9356a3, State: Initialized, Role: FOLLOWER
I20260812 06:36:33.168152  4708 consensus_queue.cc:260] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:33.169072  4708 ts_tablet_manager.cc:1434] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 06:36:33.169435  4708 tablet_bootstrap.cc:492] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3: Bootstrap starting.
I20260812 06:36:33.169965  4709 raft_consensus.cc:359] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:33.170194  4709 raft_consensus.cc:385] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:33.170303  4709 raft_consensus.cc:740] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8c36d75aa834cca96b042eb3bf8a6d2, State: Initialized, Role: FOLLOWER
I20260812 06:36:33.170596  4708 tablet_bootstrap.cc:654] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:33.170751  4709 consensus_queue.cc:260] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:33.171805  4709 ts_tablet_manager.cc:1434] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 06:36:33.173139  4709 tablet_bootstrap.cc:492] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2: Bootstrap starting.
I20260812 06:36:33.175223  4709 tablet_bootstrap.cc:654] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:33.176537  4709 tablet_bootstrap.cc:492] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2: No bootstrap required, opened a new log
I20260812 06:36:33.176537  4708 tablet_bootstrap.cc:492] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3: No bootstrap required, opened a new log
I20260812 06:36:33.176630  4709 ts_tablet_manager.cc:1403] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2: Time spent bootstrapping tablet: real 0.004s	user 0.002s	sys 0.000s
I20260812 06:36:33.176630  4708 ts_tablet_manager.cc:1403] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3: Time spent bootstrapping tablet: real 0.007s	user 0.002s	sys 0.000s
I20260812 06:36:33.176812  4700 tablet_bootstrap.cc:492] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85: No bootstrap required, opened a new log
I20260812 06:36:33.176947  4700 ts_tablet_manager.cc:1403] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85: Time spent bootstrapping tablet: real 0.013s	user 0.002s	sys 0.000s
I20260812 06:36:33.177100  4708 raft_consensus.cc:359] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:33.177093  4709 raft_consensus.cc:359] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:33.177207  4708 raft_consensus.cc:385] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:33.177207  4709 raft_consensus.cc:385] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:33.177242  4708 raft_consensus.cc:740] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8af5f280df94129ba27e02fff9356a3, State: Initialized, Role: FOLLOWER
I20260812 06:36:33.177242  4709 raft_consensus.cc:740] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8c36d75aa834cca96b042eb3bf8a6d2, State: Initialized, Role: FOLLOWER
I20260812 06:36:33.177376  4708 consensus_queue.cc:260] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:33.177376  4709 consensus_queue.cc:260] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:33.177460  4700 raft_consensus.cc:359] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:33.177573  4700 raft_consensus.cc:385] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:33.177608  4700 raft_consensus.cc:740] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df4e0103d6e942fb850d4f18b1cd2b85, State: Initialized, Role: FOLLOWER
I20260812 06:36:33.177636  4709 ts_tablet_manager.cc:1434] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.000s
I20260812 06:36:33.177636  4708 ts_tablet_manager.cc:1434] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20260812 06:36:33.177732  4700 consensus_queue.cc:260] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:33.177802  4708 tablet_bootstrap.cc:492] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3: Bootstrap starting.
I20260812 06:36:33.177802  4709 tablet_bootstrap.cc:492] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2: Bootstrap starting.
I20260812 06:36:33.178016  4700 ts_tablet_manager.cc:1434] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20260812 06:36:33.178197  4700 tablet_bootstrap.cc:492] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85: Bootstrap starting.
I20260812 06:36:33.178776  4708 tablet_bootstrap.cc:654] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:33.178776  4709 tablet_bootstrap.cc:654] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:33.179219  4700 tablet_bootstrap.cc:654] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:33.179996  4708 tablet_bootstrap.cc:492] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3: No bootstrap required, opened a new log
I20260812 06:36:33.180099  4708 ts_tablet_manager.cc:1403] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3: Time spent bootstrapping tablet: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:36:33.180578  4708 raft_consensus.cc:359] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:33.180677  4708 raft_consensus.cc:385] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:33.180718  4708 raft_consensus.cc:740] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8af5f280df94129ba27e02fff9356a3, State: Initialized, Role: FOLLOWER
I20260812 06:36:33.180843  4708 consensus_queue.cc:260] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:33.181108  4708 ts_tablet_manager.cc:1434] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.000s
I20260812 06:36:33.181532  4700 tablet_bootstrap.cc:492] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85: No bootstrap required, opened a new log
I20260812 06:36:33.181615  4700 ts_tablet_manager.cc:1403] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85: Time spent bootstrapping tablet: real 0.003s	user 0.002s	sys 0.000s
I20260812 06:36:33.181644  4709 tablet_bootstrap.cc:492] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2: No bootstrap required, opened a new log
I20260812 06:36:33.181715  4709 ts_tablet_manager.cc:1403] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2: Time spent bootstrapping tablet: real 0.004s	user 0.002s	sys 0.000s
I20260812 06:36:33.182066  4700 raft_consensus.cc:359] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:33.182168  4700 raft_consensus.cc:385] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:33.182202  4700 raft_consensus.cc:740] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df4e0103d6e942fb850d4f18b1cd2b85, State: Initialized, Role: FOLLOWER
I20260812 06:36:33.182111  4709 raft_consensus.cc:359] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:33.182256  4709 raft_consensus.cc:385] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:33.182292  4709 raft_consensus.cc:740] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8c36d75aa834cca96b042eb3bf8a6d2, State: Initialized, Role: FOLLOWER
I20260812 06:36:33.182354  4700 consensus_queue.cc:260] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:33.182420  4709 consensus_queue.cc:260] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:33.182632  4700 ts_tablet_manager.cc:1434] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20260812 06:36:33.182675  4709 ts_tablet_manager.cc:1434] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20260812 06:36:33.183732  4712 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:33.183961  4712 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:33.185277  4717 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:33.185316  4712 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:36:33.185425  4717 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:33.186581  4717 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:33.187212  4712 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:33.187295  4712 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:33.187640  4712 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:33.190804  4587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:33.191084  4587 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 0.
I20260812 06:36:33.191385  4593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:33.191484  4593 raft_consensus.cc:2468] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 0.
I20260812 06:36:33.191967  4717 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:33.191936  4245 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
W20260812 06:36:33.191998  4216 tablet.cc:2406] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:36:33.192049  4717 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:33.192212  4245 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:33.192404  4717 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:33.192643  4712 raft_consensus.cc:2804] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:36:33.192728  4712 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:36:33.192772  4712 raft_consensus.cc:3060] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:33.192780  4718 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:33.192868  4718 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:33.193279  4361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:33.193289  4346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:33.193437  4361 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 0.
I20260812 06:36:33.193467  4346 raft_consensus.cc:2468] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 0.
I20260812 06:36:33.193895  4718 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:36:33.193881  4134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:33.193894  4133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:33.194049  4133 raft_consensus.cc:2468] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 0.
I20260812 06:36:33.194049  4134 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 0.
I20260812 06:36:33.193894  4032 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:33.194422  4717 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:36:33.194480  4717 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:36:33.194465  4032 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:33.194522  4717 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:33.195748  4717 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:33.195780  4593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:33.195884  4593 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 0.
I20260812 06:36:33.196097  4717 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 1 election: Requested vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:33.196224  4717 raft_consensus.cc:2804] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:36:33.196259  4717 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:36:33.196291  4717 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:33.196632  4593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:33.196722  4593 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:33.197113  4717 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:33.197443  4717 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 1 election: Requested vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:33.197784  4593 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 1.
I20260812 06:36:33.198386  4034 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2; no voters: 
I20260812 06:36:33.198622  4587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:33.198733  4587 raft_consensus.cc:2468] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 0.
I20260812 06:36:33.198839  4134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:33.199012  4134 raft_consensus.cc:2393] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8c36d75aa834cca96b042eb3bf8a6d2 in current term 1: Already voted for candidate c8af5f280df94129ba27e02fff9356a3 in this term.
I20260812 06:36:33.199085  4361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:36:33.199155  4593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:33.199183  4361 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:33.199245  4593 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:33.199764  4717 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:36:33.199779  4346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:36:33.199849  4717 raft_consensus.cc:697] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 1 LEADER]: Becoming Leader. State: Replica: c8af5f280df94129ba27e02fff9356a3, State: Running, Role: LEADER
I20260812 06:36:33.199882  4346 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:33.200124  4593 raft_consensus.cc:2468] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 1.
I20260812 06:36:33.200157  4717 consensus_queue.cc:237] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:33.200651  4034 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2; no voters: 
I20260812 06:36:33.200857  4346 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 1.
I20260812 06:36:33.201089  4712 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:33.201288  4740 raft_consensus.cc:2804] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:36:33.201387  4740 raft_consensus.cc:697] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 1 LEADER]: Becoming Leader. State: Replica: c8af5f280df94129ba27e02fff9356a3, State: Running, Role: LEADER
I20260812 06:36:33.201540  4712 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 1 election: Requested vote from peers d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:36:33.201540  4740 consensus_queue.cc:237] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:33.202534  4361 raft_consensus.cc:2468] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 1.
I20260812 06:36:33.202665  4712 raft_consensus.cc:2764] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260812 06:36:33.203140  4134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:33.203246  4134 raft_consensus.cc:3060] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:33.204082  4134 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 1.
I20260812 06:36:33.204424  4244 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:33.204594  4712 raft_consensus.cc:2804] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:36:33.204665  4593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:33.204763  4593 raft_consensus.cc:3060] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:33.205013  4712 raft_consensus.cc:697] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 1 LEADER]: Becoming Leader. State: Replica: df4e0103d6e942fb850d4f18b1cd2b85, State: Running, Role: LEADER
I20260812 06:36:33.205286  4712 consensus_queue.cc:237] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:33.205155  3908 catalog_manager.cc:5719] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8af5f280df94129ba27e02fff9356a3 (127.3.193.129). New cstate: current_term: 1 leader_uuid: "c8af5f280df94129ba27e02fff9356a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: UNKNOWN } } }
I20260812 06:36:33.205636  4593 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 1.
I20260812 06:36:33.207355  4361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:33.207533  4361 raft_consensus.cc:2393] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8c36d75aa834cca96b042eb3bf8a6d2 in current term 1: Already voted for candidate c8af5f280df94129ba27e02fff9356a3 in this term.
I20260812 06:36:33.208179  4489 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:36:33.208606  3905 catalog_manager.cc:5719] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 reported cstate change: term changed from 0 to 1, leader changed from <none> to df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130). New cstate: current_term: 1 leader_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: UNKNOWN } } }
I20260812 06:36:33.209672  4718 raft_consensus.cc:2749] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 06:36:33.210911  3906 catalog_manager.cc:5719] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8af5f280df94129ba27e02fff9356a3 (127.3.193.129). New cstate: current_term: 1 leader_uuid: "c8af5f280df94129ba27e02fff9356a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: UNKNOWN } } }
I20260812 06:36:33.223244  4740 consensus_queue.cc:1048] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:36:33.223965  4740 consensus_queue.cc:1048] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:36:33.227438  4738 consensus_queue.cc:1048] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:36:33.227505  4736 consensus_queue.cc:1048] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:36:33.231243  4717 consensus_queue.cc:1048] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:36:33.233099  4736 consensus_queue.cc:1048] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 06:36:33.788487  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:33.272117 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.130:35779 (local address 127.0.0.1:55686)
0812 06:36:33.272247 (+   130us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:33.272260 (+    13us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:33.272323 (+    63us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:33.787970 (+515647us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 06:36:33.788753  4680 meta_cache.cc:302] tablet 3cf285c378e04319a5acdcae46870291: replica df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
I20260812 06:36:34.264889  4718 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:34.265061  4718 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:34.265336  4777 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:34.265439  4777 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:34.265584  4718 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:36:34.265664  4718 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:34.265733  4718 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:34.265839  4777 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:36:34.266100  4718 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
W20260812 06:36:34.266449  4679 tablet.cc:2406] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:36:34.268252  4781 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:34.268420  4781 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:34.269109  4781 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:34.269377  4781 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:34.269506  4781 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:34.270035  4781 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:34.270259  4361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:34.270352  4361 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8c36d75aa834cca96b042eb3bf8a6d2 in term 1.
I20260812 06:36:34.270416  4346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:34.270709  4355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:34.270823  4355 raft_consensus.cc:2468] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8c36d75aa834cca96b042eb3bf8a6d2 in term 1.
I20260812 06:36:34.271296  4593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:34.271485  4593 raft_consensus.cc:2410] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate df4e0103d6e942fb850d4f18b1cd2b85 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260812 06:36:34.271682  4489 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
W20260812 06:36:34.271783  4460 tablet.cc:2406] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:36:34.271777  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:34.271876  4598 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 1.
I20260812 06:36:34.271912  4489 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:34.272053  4718 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 06:36:34.272095  4718 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:34.272132  4718 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:36:34.272397  4245 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:34.272625  4781 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 06:36:34.272672  4781 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:34.272709  4781 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:36:34.273270  4718 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
W20260812 06:36:34.273411  4694 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:34.268981 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:55686 (local address 127.3.193.130:35779)
0812 06:36:34.272540 (+  3559us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:34.272545 (+     5us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:34.272559 (+    14us) server_negotiation.cc:408] Connection header received
0812 06:36:34.272626 (+    67us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:34.272629 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:34.272687 (+    58us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:34.272838 (+   151us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55686: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":3478}
I20260812 06:36:34.273651  4718 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 2 election: Requested vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:36:34.273756  4718 raft_consensus.cc:2804] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 06:36:34.273808  4718 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:34.273849  4718 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:36:34.273823  4781 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:34.274511  4587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:34.274648  4587 raft_consensus.cc:2393] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate df4e0103d6e942fb850d4f18b1cd2b85 in current term 2: Already voted for candidate d8c36d75aa834cca96b042eb3bf8a6d2 in this term.
I20260812 06:36:34.274746  4718 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:34.274935  4781 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 2 election: Requested vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:34.275029  4718 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 2 election: Requested vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
W20260812 06:36:34.278262  4530 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39992: Illegal state: replica d8c36d75aa834cca96b042eb3bf8a6d2 is not leader of this config: current role FOLLOWER
I20260812 06:36:34.278754  4355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:36:34.278888  4355 raft_consensus.cc:2393] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d8c36d75aa834cca96b042eb3bf8a6d2 in current term 2: Already voted for candidate df4e0103d6e942fb850d4f18b1cd2b85 in this term.
I20260812 06:36:34.279060  4346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:36:34.279141  4346 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:36:34.280114  4346 raft_consensus.cc:2468] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8c36d75aa834cca96b042eb3bf8a6d2 in term 2.
I20260812 06:36:34.280591  4489 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:34.280789  4779 raft_consensus.cc:2804] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 06:36:34.280895  4779 raft_consensus.cc:697] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 2 LEADER]: Becoming Leader. State: Replica: d8c36d75aa834cca96b042eb3bf8a6d2, State: Running, Role: LEADER
I20260812 06:36:34.281198  4779 consensus_queue.cc:237] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:34.283458  3905 catalog_manager.cc:5719] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 reported cstate change: term changed from 1 to 2, leader changed from c8af5f280df94129ba27e02fff9356a3 (127.3.193.129) to d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131). New cstate: current_term: 2 leader_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: HEALTHY } } }
I20260812 06:36:34.304095  4346 raft_consensus.cc:1275] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 2 FOLLOWER]: Refusing update from remote peer d8c36d75aa834cca96b042eb3bf8a6d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 06:36:34.304706  4779 consensus_queue.cc:1048] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 06:36:34.346652  4244 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.076s (SENT)
W20260812 06:36:34.346897  4490 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.080s (SENT)
W20260812 06:36:34.424098  4244 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.076s (SENT)
W20260812 06:36:34.427999  4490 leader_election.cc:336] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.080s (SENT)
W20260812 06:36:34.437513  4490 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.085s (SENT)
W20260812 06:36:34.440866  4490 leader_election.cc:336] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.082s (SENT)
W20260812 06:36:34.451237  4490 leader_election.cc:336] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.092s (SENT)
I20260812 06:36:34.451380  4490 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:36:34.451670  4779 raft_consensus.cc:2749] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 06:36:34.457093  4490 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.090s (SENT)
I20260812 06:36:34.457219  4490 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:36:34.457434  4779 raft_consensus.cc:2749] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260812 06:36:34.463342  4244 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.093s (SENT)
I20260812 06:36:34.463464  4244 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df4e0103d6e942fb850d4f18b1cd2b85; no voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2
I20260812 06:36:34.463730  4783 raft_consensus.cc:2749] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260812 06:36:34.476034  4244 leader_election.cc:336] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.103s (SENT)
I20260812 06:36:34.476132  4244 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df4e0103d6e942fb850d4f18b1cd2b85; no voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2
I20260812 06:36:34.476326  4783 raft_consensus.cc:2749] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 06:36:34.542629  4783 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:34.542776  4783 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:34.543200  4783 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:34.543712  4587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:34.543851  4587 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 2.
I20260812 06:36:34.544123  4245 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:34.544292  4783 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 06:36:34.544354  4783 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:36:34.544384  4783 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:36:34.545260  4783 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:34.545622  4783 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 3 election: Requested vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:34.545990  4587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:34.546115  4587 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:36:34.547015  4587 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 3.
I20260812 06:36:34.547354  4245 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:34.547593  4783 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 06:36:34.547729  4783 raft_consensus.cc:697] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 3 LEADER]: Becoming Leader. State: Replica: df4e0103d6e942fb850d4f18b1cd2b85, State: Running, Role: LEADER
I20260812 06:36:34.547879  4783 consensus_queue.cc:237] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:34.549633  3905 catalog_manager.cc:5719] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 reported cstate change: term changed from 1 to 3, leader changed from c8af5f280df94129ba27e02fff9356a3 (127.3.193.129) to df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130). New cstate: current_term: 3 leader_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: UNKNOWN } } }
I20260812 06:36:34.573957  4587 raft_consensus.cc:1275] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 3 FOLLOWER]: Refusing update from remote peer df4e0103d6e942fb850d4f18b1cd2b85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260812 06:36:34.574575  4783 consensus_queue.cc:1048] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 06:36:34.811954  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:34.284496 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.129:40455 (local address 127.0.0.1:37764)
0812 06:36:34.284626 (+   130us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:34.284649 (+    23us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:34.284722 (+    73us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:34.811849 (+527127us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 06:36:34.812243  4681 meta_cache.cc:302] tablet 3cf285c378e04319a5acdcae46870291: replica c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
W20260812 06:36:34.851248  4244 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.152s (SENT)
W20260812 06:36:34.856568  4244 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.156s (SENT)
W20260812 06:36:35.388063  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:34.884902 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.130:35779 (local address 127.0.0.1:55724)
0812 06:36:34.885075 (+   173us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:34.885090 (+    15us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:34.885175 (+    85us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:35.387954 (+502779us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 06:36:35.885273  4779 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:35.885445  4779 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:35.885963  4779 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:36:35.886040  4825 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:35.886157  4825 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:35.888317  4355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:35.888684  4825 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:36:35.888867  4829 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d8c36d75aa834cca96b042eb3bf8a6d2)
I20260812 06:36:35.889055  4355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:35.888967  4829 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:35.889573  4829 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:35.889923  4587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
W20260812 06:36:35.891398  4827 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:35.887954 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:55724 (local address 127.3.193.130:35779)
0812 06:36:35.889131 (+  1177us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:35.889135 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:35.889149 (+    14us) server_negotiation.cc:408] Connection header received
0812 06:36:35.889190 (+    41us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:35.889193 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:35.889469 (+   276us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:35.891276 (+  1807us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55724: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":1100,"thread_start_us":61,"threads_started":1}
W20260812 06:36:35.903940  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:35.392246 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.129:40455 (local address 127.0.0.1:37776)
0812 06:36:35.392456 (+   210us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:35.392478 (+    22us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:35.392598 (+   120us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:35.903848 (+511250us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":75}
W20260812 06:36:35.904098  4681 meta_cache.cc:302] tablet 30fb1aacee684c4eacf9e756aa9927fc: replica c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
W20260812 06:36:35.912469  4530 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41046: Illegal state: replica d8c36d75aa834cca96b042eb3bf8a6d2 is not leader of this config: current role FOLLOWER
W20260812 06:36:36.476109  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:35.975163 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.131:38503 (local address 127.0.0.1:41064)
0812 06:36:35.975342 (+   179us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:35.975359 (+    17us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:35.975451 (+    92us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:36.475995 (+500544us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.131:38503: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 06:36:36.980669  4832 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:36.976230 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:41064 (local address 127.3.193.131:38503)
0812 06:36:36.979773 (+  3543us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:36.979777 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:36.979793 (+    16us) server_negotiation.cc:408] Connection header received
0812 06:36:36.979841 (+    48us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:36.979847 (+     6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:36.979924 (+    77us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:36.980072 (+   148us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41064: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":3434}
W20260812 06:36:36.987974  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:36.481040 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.129:40455 (local address 127.0.0.1:51392)
0812 06:36:36.481220 (+   180us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:36.481237 (+    17us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:36.481329 (+    92us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:36.987861 (+506532us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":66}
W20260812 06:36:36.988157  4681 meta_cache.cc:302] tablet 27b60a3088e74447b13844f7b49da4ff: replica c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
W20260812 06:36:36.995784  4285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47524: Illegal state: replica df4e0103d6e942fb850d4f18b1cd2b85 is not leader of this config: current role FOLLOWER
W20260812 06:36:37.564071  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:37.056246 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.130:35779 (local address 127.0.0.1:47536)
0812 06:36:37.056424 (+   178us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:37.056438 (+    14us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:37.056518 (+    80us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:37.563943 (+507425us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 06:36:38.057782  4738 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:38.058322  4133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
W20260812 06:36:38.058485  4034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.193.131:38503, user_credentials={real_user=slave}} blocked reactor thread for 4.82739e+06us
I20260812 06:36:38.059152  4130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
W20260812 06:36:38.059577  4132 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.3.193.130:47027 (request call id 5) took 1034 ms (client timeout 103 ms). Trace:
W20260812 06:36:38.059679  4132 rpcz_store.cc:269] 0812 06:36:38.058545 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:38.058825 (+   280us) service_pool.cc:211] Skipping call since client already timed out
0812 06:36:38.058937 (+   112us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260812 06:36:38.060142  4244 leader_election.cc:336] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.095s (SENT)
I20260812 06:36:38.057924  4738 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:38.060235  4244 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df4e0103d6e942fb850d4f18b1cd2b85; no voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2
I20260812 06:36:38.060549  4738 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:38.061079  4133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:38.061419  4346 raft_consensus.cc:1217] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.1]   Dedup: 1.1->[]
I20260812 06:36:38.061555  4346 raft_consensus.cc:1240] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 2 FOLLOWER]: Rejecting Update request from peer c8af5f280df94129ba27e02fff9356a3 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260812 06:36:38.061954  4829 raft_consensus.cc:2749] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 06:36:38.063160  4118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:38.063397  4125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:38.063788  4122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:38.064101  4127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:38.064381  4128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:38.064710  4126 raft_consensus.cc:3055] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 06:36:38.064782  4126 raft_consensus.cc:740] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c8af5f280df94129ba27e02fff9356a3, State: Running, Role: LEADER
I20260812 06:36:38.064981  4126 consensus_queue.cc:260] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:38.065128  4126 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 1 FOLLOWER]: Advancing to term 3
I20260812 06:36:38.065791  4134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.193.130:47027 (request call id 4) took 4837 ms. Trace:
I20260812 06:36:38.065850  4134 rpcz_store.cc:276] 0812 06:36:33.228240 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:33.228285 (+    45us) service_pool.cc:224] Handling call
0812 06:36:33.228454 (+   169us) raft_consensus.cc:1471] Updating replica for 1 ops
0812 06:36:33.228599 (+   145us) raft_consensus.cc:1534] Early marking committed up to index 0
0812 06:36:33.228607 (+     8us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0812 06:36:33.228743 (+   136us) log.cc:844] Serialized 26 byte log entry
0812 06:36:33.228940 (+   197us) raft_consensus.cc:1645] Marking committed up to 0
0812 06:36:33.228953 (+    13us) raft_consensus.cc:1697] Filling consensus response to leader.
0812 06:36:33.228965 (+    12us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0812 06:36:38.065741 (+4836776us) raft_consensus.cc:1685] finished
0812 06:36:38.065745 (+     4us) raft_consensus.cc:1691] UpdateReplicas() finished
0812 06:36:38.065779 (+    34us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":454,"raft.run_cpu_time_us":32,"raft.run_wall_time_us":32,"thread_start_us":139,"threads_started":1,"wal-append.queue_time_us":4836114}
I20260812 06:36:38.066247  4126 raft_consensus.cc:1275] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Refusing update from remote peer df4e0103d6e942fb850d4f18b1cd2b85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260812 06:36:38.063602  4120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:38.066787  4120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:38.067014  4128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:38.067231  4125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:38.067473  4115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:38.067574  4115 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8c36d75aa834cca96b042eb3bf8a6d2 in term 3.
I20260812 06:36:38.068106  4117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:38.068346  4116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:38.068447  4116 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8c36d75aa834cca96b042eb3bf8a6d2 in term 3.
I20260812 06:36:38.068629  4118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:38.068811  4122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:38.069027  4740 consensus_queue.cc:1059] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 06:36:38.069173  4130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:38.069306  4127 raft_consensus.cc:3055] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 06:36:38.069363  4127 raft_consensus.cc:740] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c8af5f280df94129ba27e02fff9356a3, State: Running, Role: LEADER
I20260812 06:36:38.069491  4127 consensus_queue.cc:260] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:38.069770  4127 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:36:38.069775  4125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:38.070091  4128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:38.070199  4128 raft_consensus.cc:2368] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d8c36d75aa834cca96b042eb3bf8a6d2 for earlier term 2. Current term is 3.
I20260812 06:36:38.070413  4120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:38.070526  4120 raft_consensus.cc:2368] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8c36d75aa834cca96b042eb3bf8a6d2 for earlier term 2. Current term is 3.
I20260812 06:36:38.070899  4355 raft_consensus.cc:1240] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 3 LEADER]: Rejecting Update request from peer c8af5f280df94129ba27e02fff9356a3 for earlier term 1. Current term is 3. Ops: []
I20260812 06:36:38.070981  4127 raft_consensus.cc:1275] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 2 FOLLOWER]: Refusing update from remote peer d8c36d75aa834cca96b042eb3bf8a6d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260812 06:36:38.071149  4740 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260812 06:36:38.071233  4133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:38.071343  4133 raft_consensus.cc:2368] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate df4e0103d6e942fb850d4f18b1cd2b85 for earlier term 2. Current term is 3.
I20260812 06:36:38.071597  4134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:38.071739  4134 raft_consensus.cc:2368] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d8c36d75aa834cca96b042eb3bf8a6d2 for earlier term 2. Current term is 3.
I20260812 06:36:38.071861  4132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:38.071961  4132 raft_consensus.cc:2368] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate df4e0103d6e942fb850d4f18b1cd2b85 for earlier term 2. Current term is 3.
W20260812 06:36:38.072475  4881 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:38.060596 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:47536 (local address 127.3.193.130:35779)
0812 06:36:38.072015 (+ 11419us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:38.072021 (+     6us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:38.072038 (+    17us) server_negotiation.cc:408] Connection header received
0812 06:36:38.072091 (+    53us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:38.072096 (+     5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:38.072164 (+    68us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:38.072319 (+   155us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47536: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":11318,"thread_start_us":51,"threads_started":1}
I20260812 06:36:38.073444  4346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
W20260812 06:36:38.074303  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:37.568137 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.129:40455 (local address 127.0.0.1:51394)
0812 06:36:37.568302 (+   165us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:37.568318 (+    16us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:37.568409 (+    91us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:38.074203 (+505794us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 25 requested bytes
Metrics: {"client-negotiator.queue_time_us":61}
W20260812 06:36:38.074537  4681 meta_cache.cc:302] tablet 30fb1aacee684c4eacf9e756aa9927fc: replica c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 25 requested bytes
I20260812 06:36:38.075603  4126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:38.075733  4126 raft_consensus.cc:2468] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8c36d75aa834cca96b042eb3bf8a6d2 in term 2.
I20260812 06:36:38.075935  4855 consensus_queue.cc:1048] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260812 06:36:38.082582  4886 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:38.067919 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:37776 (local address 127.3.193.129:40455)
0812 06:36:38.082060 (+ 14141us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:38.082066 (+     6us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:38.082238 (+   172us) server_negotiation.cc:408] Connection header received
0812 06:36:38.082277 (+    39us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:38.082280 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:38.082325 (+    45us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:38.082468 (+   143us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37776: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":14082,"thread_start_us":53,"threads_started":1}
W20260812 06:36:38.083019  4884 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:38.067790 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:37764 (local address 127.3.193.129:40455)
0812 06:36:38.081799 (+ 14009us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:38.081804 (+     5us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:38.081818 (+    14us) server_negotiation.cc:408] Connection header received
0812 06:36:38.081859 (+    41us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:38.081862 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:38.082797 (+   935us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:38.082922 (+   125us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37764: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":13934,"thread_start_us":67,"threads_started":1}
W20260812 06:36:38.083751  4886 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:38.074876 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:51392 (local address 127.3.193.129:40455)
0812 06:36:38.083399 (+  8523us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:38.083405 (+     6us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:38.083421 (+    16us) server_negotiation.cc:408] Connection header received
0812 06:36:38.083464 (+    43us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:38.083467 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:38.083522 (+    55us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:38.083629 (+   107us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51392: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":8449,"thread_start_us":66,"threads_started":1}
W20260812 06:36:38.092585  4033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:38.094187  4882 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:38.066297 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:51394 (local address 127.3.193.129:40455)
0812 06:36:38.070757 (+  4460us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:38.070761 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:38.070778 (+    17us) server_negotiation.cc:408] Connection header received
0812 06:36:38.070857 (+    79us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:38.070860 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:38.070917 (+    57us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:38.071037 (+   120us) server_negotiation.cc:227] Negotiated authn=TOKEN
0812 06:36:38.094071 (+ 23034us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51394: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":4376,"thread_start_us":86,"threads_started":1}
W20260812 06:36:38.111492  4033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:38.136664  4033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:38.157459  4033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260812 06:36:38.162623  4855 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d8c36d75aa834cca96b042eb3bf8a6d2)
I20260812 06:36:38.162775  4855 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:38.163218  4855 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:38.163621  4132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:38.163791  4132 raft_consensus.cc:2468] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 2.
I20260812 06:36:38.164127  4244 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:38.164299  4855 raft_consensus.cc:2804] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 06:36:38.164361  4855 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d8c36d75aa834cca96b042eb3bf8a6d2)
I20260812 06:36:38.164391  4855 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:36:38.165239  4855 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:38.165601  4855 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 3 election: Requested vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:38.165951  4132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:38.166056  4132 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 2 FOLLOWER]: Advancing to term 3
W20260812 06:36:38.166923  4034 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.093s (SENT)
I20260812 06:36:38.166956  4132 raft_consensus.cc:2468] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 3.
I20260812 06:36:38.167317  4244 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:38.167512  4855 raft_consensus.cc:2804] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 06:36:38.167614  4855 raft_consensus.cc:697] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 3 LEADER]: Becoming Leader. State: Replica: df4e0103d6e942fb850d4f18b1cd2b85, State: Running, Role: LEADER
I20260812 06:36:38.167793  4855 consensus_queue.cc:237] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 6, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:38.169708  3906 catalog_manager.cc:5719] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 reported cstate change: term changed from 2 to 3, leader changed from d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131) to df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130). New cstate: current_term: 3 leader_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: UNKNOWN } } }
W20260812 06:36:38.185535  4033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260812 06:36:38.189247  4132 raft_consensus.cc:1275] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Refusing update from remote peer df4e0103d6e942fb850d4f18b1cd2b85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 7. (index mismatch)
I20260812 06:36:38.189906  4879 consensus_queue.cc:1048] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 06:36:38.192106  4763 mvcc.cc:204] Tried to move back new op lower bound from 7317571981370626048 to 7317571970178093056. Current Snapshot: MvccSnapshot[applied={T|T < 7317571972472045568}]
W20260812 06:36:38.208971  4033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:38.237733  4033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:38.259413  4033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:38.261209  4034 leader_election.cc:336] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503): Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.093s (SENT)
I20260812 06:36:38.261292  4034 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3; no voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:36:38.261492  4740 raft_consensus.cc:2749] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 06:36:38.285460  4033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:38.307379  4033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:38.330899  4033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:38.354242  4033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:38.379489  4033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:38.407392  4033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:38.414441  4245 leader_election.cc:336] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503): Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.125s (SENT)
W20260812 06:36:38.436779  4033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:38.466212  4033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:38.486361  4033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:38.500569  4245 leader_election.cc:336] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503): Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.167s (SENT)
W20260812 06:36:38.514943  4033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:38.534708  4033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:38.557281  4033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:38.579025  4033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:38.588013  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:38.078175 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.131:38503 (local address 127.0.0.1:41088)
0812 06:36:38.080218 (+  2043us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:38.080236 (+    18us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:38.080341 (+   105us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:38.587890 (+507549us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.131:38503: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":1875}
W20260812 06:36:38.606577  4033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:38.616509  4490 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.085s (SENT)
I20260812 06:36:38.616621  4490 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
W20260812 06:36:38.616848  4490 leader_election.cc:336] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.111s (SENT)
I20260812 06:36:38.616920  4490 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:36:38.617404  4867 raft_consensus.cc:2749] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 06:36:38.617620  4867 raft_consensus.cc:2749] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 06:36:38.617973  4593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:38.618155  4586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:38.618274  4586 raft_consensus.cc:2410] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c8af5f280df94129ba27e02fff9356a3 for term 2 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260812 06:36:38.618343  4922 consensus_queue.cc:1048] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260812 06:36:38.618528  4588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:38.618822  4590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:38.618937  4590 raft_consensus.cc:2410] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c8af5f280df94129ba27e02fff9356a3 for term 2 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260812 06:36:38.619259  4594 raft_consensus.cc:1240] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 3 FOLLOWER]: Rejecting Update request from peer c8af5f280df94129ba27e02fff9356a3 for earlier term 1. Current term is 3. Ops: []
I20260812 06:36:38.619498  4595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:38.619773  4596 raft_consensus.cc:3055] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 06:36:38.619833  4596 raft_consensus.cc:740] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d8c36d75aa834cca96b042eb3bf8a6d2, State: Running, Role: LEADER
I20260812 06:36:38.620024  4596 consensus_queue.cc:260] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 6, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:38.620296  4596 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:36:38.620339  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
W20260812 06:36:38.620527  4832 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:38.617236 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:41088 (local address 127.3.193.131:38503)
0812 06:36:38.620253 (+  3017us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:38.620256 (+     3us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:38.620269 (+    13us) server_negotiation.cc:408] Connection header received
0812 06:36:38.620306 (+    37us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:38.620308 (+     2us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:38.620347 (+    39us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:38.620455 (+   108us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41088: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":2940}
I20260812 06:36:38.621292  4596 raft_consensus.cc:1275] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 3 FOLLOWER]: Refusing update from remote peer df4e0103d6e942fb850d4f18b1cd2b85: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 3 index: 7. (index mismatch)
I20260812 06:36:38.624617  4879 consensus_queue.cc:1048] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260812 06:36:39.164099  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:38.649847 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.130:35779 (local address 127.0.0.1:47584)
0812 06:36:38.650048 (+   201us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:38.650062 (+    14us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:38.650138 (+    76us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:39.163978 (+513840us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":82}
W20260812 06:36:39.164439  4680 meta_cache.cc:302] tablet 3cf285c378e04319a5acdcae46870291: replica df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
I20260812 06:36:39.650583  4740 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:39.650765  4740 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:39.651264  4740 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:39.651968  4132 raft_consensus.cc:1217] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[2.2-2.6]   Dedup: 2.6->[]
I20260812 06:36:39.652124  4132 raft_consensus.cc:1240] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Rejecting Update request from peer d8c36d75aa834cca96b042eb3bf8a6d2 for earlier term 2. Current term is 3. Ops: [2.2-2.6]
I20260812 06:36:39.652900  4937 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:39.652998  4937 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:39.653378  4937 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:39.653462  4937 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:39.653532  4937 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:39.653900  4937 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:39.654443  4919 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:39.654608  4919 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:39.655246  4919 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
W20260812 06:36:39.655754  4490 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260812 06:36:39.656641  4587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:39.656786  4587 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 3.
I20260812 06:36:39.657023  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:39.657119  4132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:39.657145  4596 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 1.
I20260812 06:36:39.657577  4919 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:39.657622  4034 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2; no voters: 
I20260812 06:36:39.657771  4919 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:39.657845  4034 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2; no voters: 
I20260812 06:36:39.658025  4740 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 06:36:39.658089  4740 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:39.658159  4740 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Advancing to term 4
I20260812 06:36:39.658703  4132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:39.659178  4919 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:36:39.659346  4919 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:39.659420  4740 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:39.659662  4132 raft_consensus.cc:2393] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8c36d75aa834cca96b042eb3bf8a6d2 in current term 4: Already voted for candidate c8af5f280df94129ba27e02fff9356a3 in this term.
I20260812 06:36:39.659873  4740 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 4 election: Requested vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:39.659991  4740 raft_consensus.cc:2804] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 06:36:39.660043  4740 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:39.660101  4740 raft_consensus.cc:3060] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:36:39.659477  4919 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
W20260812 06:36:39.660346  4530 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41092: Illegal state: replica d8c36d75aa834cca96b042eb3bf8a6d2 is not leader of this config: current role FOLLOWER
I20260812 06:36:39.661036  4740 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:39.661154  4132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:39.661283  4132 raft_consensus.cc:2393] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8c36d75aa834cca96b042eb3bf8a6d2 in current term 2: Already voted for candidate c8af5f280df94129ba27e02fff9356a3 in this term.
I20260812 06:36:39.661418  4740 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 2 election: Requested vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:39.661826  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 4 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:39.661904  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:39.661947  4597 raft_consensus.cc:2468] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 3.
I20260812 06:36:39.661980  4596 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 3 FOLLOWER]: Advancing to term 4
I20260812 06:36:39.662398  4034 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2; no voters: 
I20260812 06:36:39.662611  4740 raft_consensus.cc:2804] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 06:36:39.662689  4740 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:39.662762  4740 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Advancing to term 4
I20260812 06:36:39.663198  4596 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 4.
I20260812 06:36:39.663403  4587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:39.663491  4587 raft_consensus.cc:3060] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:36:39.663717  4740 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:39.664151  4740 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 4 election: Requested vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:39.664516  4587 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 2.
I20260812 06:36:39.664765  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 4 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:39.664798  4034 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2; no voters: 
I20260812 06:36:39.664875  4596 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 3 FOLLOWER]: Advancing to term 4
I20260812 06:36:39.664958  4034 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2; no voters: 
I20260812 06:36:39.665103  4740 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 06:36:39.665228  4740 raft_consensus.cc:697] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 4 LEADER]: Becoming Leader. State: Replica: c8af5f280df94129ba27e02fff9356a3, State: Running, Role: LEADER
I20260812 06:36:39.665366  4740 consensus_queue.cc:237] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:39.665673  4596 raft_consensus.cc:2468] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 4.
I20260812 06:36:39.666059  4034 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2; no voters: 
I20260812 06:36:39.666164  4740 raft_consensus.cc:2804] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 06:36:39.665673  4919 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:36:39.666319  4740 raft_consensus.cc:697] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 2 LEADER]: Becoming Leader. State: Replica: c8af5f280df94129ba27e02fff9356a3, State: Running, Role: LEADER
I20260812 06:36:39.666457  4740 consensus_queue.cc:237] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:39.667886  3905 catalog_manager.cc:5719] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 reported cstate change: term changed from 3 to 4, leader changed from df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130) to c8af5f280df94129ba27e02fff9356a3 (127.3.193.129). New cstate: current_term: 4 leader_uuid: "c8af5f280df94129ba27e02fff9356a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: UNKNOWN } } }
I20260812 06:36:39.667037  4740 raft_consensus.cc:2804] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 06:36:39.668160  4740 raft_consensus.cc:697] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 4 LEADER]: Becoming Leader. State: Replica: c8af5f280df94129ba27e02fff9356a3, State: Running, Role: LEADER
I20260812 06:36:39.668265  4740 consensus_queue.cc:237] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 3.7, Last appended by leader: 7, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:39.673647  3905 catalog_manager.cc:5719] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 reported cstate change: term changed from 3 to 4, leader changed from df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130) to c8af5f280df94129ba27e02fff9356a3 (127.3.193.129). New cstate: current_term: 4 leader_uuid: "c8af5f280df94129ba27e02fff9356a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: UNKNOWN } } }
I20260812 06:36:39.673925  3905 catalog_manager.cc:5719] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 reported cstate change: term changed from 1 to 2, leader changed from df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130) to c8af5f280df94129ba27e02fff9356a3 (127.3.193.129). New cstate: current_term: 2 leader_uuid: "c8af5f280df94129ba27e02fff9356a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: HEALTHY } } }
I20260812 06:36:39.675312  4596 raft_consensus.cc:1275] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 2 FOLLOWER]: Refusing update from remote peer c8af5f280df94129ba27e02fff9356a3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260812 06:36:39.675805  4740 consensus_queue.cc:1048] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 06:36:39.688710  4596 raft_consensus.cc:1275] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Refusing update from remote peer c8af5f280df94129ba27e02fff9356a3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 06:36:39.689160  4937 consensus_queue.cc:1048] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 06:36:39.691359  4596 raft_consensus.cc:1275] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Refusing update from remote peer c8af5f280df94129ba27e02fff9356a3: Log matching property violated. Preceding OpId in replica: term: 3 index: 7. Preceding OpId from leader: term: 4 index: 8. (index mismatch)
I20260812 06:36:39.691901  4949 consensus_queue.cc:1048] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260812 06:36:39.739068  4489 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.193.130:35779 timed out after 0.078s (SENT) [suppressed 6 similar messages]
W20260812 06:36:39.807500  4032 leader_election.cc:336] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779): Timed out: RequestConsensusVote RPC to 127.3.193.130:35779 timed out after 0.076s (SENT)
W20260812 06:36:39.818218  4489 leader_election.cc:336] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779): Timed out: RequestConsensusVote RPC to 127.3.193.130:35779 timed out after 0.078s (SENT)
I20260812 06:36:39.818393  4489 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:36:39.818712  4939 raft_consensus.cc:2749] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 06:36:39.820904  4032 leader_election.cc:336] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779): Timed out: RequestConsensusVote RPC to 127.3.193.130:35779 timed out after 0.079s (SENT)
W20260812 06:36:39.826200  4032 leader_election.cc:336] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779): Timed out: RequestConsensusVote RPC to 127.3.193.130:35779 timed out after 0.080s (SENT)
W20260812 06:36:39.838635  4489 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779): Timed out: RequestConsensusVote RPC to 127.3.193.130:35779 timed out after 0.089s (SENT)
I20260812 06:36:39.838729  4489 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:36:39.838892  4939 raft_consensus.cc:2749] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260812 06:36:39.861763  4032 leader_election.cc:336] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779): Timed out: RequestConsensusVote RPC to 127.3.193.130:35779 timed out after 0.103s (SENT)
W20260812 06:36:39.863059  4032 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779): Timed out: RequestConsensusVote RPC to 127.3.193.130:35779 timed out after 0.101s (SENT)
W20260812 06:36:39.869215  4489 leader_election.cc:336] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779): Timed out: RequestConsensusVote RPC to 127.3.193.130:35779 timed out after 0.106s (SENT)
I20260812 06:36:39.869305  4489 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:36:39.869540  4939 raft_consensus.cc:2749] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260812 06:36:39.871315  4032 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779): Timed out: RequestConsensusVote RPC to 127.3.193.130:35779 timed out after 0.109s (SENT)
W20260812 06:36:40.187963  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:39.680589 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.130:35779 (local address 127.0.0.1:47598)
0812 06:36:39.680728 (+   139us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:39.680744 (+    16us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:39.680836 (+    92us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:40.187865 (+507029us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 06:36:40.188138  4680 meta_cache.cc:302] tablet 30fb1aacee684c4eacf9e756aa9927fc: replica df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
W20260812 06:36:40.202690  4530 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41094: Illegal state: replica d8c36d75aa834cca96b042eb3bf8a6d2 is not leader of this config: current role FOLLOWER
I20260812 06:36:40.211536  4361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 4 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:40.211942  4354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:36:40.212185  4355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:40.212230  4325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 4 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:40.212430  4327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 4 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:36:40.212502  4346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:40.212569  4315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:40.212719  4329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:36:40.212759  4334 raft_consensus.cc:3055] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 06:36:40.212806  4334 raft_consensus.cc:740] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: df4e0103d6e942fb850d4f18b1cd2b85, State: Running, Role: LEADER
I20260812 06:36:40.212890  4335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:40.212996  4330 raft_consensus.cc:3055] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 06:36:40.212965  4334 consensus_queue.cc:260] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:40.213040  4330 raft_consensus.cc:740] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: df4e0103d6e942fb850d4f18b1cd2b85, State: Running, Role: LEADER
I20260812 06:36:40.213135  4342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:40.213267  4331 raft_consensus.cc:3055] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 06:36:40.213335  4331 raft_consensus.cc:740] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: df4e0103d6e942fb850d4f18b1cd2b85, State: Running, Role: LEADER
I20260812 06:36:40.213363  4354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:36:40.213414  4345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 4 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:36:40.213460  4331 consensus_queue.cc:260] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:40.213570  4361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:40.213613  4331 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 3 FOLLOWER]: Advancing to term 4
I20260812 06:36:40.213644  4350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:40.213855  4352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:36:40.214581  4331 raft_consensus.cc:1275] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 4 FOLLOWER]: Refusing update from remote peer c8af5f280df94129ba27e02fff9356a3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 06:36:40.214843  4333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:40.212570  4353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:40.213148  4330 consensus_queue.cc:260] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 3.7, Last appended by leader: 7, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:40.213276  4334 raft_consensus.cc:3060] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:36:40.216532  4360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
W20260812 06:36:40.216668  4982 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:40.215983 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:47584 (local address 127.3.193.130:35779)
0812 06:36:40.216209 (+   226us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:40.216214 (+     5us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:40.216235 (+    21us) server_negotiation.cc:408] Connection header received
0812 06:36:40.216281 (+    46us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:40.216292 (+    11us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:40.216343 (+    51us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:40.216499 (+   156us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47584: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":57,"threads_started":1}
I20260812 06:36:40.216665  4360 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8c36d75aa834cca96b042eb3bf8a6d2 in term 4.
I20260812 06:36:40.216915  4334 raft_consensus.cc:1275] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 2 FOLLOWER]: Refusing update from remote peer c8af5f280df94129ba27e02fff9356a3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260812 06:36:40.218192  4330 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 3 FOLLOWER]: Advancing to term 4
W20260812 06:36:40.219007  4982 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:40.218601 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:47598 (local address 127.3.193.130:35779)
0812 06:36:40.218728 (+   127us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:40.218732 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:40.218745 (+    13us) server_negotiation.cc:408] Connection header received
0812 06:36:40.218782 (+    37us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:40.218785 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:40.218829 (+    44us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:40.218935 (+   106us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47598: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 06:36:40.219009  4330 raft_consensus.cc:1275] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 4 FOLLOWER]: Refusing update from remote peer c8af5f280df94129ba27e02fff9356a3: Log matching property violated. Preceding OpId in replica: term: 3 index: 7. Preceding OpId from leader: term: 4 index: 8. (index mismatch)
W20260812 06:36:40.764102  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:40.243997 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.129:40455 (local address 127.0.0.1:51438)
0812 06:36:40.244205 (+   208us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:40.244219 (+    14us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:40.244301 (+    82us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:40.763979 (+519678us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":85}
I20260812 06:36:41.244884  4740 consensus_queue.cc:1048] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 06:36:41.245184  4740 consensus_queue.cc:1048] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 06:36:41.245362  4740 consensus_queue.cc:1048] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 06:36:41.246732  4939 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:41.246855  4939 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:41.247296  4939 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:36:41.248667  4939 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:41.248759  4939 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:41.249115  4939 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:36:41.249230  4939 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:41.249279  4939 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:41.249536  4939 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
W20260812 06:36:41.250377  4972 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:41.247004 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:51438 (local address 127.3.193.129:40455)
0812 06:36:41.250025 (+  3021us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:41.250029 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:41.250042 (+    13us) server_negotiation.cc:408] Connection header received
0812 06:36:41.250091 (+    49us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:41.250094 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:41.250142 (+    48us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:41.250262 (+   120us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51438: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":2947}
I20260812 06:36:41.251250  4132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:41.251538  4134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:41.251803  4126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
W20260812 06:36:41.252480  4530 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41104: Illegal state: replica d8c36d75aa834cca96b042eb3bf8a6d2 is not leader of this config: current role FOLLOWER
I20260812 06:36:41.285879  4353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:41.286041  4353 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8c36d75aa834cca96b042eb3bf8a6d2 in term 4.
I20260812 06:36:41.286171  4829 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:41.286269  4829 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:41.286494  4855 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:41.286587  4855 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:41.286693  4829 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:36:41.286823  4829 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:41.286886  4829 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:41.286986  4587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:41.287201  4829 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:41.287396  4333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:41.287602  4331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:41.288492  4587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 5 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:41.288754  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:41.288789  4489 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85; no voters: c8af5f280df94129ba27e02fff9356a3
I20260812 06:36:41.288987  4990 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 06:36:41.289042  4990 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:41.289085  4990 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Advancing to term 5
I20260812 06:36:41.286885  4855 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:41.290225  5007 mvcc.cc:204] Tried to move back new op lower bound from 7317571994401853440 to 7317571992231096320. Current Snapshot: MvccSnapshot[applied={T|T < 7317571994401853440}]
I20260812 06:36:41.290351  4990 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:41.290728  4990 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 5 election: Requested vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:36:41.289083  4489 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:36:41.291160  4489 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:36:41.291326  4990 raft_consensus.cc:2749] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 06:36:41.291450  4990 raft_consensus.cc:2749] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 06:36:41.291723  4360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 11 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
W20260812 06:36:41.820114  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:41.317614 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.129:40455 (local address 127.0.0.1:51456)
0812 06:36:41.317815 (+   201us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:41.317832 (+    17us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:41.317945 (+   113us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:41.819937 (+501992us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":86}
W20260812 06:36:41.820389  4681 meta_cache.cc:302] tablet 30fb1aacee684c4eacf9e756aa9927fc: replica c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
I20260812 06:36:42.318221  4990 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:42.318351  4939 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:42.318462  4939 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:42.318850  4939 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:36:42.318372  4990 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:42.320362  4360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:42.320508  4360 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8c36d75aa834cca96b042eb3bf8a6d2 in term 2.
I20260812 06:36:42.320977  4489 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:42.321139  4939 raft_consensus.cc:2804] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 06:36:42.321193  4939 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:42.321233  4939 raft_consensus.cc:3060] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:36:42.321669  4990 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:36:42.322167  4360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:42.322266  4939 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:42.322304  4360 raft_consensus.cc:2468] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8c36d75aa834cca96b042eb3bf8a6d2 in term 4.
I20260812 06:36:42.322552  4939 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 3 election: Requested vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:36:42.322749  4489 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
W20260812 06:36:42.323087  4244 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.082s (SENT) [suppressed 7 similar messages]
I20260812 06:36:42.323552  4939 raft_consensus.cc:2804] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 06:36:42.323601  4939 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:42.323658  4939 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Advancing to term 5
I20260812 06:36:42.324471  4939 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:42.324771  4939 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 5 election: Requested vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:36:42.323084  4360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:36:42.324970  4360 raft_consensus.cc:3060] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:36:42.326033  4360 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8c36d75aa834cca96b042eb3bf8a6d2 in term 3.
I20260812 06:36:42.326501  4489 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:42.326694  4939 raft_consensus.cc:2804] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 06:36:42.326798  4939 raft_consensus.cc:697] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 3 LEADER]: Becoming Leader. State: Replica: d8c36d75aa834cca96b042eb3bf8a6d2, State: Running, Role: LEADER
W20260812 06:36:42.327000  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:41.826149 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.130:35779 (local address 127.0.0.1:47602)
0812 06:36:41.826331 (+   182us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:41.826347 (+    16us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:41.826443 (+    96us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:42.325324 (+498881us) client_negotiation.cc:281] Received NEGOTIATE NegotiatePB response
0812 06:36:42.325330 (+     6us) client_negotiation.cc:404] Received NEGOTIATE response from server
0812 06:36:42.325342 (+    12us) client_negotiation.cc:190] Negotiated authn=TOKEN
0812 06:36:42.325605 (+   263us) client_negotiation.cc:530] Sending TLS_HANDSHAKE message to server
0812 06:36:42.325618 (+    13us) client_negotiation.cc:262] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:36:42.326870 (+  1252us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 1288 requested bytes
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 06:36:42.326951  4939 consensus_queue.cc:237] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 7, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
W20260812 06:36:42.327106  5020 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:42.323421 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:47602 (local address 127.3.193.130:35779)
0812 06:36:42.325074 (+  1653us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:42.325078 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:42.325094 (+    16us) server_negotiation.cc:408] Connection header received
0812 06:36:42.325137 (+    43us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:42.325140 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:42.325187 (+    47us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:42.325274 (+    87us) server_negotiation.cc:227] Negotiated authn=TOKEN
0812 06:36:42.325775 (+   501us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0812 06:36:42.326601 (+   826us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:36:42.326993 (+   392us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47602: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":1574,"thread_start_us":35,"threads_started":1}
I20260812 06:36:42.327782  4360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:36:42.327880  4360 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 4 FOLLOWER]: Advancing to term 5
I20260812 06:36:42.328763  4360 raft_consensus.cc:2468] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8c36d75aa834cca96b042eb3bf8a6d2 in term 5.
I20260812 06:36:42.328799  3905 catalog_manager.cc:5719] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 reported cstate change: term changed from 2 to 3, leader changed from c8af5f280df94129ba27e02fff9356a3 (127.3.193.129) to d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131). New cstate: current_term: 3 leader_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: UNKNOWN } } }
I20260812 06:36:42.329501  4489 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:42.329653  4939 raft_consensus.cc:2804] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260812 06:36:42.329771  4939 raft_consensus.cc:697] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 LEADER]: Becoming Leader. State: Replica: d8c36d75aa834cca96b042eb3bf8a6d2, State: Running, Role: LEADER
I20260812 06:36:42.329898  4939 consensus_queue.cc:237] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 4.9, Last appended by leader: 9, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:42.333653  3905 catalog_manager.cc:5719] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 reported cstate change: term changed from 4 to 5, leader changed from c8af5f280df94129ba27e02fff9356a3 (127.3.193.129) to d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131). New cstate: current_term: 5 leader_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: HEALTHY } } }
W20260812 06:36:42.336378  4530 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41114: Illegal state: replica d8c36d75aa834cca96b042eb3bf8a6d2 is not leader of this config: current role FOLLOWER
I20260812 06:36:42.350545  4360 raft_consensus.cc:1275] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 3 FOLLOWER]: Refusing update from remote peer d8c36d75aa834cca96b042eb3bf8a6d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 3 index: 8. (index mismatch)
I20260812 06:36:42.351123  4939 consensus_queue.cc:1048] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 06:36:42.360627  4360 raft_consensus.cc:1275] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 5 FOLLOWER]: Refusing update from remote peer d8c36d75aa834cca96b042eb3bf8a6d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 9. Preceding OpId from leader: term: 5 index: 10. (index mismatch)
I20260812 06:36:42.361032  4990 consensus_queue.cc:1048] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260812 06:36:42.402439  4490 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.083s (SENT)
I20260812 06:36:42.402554  4490 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:36:42.402753  4939 raft_consensus.cc:2749] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260812 06:36:42.408510  4244 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.082s (SENT)
I20260812 06:36:42.408619  4244 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df4e0103d6e942fb850d4f18b1cd2b85; no voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2
I20260812 06:36:42.408838  5041 raft_consensus.cc:2749] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260812 06:36:42.426108  4244 leader_election.cc:336] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.100s (SENT)
I20260812 06:36:42.426260  4244 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df4e0103d6e942fb850d4f18b1cd2b85; no voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2
I20260812 06:36:42.426460  5041 raft_consensus.cc:2749] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260812 06:36:42.429584  4244 leader_election.cc:336] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.102s (SENT)
I20260812 06:36:42.429648  4244 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df4e0103d6e942fb850d4f18b1cd2b85; no voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2
I20260812 06:36:42.429788  5041 raft_consensus.cc:2749] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 06:36:42.484359  5041 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:42.484504  5041 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:42.484994  5041 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:42.485387  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:42.485572  4596 raft_consensus.cc:2393] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate df4e0103d6e942fb850d4f18b1cd2b85 in current term 5: Already voted for candidate d8c36d75aa834cca96b042eb3bf8a6d2 in this term.
I20260812 06:36:42.503093  4939 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:42.503298  4939 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:42.503793  4939 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:36:42.504276  4330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 11 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:42.504441  4330 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8c36d75aa834cca96b042eb3bf8a6d2 in term 4.
I20260812 06:36:42.504709  4489 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:42.504885  4939 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 06:36:42.504977  4939 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:36:42.505035  4939 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 FOLLOWER]: Advancing to term 6
I20260812 06:36:42.505910  4939 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:42.506297  4939 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 6 election: Requested vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:36:42.506747  4330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 11 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:36:42.506886  4330 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 4 FOLLOWER]: Advancing to term 6
I20260812 06:36:42.507730  4330 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8c36d75aa834cca96b042eb3bf8a6d2 in term 6.
I20260812 06:36:42.508044  4489 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:42.508213  4939 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 6 FOLLOWER]: Leader election won for term 6
I20260812 06:36:42.508337  4939 raft_consensus.cc:697] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 6 LEADER]: Becoming Leader. State: Replica: d8c36d75aa834cca96b042eb3bf8a6d2, State: Running, Role: LEADER
I20260812 06:36:42.508497  4939 consensus_queue.cc:237] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 4.11, Last appended by leader: 11, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:42.510279  3905 catalog_manager.cc:5719] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 reported cstate change: term changed from 4 to 6, leader changed from c8af5f280df94129ba27e02fff9356a3 (127.3.193.129) to d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131). New cstate: current_term: 6 leader_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: HEALTHY } } }
W20260812 06:36:42.532306  4490 leader_election.cc:336] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.103s (SENT)
W20260812 06:36:42.532552  4490 leader_election.cc:336] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.106s (SENT)
I20260812 06:36:42.539263  4330 raft_consensus.cc:1275] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 6 FOLLOWER]: Refusing update from remote peer d8c36d75aa834cca96b042eb3bf8a6d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 8. Preceding OpId from leader: term: 6 index: 12. (index mismatch)
I20260812 06:36:42.539810  4939 consensus_queue.cc:1048] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.000s
W20260812 06:36:42.543821  4490 leader_election.cc:336] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.110s (SENT)
W20260812 06:36:42.626358  4490 leader_election.cc:336] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.151s (SENT)
W20260812 06:36:42.657750  4490 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.075s (SENT)
W20260812 06:36:42.665145  4490 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.080s (SENT)
W20260812 06:36:42.682078  4244 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.098s (SENT)
I20260812 06:36:42.682195  4244 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df4e0103d6e942fb850d4f18b1cd2b85; no voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2
I20260812 06:36:42.682480  5041 raft_consensus.cc:2749] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260812 06:36:42.875989  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:42.346816 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.129:40455 (local address 127.0.0.1:51458)
0812 06:36:42.346959 (+   143us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:42.346976 (+    17us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:42.347056 (+    80us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:42.875888 (+528832us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 06:36:42.876165  4681 meta_cache.cc:302] tablet 30fb1aacee684c4eacf9e756aa9927fc: replica c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
W20260812 06:36:42.887908  4285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47610: Illegal state: replica df4e0103d6e942fb850d4f18b1cd2b85 is not leader of this config: current role FOLLOWER
W20260812 06:36:43.420007  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:42.904807 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.129:40455 (local address 127.0.0.1:51466)
0812 06:36:42.904941 (+   134us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:42.904959 (+    18us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:42.905023 (+    64us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:43.419868 (+514845us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 06:36:43.964133  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:43.431417 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.129:40455 (local address 127.0.0.1:51470)
0812 06:36:43.431547 (+   130us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:43.431560 (+    13us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:43.431628 (+    68us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:43.964031 (+532403us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 06:36:43.964371  4681 meta_cache.cc:302] tablet 27b60a3088e74447b13844f7b49da4ff: replica c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
W20260812 06:36:43.972647  4285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47624: Illegal state: replica df4e0103d6e942fb850d4f18b1cd2b85 is not leader of this config: current role FOLLOWER
W20260812 06:36:44.540077  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:44.017470 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.131:38503 (local address 127.0.0.1:41132)
0812 06:36:44.017653 (+   183us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:44.017670 (+    17us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:44.017761 (+    91us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:44.539962 (+522201us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.131:38503: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":62}
W20260812 06:36:45.018905  5068 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:45.017963 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:41132 (local address 127.3.193.131:38503)
0812 06:36:45.018496 (+   533us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:45.018502 (+     6us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:45.018520 (+    18us) server_negotiation.cc:408] Connection header received
0812 06:36:45.018587 (+    67us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:45.018590 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:45.018636 (+    46us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:45.018777 (+   141us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41132: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":438}
I20260812 06:36:45.022140  5085 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d8c36d75aa834cca96b042eb3bf8a6d2)
I20260812 06:36:45.022339  5085 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:45.022970  5085 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:45.023591  5085 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d8c36d75aa834cca96b042eb3bf8a6d2)
I20260812 06:36:45.023671  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 6 candidate_status { last_received { term: 5 index: 11 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:45.023754  5085 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:45.024093  5082 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d8c36d75aa834cca96b042eb3bf8a6d2)
I20260812 06:36:45.024187  5082 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:45.024392  5085 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:45.024794  5082 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:36:45.024809  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 7 candidate_status { last_received { term: 6 index: 13 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:45.025197  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 4 candidate_status { last_received { term: 3 index: 9 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
W20260812 06:36:45.028039  4285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47630: Illegal state: replica df4e0103d6e942fb850d4f18b1cd2b85 is not leader of this config: current role FOLLOWER
W20260812 06:36:45.596061  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:45.092752 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.131:38503 (local address 127.0.0.1:41178)
0812 06:36:45.092921 (+   169us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:45.092937 (+    16us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:45.093044 (+   107us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:45.595941 (+502897us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.131:38503: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":63}
W20260812 06:36:45.596371  4683 meta_cache.cc:302] tablet 3cf285c378e04319a5acdcae46870291: replica d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.131:38503: received 0 of 4 requested bytes
I20260812 06:36:46.095476  4127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:46.095803  4122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:46.096095  4115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 11 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:46.096309  4116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:46.096500  4118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:46.096755  4126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 11 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:46.097026  4130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:46.097309  4132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:46.097504  4132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 6 candidate_status { last_received { term: 5 index: 11 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:46.097564  4116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 7 candidate_status { last_received { term: 6 index: 13 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:46.097719  4133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:46.097913  4134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 11 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:46.098215  4126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 5 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:46.098881  4130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:46.099078  4134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 6 candidate_status { last_received { term: 5 index: 11 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:46.099335  4122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 5 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:46.099474  4117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 11 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:46.099637  4125 raft_consensus.cc:3055] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 4 LEADER]: Stepping down as leader of term 4
I20260812 06:36:46.099716  4125 raft_consensus.cc:740] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c8af5f280df94129ba27e02fff9356a3, State: Running, Role: LEADER
I20260812 06:36:46.099699  4113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 11 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:46.099843  4125 consensus_queue.cc:260] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 4.9, Last appended by leader: 9, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:46.099960  4128 raft_consensus.cc:3055] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 06:36:46.099989  4125 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 4 FOLLOWER]: Advancing to term 5
I20260812 06:36:46.100014  4128 raft_consensus.cc:740] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c8af5f280df94129ba27e02fff9356a3, State: Running, Role: LEADER
I20260812 06:36:46.100142  4128 consensus_queue.cc:260] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 7, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:46.100265  4128 raft_consensus.cc:3060] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:36:46.101200  4133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 7 candidate_status { last_received { term: 6 index: 13 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:46.101203  4126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 4 candidate_status { last_received { term: 3 index: 9 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:46.101501  4244 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df4e0103d6e942fb850d4f18b1cd2b85; no voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2
I20260812 06:36:46.101557  4130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 4 candidate_status { last_received { term: 3 index: 9 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:46.102058  4244 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df4e0103d6e942fb850d4f18b1cd2b85; no voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2
I20260812 06:36:46.102530  4120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:46.103543  4128 raft_consensus.cc:1275] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Refusing update from remote peer d8c36d75aa834cca96b042eb3bf8a6d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 3 index: 8. (index mismatch)
I20260812 06:36:46.103778  4112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 11 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:46.103968  4127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:46.104621  4125 raft_consensus.cc:1275] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 5 FOLLOWER]: Refusing update from remote peer d8c36d75aa834cca96b042eb3bf8a6d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 9. Preceding OpId from leader: term: 5 index: 10. (index mismatch)
I20260812 06:36:46.104691  4111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:46.104946  4110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:46.105077  4110 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 3.
W20260812 06:36:46.105108  5123 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:46.096356 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:51456 (local address 127.3.193.129:40455)
0812 06:36:46.104722 (+  8366us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:46.104727 (+     5us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:46.104742 (+    15us) server_negotiation.cc:408] Connection header received
0812 06:36:46.104797 (+    55us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:46.104800 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:46.104846 (+    46us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:46.105006 (+   160us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51456: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":8267,"thread_start_us":53,"threads_started":1}
W20260812 06:36:46.105479  5123 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:46.097155 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:51466 (local address 127.3.193.129:40455)
0812 06:36:46.105228 (+  8073us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:46.105232 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:46.105243 (+    11us) server_negotiation.cc:408] Connection header received
0812 06:36:46.105274 (+    31us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:46.105289 (+    15us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:46.105325 (+    36us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:46.105403 (+    78us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51466: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":8019,"thread_start_us":73,"threads_started":1}
I20260812 06:36:46.105556  4104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
W20260812 06:36:46.105811  5123 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:46.099404 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:51470 (local address 127.3.193.129:40455)
0812 06:36:46.105594 (+  6190us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:46.105598 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:46.105607 (+     9us) server_negotiation.cc:408] Connection header received
0812 06:36:46.105634 (+    27us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:46.105637 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:46.105669 (+    32us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:46.105744 (+    75us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51470: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"mutex_wait_us":2353,"server-negotiator.queue_time_us":6144,"thread_start_us":50,"threads_started":1}
I20260812 06:36:46.106186  5128 raft_consensus.cc:2749] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 06:36:46.106297  4114 raft_consensus.cc:3055] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 4 LEADER]: Stepping down as leader of term 4
I20260812 06:36:46.106355  4114 raft_consensus.cc:740] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c8af5f280df94129ba27e02fff9356a3, State: Running, Role: LEADER
I20260812 06:36:46.106472  5128 raft_consensus.cc:2749] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 06:36:46.106539  4114 consensus_queue.cc:260] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 4.11, Last appended by leader: 11, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:46.106648  4114 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 4 FOLLOWER]: Advancing to term 6
I20260812 06:36:46.106732  4118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:46.106827  4118 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 3.
I20260812 06:36:46.107406  4244 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df4e0103d6e942fb850d4f18b1cd2b85; no voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2
I20260812 06:36:46.107590  4114 raft_consensus.cc:1275] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 6 FOLLOWER]: Refusing update from remote peer d8c36d75aa834cca96b042eb3bf8a6d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 11. Preceding OpId from leader: term: 6 index: 12. (index mismatch)
I20260812 06:36:46.107751  5128 raft_consensus.cc:2749] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260812 06:36:46.108053  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:45.603624 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.131:38503 (local address 127.0.0.1:45988)
0812 06:36:45.603798 (+   174us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:45.603811 (+    13us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:45.603872 (+    61us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:46.107942 (+504070us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.131:38503: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":91}
I20260812 06:36:46.108609  4330 raft_consensus.cc:1217] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 6 FOLLOWER]: Deduplicated request from leader. Original: 4.8->[4.9-4.11]   Dedup: 4.11->[]
I20260812 06:36:46.108683  4330 raft_consensus.cc:1240] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 6 FOLLOWER]: Rejecting Update request from peer c8af5f280df94129ba27e02fff9356a3 for earlier term 4. Current term is 6. Ops: [4.9-4.11]
W20260812 06:36:46.109314  5119 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:46.096061 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:51458 (local address 127.3.193.129:40455)
0812 06:36:46.109002 (+ 12941us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:46.109007 (+     5us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:46.109021 (+    14us) server_negotiation.cc:408] Connection header received
0812 06:36:46.109057 (+    36us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:46.109061 (+     4us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:46.109098 (+    37us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:46.109202 (+   104us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51458: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":5682,"thread_start_us":80,"threads_started":1}
W20260812 06:36:46.118340  4285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48884: Illegal state: replica df4e0103d6e942fb850d4f18b1cd2b85 is not leader of this config: current role FOLLOWER
W20260812 06:36:46.118707  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.121912  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.129609  4071 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45174: Illegal state: replica c8af5f280df94129ba27e02fff9356a3 is not leader of this config: current role FOLLOWER
W20260812 06:36:46.145411  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.145527  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260812 06:36:46.164197  4986 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:46.164367  4986 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:46.164845  4986 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:46.165304  4330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 7 candidate_status { last_received { term: 4 index: 11 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
W20260812 06:36:46.165838  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.172088  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260812 06:36:46.176990  4986 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:46.177112  4986 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:46.177596  4986 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:46.177904  4330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 6 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:46.178051  4330 raft_consensus.cc:2410] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c8af5f280df94129ba27e02fff9356a3 for term 6 because replica has last-logged OpId of term: 5 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 9.
W20260812 06:36:46.190502  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.192734  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260812 06:36:46.198935  5130 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d8c36d75aa834cca96b042eb3bf8a6d2)
I20260812 06:36:46.199050  5130 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:46.199435  5130 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:36:46.199927  4114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 4 candidate_status { last_received { term: 3 index: 9 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:46.200096  4114 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 3.
I20260812 06:36:46.200421  4244 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:46.200596  5130 raft_consensus.cc:2804] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 06:36:46.200661  5130 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d8c36d75aa834cca96b042eb3bf8a6d2)
I20260812 06:36:46.200690  5130 raft_consensus.cc:3060] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 3 FOLLOWER]: Advancing to term 4
I20260812 06:36:46.201561  5128 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d8c36d75aa834cca96b042eb3bf8a6d2)
I20260812 06:36:46.201661  5128 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:46.201642  5130 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:46.202086  5128 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:46.202107  5130 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 4 election: Requested vote from peers d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:36:46.202450  4114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 6 candidate_status { last_received { term: 5 index: 13 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:46.202507  4126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 4 candidate_status { last_received { term: 3 index: 9 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:46.202580  4114 raft_consensus.cc:2468] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 5.
I20260812 06:36:46.202633  4126 raft_consensus.cc:3060] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 3 FOLLOWER]: Advancing to term 4
I20260812 06:36:46.202875  4244 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:46.203150  5130 raft_consensus.cc:2804] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 06:36:46.203204  5130 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d8c36d75aa834cca96b042eb3bf8a6d2)
I20260812 06:36:46.203246  5130 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 5 FOLLOWER]: Advancing to term 6
I20260812 06:36:46.203832  4126 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 4.
I20260812 06:36:46.204154  4244 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:46.204253  5130 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:46.204360  5128 raft_consensus.cc:2804] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 06:36:46.204584  5128 raft_consensus.cc:697] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 4 LEADER]: Becoming Leader. State: Replica: df4e0103d6e942fb850d4f18b1cd2b85, State: Running, Role: LEADER
I20260812 06:36:46.204615  5130 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 6 election: Requested vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:46.204754  5128 consensus_queue.cc:237] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 3.9, Last appended by leader: 9, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:46.204855  4126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 6 candidate_status { last_received { term: 5 index: 13 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:46.204936  4126 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 5 FOLLOWER]: Advancing to term 6
I20260812 06:36:46.205704  4126 raft_consensus.cc:2468] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 6.
I20260812 06:36:46.206015  4244 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:46.206202  5128 raft_consensus.cc:2804] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 6 FOLLOWER]: Leader election won for term 6
I20260812 06:36:46.206287  5128 raft_consensus.cc:697] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 6 LEADER]: Becoming Leader. State: Replica: df4e0103d6e942fb850d4f18b1cd2b85, State: Running, Role: LEADER
I20260812 06:36:46.206405  5128 consensus_queue.cc:237] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 5.13, Last appended by leader: 13, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:46.206421  3905 catalog_manager.cc:5719] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 reported cstate change: term changed from 3 to 4, leader changed from d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131) to df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130). New cstate: current_term: 4 leader_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: UNKNOWN } } }
W20260812 06:36:46.209023  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260812 06:36:46.210114  3905 catalog_manager.cc:5719] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 reported cstate change: term changed from 5 to 6, leader changed from d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131) to df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130). New cstate: current_term: 6 leader_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: UNKNOWN } } }
I20260812 06:36:46.218789  5135 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d8c36d75aa834cca96b042eb3bf8a6d2)
I20260812 06:36:46.218906  5135 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:46.219306  5135 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
W20260812 06:36:46.219405  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260812 06:36:46.219712  4126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:46.219862  4126 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 6.
I20260812 06:36:46.220137  4244 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:46.220309  5135 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 06:36:46.220371  5135 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d8c36d75aa834cca96b042eb3bf8a6d2)
I20260812 06:36:46.220438  5135 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 6 FOLLOWER]: Advancing to term 7
I20260812 06:36:46.221333  5135 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:46.221727  5135 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 7 election: Requested vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:46.222048  4126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:46.222144  4126 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 6 FOLLOWER]: Advancing to term 7
I20260812 06:36:46.222913  4126 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 7.
I20260812 06:36:46.223235  4244 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:46.223388  5135 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Leader election won for term 7
I20260812 06:36:46.223512  5135 raft_consensus.cc:697] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 LEADER]: Becoming Leader. State: Replica: df4e0103d6e942fb850d4f18b1cd2b85, State: Running, Role: LEADER
I20260812 06:36:46.223685  5135 consensus_queue.cc:237] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 6.14, Last appended by leader: 14, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:46.225265  3905 catalog_manager.cc:5719] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 reported cstate change: term changed from 6 to 7, leader changed from d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131) to df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130). New cstate: current_term: 7 leader_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: UNKNOWN } } }
I20260812 06:36:46.230897  4126 raft_consensus.cc:1275] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 4 FOLLOWER]: Refusing update from remote peer df4e0103d6e942fb850d4f18b1cd2b85: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 4 index: 10. (index mismatch)
I20260812 06:36:46.231287  5135 consensus_queue.cc:1048] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
W20260812 06:36:46.233965  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260812 06:36:46.235497  4126 raft_consensus.cc:1275] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 6 FOLLOWER]: Refusing update from remote peer df4e0103d6e942fb850d4f18b1cd2b85: Log matching property violated. Preceding OpId in replica: term: 4 index: 9. Preceding OpId from leader: term: 6 index: 14. (index mismatch)
I20260812 06:36:46.235921  5135 consensus_queue.cc:1048] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 9, Time since last communication: 0.000s
W20260812 06:36:46.241093  4034 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.076s (SENT) [suppressed 6 similar messages]
I20260812 06:36:46.248088  4126 raft_consensus.cc:1275] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 7 FOLLOWER]: Refusing update from remote peer df4e0103d6e942fb850d4f18b1cd2b85: Log matching property violated. Preceding OpId in replica: term: 4 index: 11. Preceding OpId from leader: term: 7 index: 15. (index mismatch)
W20260812 06:36:46.248378  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260812 06:36:46.248545  5130 consensus_queue.cc:1048] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.000s
W20260812 06:36:46.257794  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.268148  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.281558  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.297078  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.309432  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.316844  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.318481  4034 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503): Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.076s (SENT)
I20260812 06:36:46.318554  4034 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3; no voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:36:46.318715  4986 raft_consensus.cc:2749] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260812 06:36:46.336362  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.339676  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.360241  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.363513  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.385004  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.386289  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.391146  4034 leader_election.cc:336] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503): Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.106s (SENT)
I20260812 06:36:46.391227  4034 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3; no voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:36:46.391398  4986 raft_consensus.cc:2749] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260812 06:36:46.408403  4245 leader_election.cc:336] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503): Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.102s (SENT)
W20260812 06:36:46.409009  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.410240  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.434145  4245 leader_election.cc:336] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503): Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.115s (SENT)
W20260812 06:36:46.435712  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.438989  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.452443  4245 leader_election.cc:336] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503): Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.126s (SENT)
W20260812 06:36:46.458621  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.460861  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.474812  4245 leader_election.cc:336] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503): Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.134s (SENT)
W20260812 06:36:46.485312  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.486552  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.505010  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.510310  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.511240  4245 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503): Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.145s (SENT)
W20260812 06:36:46.525110  4245 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503): Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.151s (SENT)
W20260812 06:36:46.527860  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.533241  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.551676  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.552917  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.573513  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.578749  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.599308  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.609589  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.620086  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.637434  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.639817  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.652007  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:46.140941 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.131:38503 (local address 127.0.0.1:45998)
0812 06:36:46.141079 (+   138us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:46.141094 (+    15us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:46.141161 (+    67us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:46.651884 (+510723us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.131:38503: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 06:36:46.652251  4683 meta_cache.cc:302] tablet 3cf285c378e04319a5acdcae46870291: replica d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.131:38503: received 0 of 4 requested bytes
W20260812 06:36:46.661319  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.664587  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.681041  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.689422  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.708969  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.720220  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.739836  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.751104  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.760496  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.774797  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.787303  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.801023  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.814404  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.828821  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.843321  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.847539  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.867111  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.870342  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.896862  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.896965  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.925369  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.926589  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.946043  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.956473  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.969904  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.987386  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:46.998847  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:47.014346  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:47.017588  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:47.034968  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:47.038405  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:47.056758  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:47.062043  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:47.076536  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:47.084973  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:47.105611  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:47.107842  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:47.131413  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:47.136646  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:47.157127  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:47.160321  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:47.180900  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:47.185158  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:47.196002  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:46.668726 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.131:38503 (local address 127.0.0.1:46006)
0812 06:36:46.668850 (+   124us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:46.668863 (+    13us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:46.668927 (+    64us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:47.195891 (+526964us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.131:38503: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 06:36:47.199482  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260812 06:36:47.205277  4071 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45182: Illegal state: replica c8af5f280df94129ba27e02fff9356a3 is not leader of this config: current role FOLLOWER
W20260812 06:36:47.213858  4037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260812 06:36:47.222541  4594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 4 candidate_status { last_received { term: 3 index: 9 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:47.222819  4583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:47.223052  4594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 6 candidate_status { last_received { term: 5 index: 13 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:47.223109  4585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:47.223212  4570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 4 candidate_status { last_received { term: 3 index: 9 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:47.223315  4595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 7 candidate_status { last_received { term: 4 index: 11 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:47.223372  5182 consensus_queue.cc:1048] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 06:36:47.223547  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 4 candidate_status { last_received { term: 3 index: 9 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:47.223691  4587 raft_consensus.cc:1240] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 6 LEADER]: Rejecting Update request from peer c8af5f280df94129ba27e02fff9356a3 for earlier term 4. Current term is 6. Ops: []
I20260812 06:36:47.223690  5182 consensus_queue.cc:1048] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260812 06:36:47.223799  4939 consensus_queue.cc:1048] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260812 06:36:47.224042  4588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 7 candidate_status { last_received { term: 4 index: 11 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:47.224248  4590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 6 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:47.224475  4593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 6 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:47.224718  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 6 candidate_status { last_received { term: 5 index: 13 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:47.224967  4598 raft_consensus.cc:1240] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 3 LEADER]: Rejecting Update request from peer c8af5f280df94129ba27e02fff9356a3 for earlier term 2. Current term is 3. Ops: []
I20260812 06:36:47.225260  4582 raft_consensus.cc:3055] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 06:36:47.225317  4567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:47.225340  4582 raft_consensus.cc:740] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d8c36d75aa834cca96b042eb3bf8a6d2, State: Running, Role: LEADER
I20260812 06:36:47.225481  4582 consensus_queue.cc:260] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 3.9, Last appended by leader: 9, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:47.225525  4566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:47.225680  4578 raft_consensus.cc:3055] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 6 LEADER]: Stepping down as leader of term 6
I20260812 06:36:47.225710  4578 raft_consensus.cc:740] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d8c36d75aa834cca96b042eb3bf8a6d2, State: Running, Role: LEADER
I20260812 06:36:47.225790  4578 consensus_queue.cc:260] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 6.14, Last appended by leader: 14, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:47.225879  4578 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 6 FOLLOWER]: Advancing to term 7
I20260812 06:36:47.226007  4581 raft_consensus.cc:3055] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260812 06:36:47.226050  4581 raft_consensus.cc:740] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d8c36d75aa834cca96b042eb3bf8a6d2, State: Running, Role: LEADER
I20260812 06:36:47.226158  4581 consensus_queue.cc:260] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 5.13, Last appended by leader: 13, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:47.226248  4581 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 FOLLOWER]: Advancing to term 6
I20260812 06:36:47.226775  4578 raft_consensus.cc:1275] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Refusing update from remote peer df4e0103d6e942fb850d4f18b1cd2b85: Log matching property violated. Preceding OpId in replica: term: 6 index: 14. Preceding OpId from leader: term: 7 index: 15. (index mismatch)
I20260812 06:36:47.226981  4583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 6 candidate_status { last_received { term: 5 index: 13 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:47.227219  4581 raft_consensus.cc:1275] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 6 FOLLOWER]: Refusing update from remote peer df4e0103d6e942fb850d4f18b1cd2b85: Log matching property violated. Preceding OpId in replica: term: 5 index: 13. Preceding OpId from leader: term: 6 index: 14. (index mismatch)
I20260812 06:36:47.225554  4582 raft_consensus.cc:3060] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 3 FOLLOWER]: Advancing to term 4
W20260812 06:36:47.228160  5184 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:47.227139 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:45988 (local address 127.3.193.131:38503)
0812 06:36:47.227732 (+   593us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:47.227738 (+     6us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:47.227754 (+    16us) server_negotiation.cc:408] Connection header received
0812 06:36:47.227806 (+    52us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:47.227809 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:47.227863 (+    54us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:47.228002 (+   139us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45988: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":77,"threads_started":1}
I20260812 06:36:47.228413  4582 raft_consensus.cc:1275] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Refusing update from remote peer df4e0103d6e942fb850d4f18b1cd2b85: Log matching property violated. Preceding OpId in replica: term: 3 index: 9. Preceding OpId from leader: term: 4 index: 10. (index mismatch)
I20260812 06:36:47.228612  4330 raft_consensus.cc:1240] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 4 LEADER]: Rejecting Update request from peer d8c36d75aa834cca96b042eb3bf8a6d2 for earlier term 3. Current term is 4. Ops: []
W20260812 06:36:47.228688  5184 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:47.227489 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:46006 (local address 127.3.193.131:38503)
0812 06:36:47.228368 (+   879us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:47.228372 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:47.228387 (+    15us) server_negotiation.cc:408] Connection header received
0812 06:36:47.228425 (+    38us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:47.228428 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:47.228480 (+    52us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:47.228584 (+   104us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46006: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":812,"thread_start_us":69,"threads_started":1}
I20260812 06:36:47.228976  4334 raft_consensus.cc:1240] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 6 LEADER]: Rejecting Update request from peer d8c36d75aa834cca96b042eb3bf8a6d2 for earlier term 5. Current term is 6. Ops: []
W20260812 06:36:47.229228  5184 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:47.228777 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:41178 (local address 127.3.193.131:38503)
0812 06:36:47.228928 (+   151us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:47.228932 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:47.228946 (+    14us) server_negotiation.cc:408] Connection header received
0812 06:36:47.228985 (+    39us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:47.228988 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:47.229031 (+    43us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:47.229124 (+    93us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41178: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":78}
I20260812 06:36:47.229930  5135 consensus_queue.cc:1048] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260812 06:36:47.230031  5130 consensus_queue.cc:1048] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260812 06:36:47.223526  4574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 4 candidate_status { last_received { term: 3 index: 9 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:47.230201  5185 consensus_queue.cc:1048] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260812 06:36:47.230868  4574 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 4.
I20260812 06:36:47.223520  4586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 6 candidate_status { last_received { term: 5 index: 13 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:47.231767  4586 raft_consensus.cc:2468] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 6.
W20260812 06:36:47.232280  5184 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:47.231836 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:45998 (local address 127.3.193.131:38503)
0812 06:36:47.231975 (+   139us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:47.231979 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:47.231994 (+    15us) server_negotiation.cc:408] Connection header received
0812 06:36:47.232030 (+    36us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:47.232033 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:47.232077 (+    44us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:47.232186 (+   109us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45998: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 06:36:47.234290  5188 mvcc.cc:204] Tried to move back new op lower bound from 7317572023141363712 to 7317572019092865024. Current Snapshot: MvccSnapshot[applied={T|T < 7317572023141363712}]
W20260812 06:36:47.772101  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:47.255184 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.130:35779 (local address 127.0.0.1:48924)
0812 06:36:47.255372 (+   188us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:47.255387 (+    15us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:47.255469 (+    82us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:47.771983 (+516514us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":61}
W20260812 06:36:47.772358  4680 meta_cache.cc:302] tablet 3cf285c378e04319a5acdcae46870291: replica df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
I20260812 06:36:48.259591  4126 raft_consensus.cc:1217] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 4 FOLLOWER]: Deduplicated request from leader. Original: 2.7->[3.8-3.9]   Dedup: 3.9->[]
I20260812 06:36:48.259682  4114 raft_consensus.cc:1217] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 7 FOLLOWER]: Deduplicated request from leader. Original: 4.11->[6.12-6.14]   Dedup: 6.14->[]
I20260812 06:36:48.259727  4126 raft_consensus.cc:1240] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 4 FOLLOWER]: Rejecting Update request from peer d8c36d75aa834cca96b042eb3bf8a6d2 for earlier term 3. Current term is 4. Ops: [3.8-3.9]
I20260812 06:36:48.259768  4114 raft_consensus.cc:1240] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 7 FOLLOWER]: Rejecting Update request from peer d8c36d75aa834cca96b042eb3bf8a6d2 for earlier term 6. Current term is 7. Ops: [6.12-6.14]
I20260812 06:36:48.260259  4133 raft_consensus.cc:1217] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 6 FOLLOWER]: Deduplicated request from leader. Original: 4.9->[5.10-5.13]   Dedup: 5.13->[]
I20260812 06:36:48.260356  4133 raft_consensus.cc:1240] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 6 FOLLOWER]: Rejecting Update request from peer d8c36d75aa834cca96b042eb3bf8a6d2 for earlier term 5. Current term is 6. Ops: [5.10-5.13]
I20260812 06:36:48.262866  5206 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
W20260812 06:36:48.263062  5204 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:48.256934 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:48924 (local address 127.3.193.130:35779)
0812 06:36:48.257272 (+   338us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:48.257277 (+     5us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:48.257294 (+    17us) server_negotiation.cc:408] Connection header received
0812 06:36:48.257350 (+    56us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:48.257353 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:48.257410 (+    57us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:48.262649 (+  5239us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48924: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":54,"threads_started":1}
I20260812 06:36:48.263015  5206 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:48.263613  4334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 8 candidate_status { last_received { term: 7 index: 16 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:48.264164  5206 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:48.264281  5206 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:48.264369  5206 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:48.264943  4334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 5 candidate_status { last_received { term: 4 index: 12 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:48.265491  5206 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
W20260812 06:36:48.267453  4071 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45198: Illegal state: replica c8af5f280df94129ba27e02fff9356a3 is not leader of this config: current role FOLLOWER
W20260812 06:36:48.298275  4492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260812 06:36:48.298619  4990 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:48.299166  4990 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:48.299610  4990 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:36:48.299800  4939 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:48.299899  4939 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:48.300173  4110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:48.300257  4939 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:36:48.300642  4110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 13 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:48.304335  4581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 8 candidate_status { last_received { term: 7 index: 16 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:48.304680  4583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 5 candidate_status { last_received { term: 4 index: 12 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:48.304782  4034 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3; no voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85
W20260812 06:36:48.298656  4489 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260812 06:36:48.305063  4034 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3; no voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:36:48.305289  5205 raft_consensus.cc:2749] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 06:36:48.305393  5205 raft_consensus.cc:2749] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 06:36:48.305411  4334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:48.305459  4330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 13 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:48.305929  4489 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:36:48.306188  4489 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:36:48.306627  5178 raft_consensus.cc:2749] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 06:36:48.298731  5081 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:48.306744  5178 raft_consensus.cc:2749] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 06:36:48.306777  5081 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:48.307176  5081 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:36:48.307840  4330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 16 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
W20260812 06:36:48.860016  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:48.338671 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.130:35779 (local address 127.0.0.1:48950)
0812 06:36:48.338871 (+   200us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:48.338899 (+    28us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:48.338983 (+    84us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:48.859906 (+520923us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":79}
W20260812 06:36:48.860316  4680 meta_cache.cc:302] tablet 30fb1aacee684c4eacf9e756aa9927fc: replica df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
W20260812 06:36:49.339179  4490 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.101s (SENT) [suppressed 9 similar messages]
I20260812 06:36:49.339869  5242 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:49.339983  5242 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:49.340534  5242 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:36:49.340627  5242 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:49.340691  5242 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:49.341074  5242 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:36:49.342432  4330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 14 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:49.342841  4334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
W20260812 06:36:49.346012  5249 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:49.345452 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:48950 (local address 127.3.193.130:35779)
0812 06:36:49.345667 (+   215us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:49.345671 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:49.345685 (+    14us) server_negotiation.cc:408] Connection header received
0812 06:36:49.345730 (+    45us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:49.345733 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:49.345781 (+    48us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:49.345902 (+   121us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48950: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":56,"threads_started":1}
W20260812 06:36:49.371935  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:48.862045 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.129:40455 (local address 127.0.0.1:45200)
0812 06:36:48.862212 (+   167us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:48.862227 (+    15us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:48.862308 (+    81us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:49.371819 (+509511us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 06:36:49.379057  4530 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46016: Illegal state: replica d8c36d75aa834cca96b042eb3bf8a6d2 is not leader of this config: current role FOLLOWER
W20260812 06:36:49.948086  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:49.442093 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.130:35779 (local address 127.0.0.1:48988)
0812 06:36:49.442255 (+   162us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:49.442269 (+    14us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:49.442357 (+    88us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:49.947975 (+505618us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 06:36:49.948417  4680 meta_cache.cc:302] tablet 3cf285c378e04319a5acdcae46870291: replica df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
I20260812 06:36:50.442368  5205 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:50.442502  5205 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:50.442863  4126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 16 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:50.442976  5205 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:50.443049  5263 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:50.443082  4126 raft_consensus.cc:2410] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8c36d75aa834cca96b042eb3bf8a6d2 for term 8 because replica has last-logged OpId of term: 7 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 16.
W20260812 06:36:50.443193  4110 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.3.193.131:43093 (request call id 45) took 1033 ms (client timeout 101 ms). Trace:
W20260812 06:36:50.443238  4110 rpcz_store.cc:269] 0812 06:36:49.409808 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:50.443151 (+1033343us) service_pool.cc:211] Skipping call since client already timed out
0812 06:36:50.443184 (+    33us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:36:50.443327  5263 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:50.443859  4133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.193.130:47027 (request call id 258) took 1034 ms. Trace:
I20260812 06:36:50.443948  4133 rpcz_store.cc:276] 0812 06:36:49.409807 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:50.443665 (+1033858us) service_pool.cc:224] Handling call
0812 06:36:50.443744 (+    79us) raft_consensus.cc:1471] Updating replica for 0 ops
0812 06:36:50.443786 (+    42us) raft_consensus.cc:1534] Early marking committed up to index 0
0812 06:36:50.443789 (+     3us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0812 06:36:50.443799 (+    10us) raft_consensus.cc:1645] Marking committed up to 14
0812 06:36:50.443803 (+     4us) raft_consensus.cc:1697] Filling consensus response to leader.
0812 06:36:50.443811 (+     8us) raft_consensus.cc:1691] UpdateReplicas() finished
0812 06:36:50.443839 (+    28us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 06:36:50.444110  5263 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:50.444301  4104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:50.445415  4114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:50.445709  4118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 14 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:50.446856  5205 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:50.446966  5205 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:50.447532  5205 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
W20260812 06:36:50.447955  4490 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.101s (SENT)
I20260812 06:36:50.448032  4490 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
W20260812 06:36:50.448276  4490 leader_election.cc:336] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.076s (SENT)
I20260812 06:36:50.448329  4490 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:36:50.449044  4582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 8 candidate_status { last_received { term: 7 index: 17 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:50.449211  4582 raft_consensus.cc:2410] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c8af5f280df94129ba27e02fff9356a3 for term 8 because replica has last-logged OpId of term: 7 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 17.
W20260812 06:36:50.449538  5272 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:50.448482 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:45200 (local address 127.3.193.129:40455)
0812 06:36:50.449052 (+   570us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:50.449059 (+     7us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:50.449077 (+    18us) server_negotiation.cc:408] Connection header received
0812 06:36:50.449123 (+    46us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:50.449130 (+     7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:50.449185 (+    55us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:50.449394 (+   209us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45200: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":77,"threads_started":1}
I20260812 06:36:50.449561  4583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 5 candidate_status { last_received { term: 4 index: 14 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:50.449697  4583 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 4.
I20260812 06:36:50.449899  5267 raft_consensus.cc:2749] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 06:36:50.450062  5267 raft_consensus.cc:2749] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260812 06:36:50.450453  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:49.950244 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.131:38503 (local address 127.0.0.1:46028)
0812 06:36:49.950422 (+   178us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:49.950444 (+    22us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:49.950531 (+    87us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:50.448940 (+498409us) client_negotiation.cc:281] Received NEGOTIATE NegotiatePB response
0812 06:36:50.448945 (+     5us) client_negotiation.cc:404] Received NEGOTIATE response from server
0812 06:36:50.448958 (+    13us) client_negotiation.cc:190] Negotiated authn=TOKEN
0812 06:36:50.449215 (+   257us) client_negotiation.cc:530] Sending TLS_HANDSHAKE message to server
0812 06:36:50.449226 (+    11us) client_negotiation.cc:262] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:36:50.450338 (+  1112us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.131:38503: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":66}
W20260812 06:36:50.450739  5250 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:50.447736 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:46028 (local address 127.3.193.131:38503)
0812 06:36:50.448050 (+   314us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:50.448055 (+     5us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:50.448070 (+    15us) server_negotiation.cc:408] Connection header received
0812 06:36:50.448120 (+    50us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:50.448123 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:50.448178 (+    55us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:50.448262 (+    84us) server_negotiation.cc:227] Negotiated authn=TOKEN
0812 06:36:50.449372 (+  1110us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0812 06:36:50.450119 (+   747us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:36:50.450637 (+   518us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46028: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":220}
I20260812 06:36:50.450995  4034 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2; no voters: 
I20260812 06:36:50.451164  5205 raft_consensus.cc:2804] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 06:36:50.451206  5205 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:50.451244  5205 raft_consensus.cc:3060] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 4 FOLLOWER]: Advancing to term 5
I20260812 06:36:50.452451  5205 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:50.452790  5205 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 5 election: Requested vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:50.453339  4583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 5 candidate_status { last_received { term: 4 index: 14 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:50.453378  4574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:50.453456  4583 raft_consensus.cc:3060] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 4 FOLLOWER]: Advancing to term 5
I20260812 06:36:50.453466  4574 raft_consensus.cc:2468] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 6.
I20260812 06:36:50.454363  4034 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2; no voters: 
I20260812 06:36:50.454607  5205 raft_consensus.cc:2804] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 06:36:50.454674  5205 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:50.454727  5205 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 6 FOLLOWER]: Advancing to term 7
I20260812 06:36:50.454735  4583 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 5.
I20260812 06:36:50.455078  4034 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2; no voters: 
I20260812 06:36:50.455240  5263 raft_consensus.cc:2804] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 5 FOLLOWER]: Leader election won for term 5
I20260812 06:36:50.455610  5263 raft_consensus.cc:697] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 5 LEADER]: Becoming Leader. State: Replica: c8af5f280df94129ba27e02fff9356a3, State: Running, Role: LEADER
I20260812 06:36:50.455837  5263 consensus_queue.cc:237] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 4.14, Last appended by leader: 14, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:50.457850  3906 catalog_manager.cc:5719] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 reported cstate change: term changed from 4 to 5, leader changed from df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130) to c8af5f280df94129ba27e02fff9356a3 (127.3.193.129). New cstate: current_term: 5 leader_uuid: "c8af5f280df94129ba27e02fff9356a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: HEALTHY } } }
I20260812 06:36:50.459532  5205 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:50.460642  4583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:50.460747  4583 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 6 FOLLOWER]: Advancing to term 7
I20260812 06:36:50.460917  4118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 14 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:50.461347  4490 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:36:50.461615  5205 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 7 election: Requested vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:50.461653  4583 raft_consensus.cc:2468] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 7.
I20260812 06:36:50.461979  4034 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2; no voters: 
I20260812 06:36:50.462056  5266 raft_consensus.cc:2749] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 06:36:50.462280  5205 raft_consensus.cc:2804] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 7 FOLLOWER]: Leader election won for term 7
I20260812 06:36:50.462707  5205 raft_consensus.cc:697] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 7 LEADER]: Becoming Leader. State: Replica: c8af5f280df94129ba27e02fff9356a3, State: Running, Role: LEADER
I20260812 06:36:50.462913  5205 consensus_queue.cc:237] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 6.14, Last appended by leader: 14, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:50.464682  3906 catalog_manager.cc:5719] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 reported cstate change: term changed from 6 to 7, leader changed from df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130) to c8af5f280df94129ba27e02fff9356a3 (127.3.193.129). New cstate: current_term: 7 leader_uuid: "c8af5f280df94129ba27e02fff9356a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: UNKNOWN } } }
I20260812 06:36:50.468439  4583 raft_consensus.cc:1275] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 FOLLOWER]: Refusing update from remote peer c8af5f280df94129ba27e02fff9356a3: Log matching property violated. Preceding OpId in replica: term: 4 index: 14. Preceding OpId from leader: term: 5 index: 16. (index mismatch)
I20260812 06:36:50.468923  5205 consensus_queue.cc:1048] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260812 06:36:50.493980  4583 raft_consensus.cc:1275] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Refusing update from remote peer c8af5f280df94129ba27e02fff9356a3: Log matching property violated. Preceding OpId in replica: term: 6 index: 14. Preceding OpId from leader: term: 7 index: 15. (index mismatch)
I20260812 06:36:50.494556  5263 consensus_queue.cc:1048] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260812 06:36:50.550978  5266 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:50.551165  5266 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:50.551738  5266 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:36:50.552203  4118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 20 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:50.552384  4118 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8c36d75aa834cca96b042eb3bf8a6d2 in term 7.
I20260812 06:36:50.552701  4490 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2; no voters: 
I20260812 06:36:50.552881  5266 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 06:36:50.552946  5266 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:50.552999  5266 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Advancing to term 8
I20260812 06:36:50.553982  5266 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:50.554405  5266 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 8 election: Requested vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:36:50.554699  4118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 20 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:50.554804  4118 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 7 FOLLOWER]: Advancing to term 8
I20260812 06:36:50.555896  4118 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8c36d75aa834cca96b042eb3bf8a6d2 in term 8.
I20260812 06:36:50.556229  4490 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2; no voters: 
I20260812 06:36:50.556434  5266 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 8 FOLLOWER]: Leader election won for term 8
I20260812 06:36:50.556555  5266 raft_consensus.cc:697] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 8 LEADER]: Becoming Leader. State: Replica: d8c36d75aa834cca96b042eb3bf8a6d2, State: Running, Role: LEADER
I20260812 06:36:50.556706  5266 consensus_queue.cc:237] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 7.20, Last appended by leader: 20, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:50.558441  3905 catalog_manager.cc:5719] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 reported cstate change: term changed from 7 to 8, leader changed from df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130) to d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131). New cstate: current_term: 8 leader_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: HEALTHY } } }
I20260812 06:36:50.583467  4118 raft_consensus.cc:1275] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Refusing update from remote peer d8c36d75aa834cca96b042eb3bf8a6d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 17. Preceding OpId from leader: term: 8 index: 21. (index mismatch)
I20260812 06:36:50.584026  5266 consensus_queue.cc:1048] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 17, Time since last communication: 0.000s
W20260812 06:36:50.615751  4032 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779): Timed out: RequestConsensusVote RPC to 127.3.193.130:35779 timed out after 0.083s (SENT)
I20260812 06:36:50.615885  4032 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3; no voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:36:50.616113  5206 raft_consensus.cc:2749] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260812 06:36:50.621608  4032 leader_election.cc:336] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779): Timed out: RequestConsensusVote RPC to 127.3.193.130:35779 timed out after 0.086s (SENT)
W20260812 06:36:50.622888  4032 leader_election.cc:336] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779): Timed out: RequestConsensusVote RPC to 127.3.193.130:35779 timed out after 0.081s (SENT)
W20260812 06:36:50.629168  4032 leader_election.cc:336] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779): Timed out: RequestConsensusVote RPC to 127.3.193.130:35779 timed out after 0.087s (SENT)
W20260812 06:36:50.727088  4489 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779): Timed out: RequestConsensusVote RPC to 127.3.193.130:35779 timed out after 0.087s (SENT)
W20260812 06:36:50.733409  4489 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779): Timed out: RequestConsensusVote RPC to 127.3.193.130:35779 timed out after 0.089s (SENT)
W20260812 06:36:50.749668  4032 leader_election.cc:336] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779): Timed out: RequestConsensusVote RPC to 127.3.193.130:35779 timed out after 0.150s (SENT)
W20260812 06:36:51.004050  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:50.473128 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.130:35779 (local address 127.0.0.1:48990)
0812 06:36:50.473302 (+   174us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:50.473321 (+    19us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:50.473408 (+    87us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:51.003932 (+530524us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":63}
W20260812 06:36:51.004238  4680 meta_cache.cc:302] tablet 27b60a3088e74447b13844f7b49da4ff: replica df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
I20260812 06:36:51.022259  4360 raft_consensus.cc:3055] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 LEADER]: Stepping down as leader of term 7
I20260812 06:36:51.022356  4360 raft_consensus.cc:740] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: df4e0103d6e942fb850d4f18b1cd2b85, State: Running, Role: LEADER
I20260812 06:36:51.022495  4360 consensus_queue.cc:260] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 7.20, Last appended by leader: 20, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:51.022670  4360 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Advancing to term 8
I20260812 06:36:51.022737  4352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 5 candidate_status { last_received { term: 4 index: 14 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:51.022943  4330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 20 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:36:51.023036  4350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 8 candidate_status { last_received { term: 7 index: 17 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:51.023216  4334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 20 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:51.023231  4333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:51.022678  4345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:36:51.023865  4360 raft_consensus.cc:1275] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 8 FOLLOWER]: Refusing update from remote peer d8c36d75aa834cca96b042eb3bf8a6d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 20. Preceding OpId from leader: term: 8 index: 21. (index mismatch)
I20260812 06:36:51.024122  4583 raft_consensus.cc:1240] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 8 LEADER]: Rejecting Update request from peer df4e0103d6e942fb850d4f18b1cd2b85 for earlier term 7. Current term is 8. Ops: []
I20260812 06:36:51.024199  4354 raft_consensus.cc:3055] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 4 LEADER]: Stepping down as leader of term 4
I20260812 06:36:51.024253  4354 raft_consensus.cc:740] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: df4e0103d6e942fb850d4f18b1cd2b85, State: Running, Role: LEADER
I20260812 06:36:51.024356  4331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 20 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:51.024389  4354 consensus_queue.cc:260] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 4.14, Last appended by leader: 14, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:51.024467  4331 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8c36d75aa834cca96b042eb3bf8a6d2 in term 8.
I20260812 06:36:51.024538  4354 raft_consensus.cc:3060] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 4 FOLLOWER]: Advancing to term 5
I20260812 06:36:51.024772  4361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 5 candidate_status { last_received { term: 4 index: 14 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:36:51.024833  4353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 20 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:36:51.025002  4329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:51.025220  4346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:36:51.025424  4327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 5 candidate_status { last_received { term: 4 index: 14 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:51.025511  4354 raft_consensus.cc:1275] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 5 FOLLOWER]: Refusing update from remote peer c8af5f280df94129ba27e02fff9356a3: Log matching property violated. Preceding OpId in replica: term: 4 index: 14. Preceding OpId from leader: term: 5 index: 16. (index mismatch)
I20260812 06:36:51.025996  4353 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8c36d75aa834cca96b042eb3bf8a6d2 in term 8.
I20260812 06:36:51.026037  4342 raft_consensus.cc:3055] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 6 LEADER]: Stepping down as leader of term 6
I20260812 06:36:51.026085  4342 raft_consensus.cc:740] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: df4e0103d6e942fb850d4f18b1cd2b85, State: Running, Role: LEADER
I20260812 06:36:51.026206  4342 consensus_queue.cc:260] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 6.14, Last appended by leader: 14, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:51.026376  4342 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 6 FOLLOWER]: Advancing to term 7
I20260812 06:36:51.027287  4342 raft_consensus.cc:1275] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Refusing update from remote peer c8af5f280df94129ba27e02fff9356a3: Log matching property violated. Preceding OpId in replica: term: 6 index: 14. Preceding OpId from leader: term: 7 index: 15. (index mismatch)
I20260812 06:36:51.027514  4315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 5 candidate_status { last_received { term: 4 index: 14 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:36:51.027786  4335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 8 candidate_status { last_received { term: 7 index: 17 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:51.027980  4335 raft_consensus.cc:2393] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c8af5f280df94129ba27e02fff9356a3 in current term 8: Already voted for candidate d8c36d75aa834cca96b042eb3bf8a6d2 in this term.
I20260812 06:36:51.028460  4315 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 5.
I20260812 06:36:51.032773  4574 raft_consensus.cc:1240] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Rejecting Update request from peer df4e0103d6e942fb850d4f18b1cd2b85 for earlier term 6. Current term is 7. Ops: []
I20260812 06:36:51.033105  4583 raft_consensus.cc:1240] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 FOLLOWER]: Rejecting Update request from peer df4e0103d6e942fb850d4f18b1cd2b85 for earlier term 4. Current term is 5. Ops: []
W20260812 06:36:51.580050  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:51.054720 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.129:40455 (local address 127.0.0.1:45248)
0812 06:36:51.054889 (+   169us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:51.054905 (+    16us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:51.054991 (+    86us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:51.579929 (+524938us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 06:36:52.057230  5206 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d8c36d75aa834cca96b042eb3bf8a6d2)
I20260812 06:36:52.057356  5206 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
W20260812 06:36:52.057540  4244 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260812 06:36:52.057753  5206 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
W20260812 06:36:52.058148  4245 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260812 06:36:52.058310  5309 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:52.058387  5309 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:52.058738  5309 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:36:52.058844  5309 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:52.058940  5309 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:52.059194  4118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 6 candidate_status { last_received { term: 4 index: 14 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:52.059252  5309 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
W20260812 06:36:52.059875  4241 thread.cc:641] server-negotiator [worker] (thread pool) Time spent creating pthread: real 1.028s	user 0.000s	sys 0.001s
W20260812 06:36:52.059928  4241 thread.cc:608] server-negotiator [worker] (thread pool) Time spent starting thread: real 1.028s	user 0.000s	sys 0.001s
I20260812 06:36:52.059901  4118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:52.060973  5206 consensus_queue.cc:1048] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260812 06:36:52.061062  5313 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:51.032215 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:48988 (local address 127.3.193.130:35779)
0812 06:36:52.060586 (+1028371us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:52.060592 (+     6us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:52.060612 (+    20us) server_negotiation.cc:408] Connection header received
0812 06:36:52.060662 (+    50us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:52.060667 (+     5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:52.060739 (+    72us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:52.060893 (+   154us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48988: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":1028259,"thread_start_us":1027707,"threads_started":1}
I20260812 06:36:52.061537  5206 consensus_queue.cc:1048] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260812 06:36:52.061791  5313 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:52.060027 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:48990 (local address 127.3.193.130:35779)
0812 06:36:52.061367 (+  1340us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:52.061372 (+     5us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:52.061390 (+    18us) server_negotiation.cc:408] Connection header received
0812 06:36:52.061439 (+    49us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:52.061445 (+     6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:52.061515 (+    70us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:52.061643 (+   128us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48990: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":1220,"thread_start_us":46,"threads_started":1}
I20260812 06:36:52.063701  5309 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:52.065979  5309 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:52.066818  4114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 9 candidate_status { last_received { term: 7 index: 20 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:52.066994  4114 raft_consensus.cc:2410] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate df4e0103d6e942fb850d4f18b1cd2b85 for term 9 because replica has last-logged OpId of term: 8 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 20.
I20260812 06:36:52.065563  4315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 9 candidate_status { last_received { term: 8 index: 21 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:52.067569  4315 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 8.
I20260812 06:36:52.068701  4032 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:52.068897  5309 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:52.068904  5206 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260812 06:36:52.068959  5206 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d8c36d75aa834cca96b042eb3bf8a6d2)
I20260812 06:36:52.068995  5206 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Advancing to term 9
I20260812 06:36:52.070183  5206 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:52.070525  5206 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 9 election: Requested vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:52.071867  4335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 9 candidate_status { last_received { term: 8 index: 21 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:36:52.071969  4335 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 8 FOLLOWER]: Advancing to term 9
I20260812 06:36:52.073045  4335 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 9.
I20260812 06:36:52.073767  4032 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:52.073971  5206 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 9 FOLLOWER]: Leader election won for term 9
I20260812 06:36:52.074141  5206 raft_consensus.cc:697] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 9 LEADER]: Becoming Leader. State: Replica: c8af5f280df94129ba27e02fff9356a3, State: Running, Role: LEADER
I20260812 06:36:52.074532  5206 consensus_queue.cc:237] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 8.21, Last appended by leader: 21, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:52.077479  3906 catalog_manager.cc:5719] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 reported cstate change: term changed from 8 to 9, leader changed from d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131) to c8af5f280df94129ba27e02fff9356a3 (127.3.193.129). New cstate: current_term: 9 leader_uuid: "c8af5f280df94129ba27e02fff9356a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: UNKNOWN } } }
W20260812 06:36:52.078650  5316 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:52.065639 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:45248 (local address 127.3.193.129:40455)
0812 06:36:52.078237 (+ 12598us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:52.078242 (+     5us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:52.078262 (+    20us) server_negotiation.cc:408] Connection header received
0812 06:36:52.078314 (+    52us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:52.078317 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:52.078388 (+    71us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:52.078532 (+   144us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45248: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":12494,"thread_start_us":74,"threads_started":1}
W20260812 06:36:52.091993  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:51.581953 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.131:38503 (local address 127.0.0.1:46034)
0812 06:36:51.582122 (+   169us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:51.582138 (+    16us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:51.582235 (+    97us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:52.091889 (+509654us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.131:38503: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":59}
W20260812 06:36:52.092206  4683 meta_cache.cc:302] tablet 27b60a3088e74447b13844f7b49da4ff: replica d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.131:38503: received 0 of 4 requested bytes
I20260812 06:36:52.102361  4342 raft_consensus.cc:1275] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 9 FOLLOWER]: Refusing update from remote peer c8af5f280df94129ba27e02fff9356a3: Log matching property violated. Preceding OpId in replica: term: 7 index: 20. Preceding OpId from leader: term: 9 index: 22. (index mismatch)
I20260812 06:36:52.103022  5341 consensus_queue.cc:1048] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.000s
I20260812 06:36:52.125948  5266 consensus_queue.cc:1048] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260812 06:36:52.126317  5266 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:52.126426  5266 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:52.126789  5266 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:36:52.126930  5266 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:52.126983  5266 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:52.127224  5266 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:36:52.127533  4586 raft_consensus.cc:3055] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 8 LEADER]: Stepping down as leader of term 8
I20260812 06:36:52.127584  4586 raft_consensus.cc:740] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d8c36d75aa834cca96b042eb3bf8a6d2, State: Running, Role: LEADER
I20260812 06:36:52.127599  4578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 6 candidate_status { last_received { term: 4 index: 14 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:52.127717  4567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 9 candidate_status { last_received { term: 7 index: 20 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:52.127771  4578 raft_consensus.cc:2410] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate df4e0103d6e942fb850d4f18b1cd2b85 for term 6 because replica has last-logged OpId of term: 5 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 14.
I20260812 06:36:52.127745  4586 consensus_queue.cc:260] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 8.21, Last appended by leader: 21, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:52.127888  4586 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 8 FOLLOWER]: Advancing to term 9
I20260812 06:36:52.127957  4583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 9 candidate_status { last_received { term: 8 index: 21 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:52.128160  4566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:52.128912  4342 raft_consensus.cc:1217] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 9 FOLLOWER]: Deduplicated request from leader. Original: 7.20->[8.21-8.21]   Dedup: 8.21->[]
I20260812 06:36:52.129000  4342 raft_consensus.cc:1240] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 9 FOLLOWER]: Rejecting Update request from peer d8c36d75aa834cca96b042eb3bf8a6d2 for earlier term 8. Current term is 9. Ops: [8.21-8.21]
I20260812 06:36:52.128976  4586 raft_consensus.cc:1275] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 9 FOLLOWER]: Refusing update from remote peer c8af5f280df94129ba27e02fff9356a3: Log matching property violated. Preceding OpId in replica: term: 8 index: 21. Preceding OpId from leader: term: 9 index: 22. (index mismatch)
I20260812 06:36:52.129232  4315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 16 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:52.129487  4582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 9 candidate_status { last_received { term: 8 index: 21 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:52.129765  4245 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df4e0103d6e942fb850d4f18b1cd2b85; no voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2
I20260812 06:36:52.129941  5323 raft_consensus.cc:2749] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260812 06:36:52.130013  4245 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df4e0103d6e942fb850d4f18b1cd2b85; no voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2
I20260812 06:36:52.130158  5323 raft_consensus.cc:2749] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 06:36:52.130232  4245 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df4e0103d6e942fb850d4f18b1cd2b85; no voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2
I20260812 06:36:52.130350  5323 raft_consensus.cc:2749] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 06:36:52.130426  4582 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 9.
I20260812 06:36:52.130894  4335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 17 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
W20260812 06:36:52.131389  5350 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:52.130560 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:46034 (local address 127.3.193.131:38503)
0812 06:36:52.130796 (+   236us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:52.130800 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:52.130816 (+    16us) server_negotiation.cc:408] Connection header received
0812 06:36:52.130861 (+    45us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:52.130865 (+     4us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:52.130926 (+    61us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:52.131285 (+   359us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46034: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":81,"threads_started":1}
W20260812 06:36:52.668051  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:52.158685 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.129:40455 (local address 127.0.0.1:45284)
0812 06:36:52.158852 (+   167us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:52.158895 (+    43us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:52.158988 (+    93us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:52.667937 (+508949us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 06:36:53.159667  5323 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:53.159832  5323 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:53.159036  5309 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:53.160425  5309 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:53.160490  5323 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:53.160557  5323 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:53.160606  5323 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:53.160743  5309 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:53.160871  5323 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
W20260812 06:36:53.161720  4034 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.082s (SENT) [suppressed 6 similar messages]
I20260812 06:36:53.163431  5363 consensus_queue.cc:1048] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
W20260812 06:36:53.165839  5375 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:53.164604 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:45284 (local address 127.3.193.129:40455)
0812 06:36:53.165437 (+   833us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:53.165441 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:53.165454 (+    13us) server_negotiation.cc:408] Connection header received
0812 06:36:53.165492 (+    38us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:53.165495 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:53.165539 (+    44us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:53.165655 (+   116us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45284: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":764,"thread_start_us":57,"threads_started":1}
I20260812 06:36:53.166136  4114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 16 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:53.166802  4118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 17 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:53.167497  4114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 18 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:53.167591  4104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 6 candidate_status { last_received { term: 5 index: 18 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:53.167614  4118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 10 candidate_status { last_received { term: 9 index: 22 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:53.192538  4581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 10 candidate_status { last_received { term: 9 index: 22 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:53.192698  4581 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 9.
I20260812 06:36:53.193339  4578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 9 candidate_status { last_received { term: 8 index: 21 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:53.193634  4586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 6 candidate_status { last_received { term: 5 index: 18 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:53.193732  4586 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 5.
I20260812 06:36:53.193897  4567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 9 candidate_status { last_received { term: 8 index: 21 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:53.194096  4582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 18 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:53.195231  4245 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85; no voters: c8af5f280df94129ba27e02fff9356a3
I20260812 06:36:53.195488  4245 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85; no voters: c8af5f280df94129ba27e02fff9356a3
I20260812 06:36:53.195696  5266 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:53.195699  4245 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df4e0103d6e942fb850d4f18b1cd2b85; no voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2
I20260812 06:36:53.195775  5266 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:53.195807  5323 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260812 06:36:53.195875  5323 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 9 FOLLOWER]: Starting leader election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:53.195945  5309 raft_consensus.cc:2749] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 06:36:53.195914  5323 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 9 FOLLOWER]: Advancing to term 10
I20260812 06:36:53.196237  5266 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:36:53.196880  4315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 22 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:53.197275  5323 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:53.197652  5323 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 10 election: Requested vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:53.195914  5357 raft_consensus.cc:2804] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 06:36:53.197885  5357 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 5 FOLLOWER]: Starting leader election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:53.197940  5357 raft_consensus.cc:3060] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 5 FOLLOWER]: Advancing to term 6
I20260812 06:36:53.198930  4581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 10 candidate_status { last_received { term: 9 index: 22 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:53.199011  5357 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:53.199350  5357 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 6 election: Requested vote from peers d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:36:53.199620  4581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 6 candidate_status { last_received { term: 5 index: 20 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
W20260812 06:36:53.756012  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:53.223829 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.129:40455 (local address 127.0.0.1:45314)
0812 06:36:53.223989 (+   160us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:53.224005 (+    16us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:53.224094 (+    89us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:53.755902 (+531808us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 06:36:53.756317  4681 meta_cache.cc:302] tablet 27b60a3088e74447b13844f7b49da4ff: replica c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
I20260812 06:36:54.224609  4104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 17 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:54.224852  4110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 10 candidate_status { last_received { term: 9 index: 22 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:54.225129  4118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 22 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:54.225241  4133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 6 candidate_status { last_received { term: 5 index: 20 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
W20260812 06:36:54.225308  5373 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:54.224437 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:45314 (local address 127.3.193.129:40455)
0812 06:36:54.224894 (+   457us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:54.224898 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:54.224915 (+    17us) server_negotiation.cc:408] Connection header received
0812 06:36:54.224964 (+    49us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:54.224970 (+     6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:54.225053 (+    83us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:54.225196 (+   143us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45314: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":363}
I20260812 06:36:54.226769  4114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 16 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
W20260812 06:36:54.228591  4490 leader_election.cc:336] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.087s (SENT)
I20260812 06:36:54.228673  4490 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:36:54.228982  5393 raft_consensus.cc:2749] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260812 06:36:54.229032  4490 leader_election.cc:336] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.097s (SENT)
I20260812 06:36:54.229085  4490 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:36:54.230026  5399 raft_consensus.cc:2749] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 06:36:54.230819  4114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 22 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:54.231261  4490 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:36:54.231480  5393 raft_consensus.cc:2749] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260812 06:36:54.268003  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:53.759063 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.130:35779 (local address 127.0.0.1:48998)
0812 06:36:53.759239 (+   176us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:53.759255 (+    16us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:53.759349 (+    94us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:54.267886 (+508537us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":61}
W20260812 06:36:54.278261  4530 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46048: Illegal state: replica d8c36d75aa834cca96b042eb3bf8a6d2 is not leader of this config: current role FOLLOWER
W20260812 06:36:54.311681  4244 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.098s (SENT) [suppressed 12 similar messages]
I20260812 06:36:54.312121  4335 raft_consensus.cc:1240] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 6 FOLLOWER]: Rejecting Update request from peer c8af5f280df94129ba27e02fff9356a3 for earlier term 5. Current term is 6. Ops: []
W20260812 06:36:54.313059  5428 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:54.312384 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:48998 (local address 127.3.193.130:35779)
0812 06:36:54.312677 (+   293us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:54.312682 (+     5us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:54.312697 (+    15us) server_negotiation.cc:408] Connection header received
0812 06:36:54.312740 (+    43us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:54.312743 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:54.312797 (+    54us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:54.312940 (+   143us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48998: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":114,"threads_started":1}
I20260812 06:36:54.313158  5364 consensus_queue.cc:1059] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 }, Status: INVALID_TERM, Last received: 5.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260812 06:36:54.313388  5364 raft_consensus.cc:3055] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 5 LEADER]: Stepping down as leader of term 5
I20260812 06:36:54.313438  5364 raft_consensus.cc:740] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c8af5f280df94129ba27e02fff9356a3, State: Running, Role: LEADER
I20260812 06:36:54.313597  5364 consensus_queue.cc:260] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 5.21, Last appended by leader: 21, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:54.313733  5364 raft_consensus.cc:3060] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 5 FOLLOWER]: Advancing to term 6
I20260812 06:36:54.317281  5357 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:54.317377  5357 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:54.317848  5357 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:54.318224  4583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 21 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:54.318889  4342 raft_consensus.cc:1240] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 10 FOLLOWER]: Rejecting Update request from peer c8af5f280df94129ba27e02fff9356a3 for earlier term 9. Current term is 10. Ops: []
W20260812 06:36:54.876015  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:54.344229 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.129:40455 (local address 127.0.0.1:45350)
0812 06:36:54.344381 (+   152us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:54.344396 (+    15us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:54.344497 (+   101us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:54.875898 (+531401us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 06:36:54.876405  4681 meta_cache.cc:302] tablet 3cf285c378e04319a5acdcae46870291: replica c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
I20260812 06:36:55.344812  4118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 6 candidate_status { last_received { term: 5 index: 20 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:55.344882  4133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 10 candidate_status { last_received { term: 9 index: 22 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:55.344839  4114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 21 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:55.345227  5362 consensus_queue.cc:1059] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 }, Status: INVALID_TERM, Last received: 9.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260812 06:36:55.345466  5362 raft_consensus.cc:3055] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 9 LEADER]: Stepping down as leader of term 9
I20260812 06:36:55.345530  5362 raft_consensus.cc:740] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c8af5f280df94129ba27e02fff9356a3, State: Running, Role: LEADER
I20260812 06:36:55.345667  5362 consensus_queue.cc:260] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 9.22, Last appended by leader: 22, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:55.345841  5362 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 9 FOLLOWER]: Advancing to term 10
I20260812 06:36:55.344885  5341 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:55.346355  5341 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
W20260812 06:36:55.346458  5373 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:55.345689 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:45350 (local address 127.3.193.129:40455)
0812 06:36:55.346149 (+   460us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:55.346154 (+     5us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:55.346169 (+    15us) server_negotiation.cc:408] Connection header received
0812 06:36:55.346213 (+    44us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:55.346216 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:55.346262 (+    46us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:55.346375 (+   113us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45350: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":373}
I20260812 06:36:55.346730  5341 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:55.347018  4118 raft_consensus.cc:2410] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate df4e0103d6e942fb850d4f18b1cd2b85 for term 6 because replica has last-logged OpId of term: 5 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 20.
I20260812 06:36:55.349608  4581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 7 candidate_status { last_received { term: 5 index: 21 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:55.349727  4581 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 5.
I20260812 06:36:55.350009  5437 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:55.350145  5437 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:55.350564  5437 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:36:55.350589  4034 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2; no voters: 
I20260812 06:36:55.350682  5437 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:55.350739  5437 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:55.350878  5341 raft_consensus.cc:2804] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 06:36:55.350926  5341 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:36:55.350961  5341 raft_consensus.cc:3060] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 6 FOLLOWER]: Advancing to term 7
I20260812 06:36:55.350984  5437 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:36:55.351683  4118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 21 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:55.351963  5341 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:55.352299  5341 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 7 election: Requested vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:55.352434  4114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 22 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:55.352520  4114 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8c36d75aa834cca96b042eb3bf8a6d2 in term 10.
I20260812 06:36:55.352949  4118 raft_consensus.cc:2368] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8c36d75aa834cca96b042eb3bf8a6d2 for earlier term 6. Current term is 7.
I20260812 06:36:55.353103  4490 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2; no voters: 
I20260812 06:36:55.353365  5434 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260812 06:36:55.353423  4490 leader_election.cc:400] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 6 pre-election: Vote denied by peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455) with higher term. Message: Invalid argument: T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8c36d75aa834cca96b042eb3bf8a6d2 for earlier term 6. Current term is 7.
I20260812 06:36:55.353446  5434 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:55.353474  4490 leader_election.cc:403] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260812 06:36:55.353487  5434 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 9 FOLLOWER]: Advancing to term 10
I20260812 06:36:55.353765  4583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 7 candidate_status { last_received { term: 5 index: 21 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:55.353864  4583 raft_consensus.cc:3060] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 5 FOLLOWER]: Advancing to term 7
I20260812 06:36:55.354435  5434 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:55.354712  4583 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 7.
I20260812 06:36:55.354866  5434 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 10 election: Requested vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:36:55.355015  5434 raft_consensus.cc:2749] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455) with higher term. Message: Invalid argument: T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8c36d75aa834cca96b042eb3bf8a6d2 for earlier term 6. Current term is 7.
I20260812 06:36:55.355134  4034 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2; no voters: 
I20260812 06:36:55.355366  4118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 22 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:55.355381  5341 raft_consensus.cc:2804] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 7 FOLLOWER]: Leader election won for term 7
I20260812 06:36:55.355494  5341 raft_consensus.cc:697] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 7 LEADER]: Becoming Leader. State: Replica: c8af5f280df94129ba27e02fff9356a3, State: Running, Role: LEADER
I20260812 06:36:55.355616  5341 consensus_queue.cc:237] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 5.21, Last appended by leader: 21, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:55.356351  4118 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8c36d75aa834cca96b042eb3bf8a6d2 in term 10.
I20260812 06:36:55.356949  4490 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2; no voters: 
I20260812 06:36:55.357162  5437 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 10 FOLLOWER]: Leader election won for term 10
I20260812 06:36:55.357234  5437 raft_consensus.cc:697] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 10 LEADER]: Becoming Leader. State: Replica: d8c36d75aa834cca96b042eb3bf8a6d2, State: Running, Role: LEADER
I20260812 06:36:55.357257  3905 catalog_manager.cc:5719] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "c8af5f280df94129ba27e02fff9356a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: HEALTHY } } }
I20260812 06:36:55.357340  5437 consensus_queue.cc:237] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 9.22, Last appended by leader: 22, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:55.359025  3906 catalog_manager.cc:5719] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 reported cstate change: term changed from 9 to 10, leader changed from c8af5f280df94129ba27e02fff9356a3 (127.3.193.129) to d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131). New cstate: current_term: 10 leader_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: HEALTHY } } }
I20260812 06:36:55.377766  4583 raft_consensus.cc:1275] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Refusing update from remote peer c8af5f280df94129ba27e02fff9356a3: Log matching property violated. Preceding OpId in replica: term: 5 index: 21. Preceding OpId from leader: term: 7 index: 22. (index mismatch)
I20260812 06:36:55.378228  5362 consensus_queue.cc:1048] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260812 06:36:55.386910  4118 raft_consensus.cc:1275] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 10 FOLLOWER]: Refusing update from remote peer d8c36d75aa834cca96b042eb3bf8a6d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 22. Preceding OpId from leader: term: 10 index: 23. (index mismatch)
I20260812 06:36:55.387399  5434 consensus_queue.cc:1048] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
W20260812 06:36:55.387979  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:54.882173 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.130:35779 (local address 127.0.0.1:49014)
0812 06:36:54.882349 (+   176us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:54.882365 (+    16us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:54.882460 (+    95us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:55.387866 (+505406us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":66}
W20260812 06:36:55.394090  4530 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46064: Illegal state: replica d8c36d75aa834cca96b042eb3bf8a6d2 is not leader of this config: current role FOLLOWER
W20260812 06:36:55.424778  4244 leader_election.cc:336] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.097s (SENT)
I20260812 06:36:55.424881  4244 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df4e0103d6e942fb850d4f18b1cd2b85; no voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2
W20260812 06:36:55.425091  4244 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.098s (SENT)
I20260812 06:36:55.425146  4244 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df4e0103d6e942fb850d4f18b1cd2b85; no voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2
I20260812 06:36:55.425530  4335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 22 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:55.425668  5357 raft_consensus.cc:2749] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260812 06:36:55.425982  4315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 22 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:36:55.426299  4353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 7 candidate_status { last_received { term: 5 index: 21 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:55.426410  4353 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 6.
I20260812 06:36:55.425683  4335 raft_consensus.cc:2393] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8c36d75aa834cca96b042eb3bf8a6d2 in current term 10: Already voted for candidate df4e0103d6e942fb850d4f18b1cd2b85 in this term.
I20260812 06:36:55.426955  4342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 21 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:55.427012  5357 raft_consensus.cc:2749] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260812 06:36:55.427119  4342 raft_consensus.cc:2393] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8c36d75aa834cca96b042eb3bf8a6d2 in current term 6: Already voted for candidate df4e0103d6e942fb850d4f18b1cd2b85 in this term.
I20260812 06:36:55.427186  4354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 7 candidate_status { last_received { term: 5 index: 21 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:36:55.427270  4354 raft_consensus.cc:3060] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 6 FOLLOWER]: Advancing to term 7
W20260812 06:36:55.427740  5428 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:55.425585 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:49014 (local address 127.3.193.130:35779)
0812 06:36:55.426160 (+   575us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:55.426164 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:55.426179 (+    15us) server_negotiation.cc:408] Connection header received
0812 06:36:55.426215 (+    36us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:55.426218 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:55.426261 (+    43us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:55.426382 (+   121us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49014: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":494}
I20260812 06:36:55.427975  4335 raft_consensus.cc:1275] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 10 FOLLOWER]: Refusing update from remote peer d8c36d75aa834cca96b042eb3bf8a6d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 22. Preceding OpId from leader: term: 10 index: 23. (index mismatch)
I20260812 06:36:55.428313  4354 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 7.
I20260812 06:36:55.428728  5434 consensus_queue.cc:1048] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260812 06:36:55.428870  4346 raft_consensus.cc:1275] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Refusing update from remote peer c8af5f280df94129ba27e02fff9356a3: Log matching property violated. Preceding OpId in replica: term: 5 index: 20. Preceding OpId from leader: term: 7 index: 22. (index mismatch)
W20260812 06:36:55.964118  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:55.457430 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.129:40455 (local address 127.0.0.1:52672)
0812 06:36:55.457630 (+   200us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:55.457644 (+    14us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:55.457723 (+    79us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:55.963988 (+506265us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":86}
W20260812 06:36:55.964313  4681 meta_cache.cc:302] tablet 3cf285c378e04319a5acdcae46870291: replica c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
I20260812 06:36:56.457948  5476 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d8c36d75aa834cca96b042eb3bf8a6d2)
I20260812 06:36:56.458168  5476 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
W20260812 06:36:56.458550  4244 leader_election.cc:336] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.106s (SENT)
I20260812 06:36:56.458617  4244 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df4e0103d6e942fb850d4f18b1cd2b85; no voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2
I20260812 06:36:56.458704  5476 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:56.458782  5476 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:56.458844  5476 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:56.459242  5476 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:36:56.459384  5476 raft_consensus.cc:2749] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 06:36:56.460893  4118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 21 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:56.460997  4133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 5 index: 20 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:56.461208  4114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 11 candidate_status { last_received { term: 10 index: 24 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:56.461314  4114 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 10.
I20260812 06:36:56.461611  5364 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d8c36d75aa834cca96b042eb3bf8a6d2)
I20260812 06:36:56.461697  5364 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:56.462047  5341 consensus_queue.cc:1048] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.000s
I20260812 06:36:56.463618  4244 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:56.462065  5364 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:56.463769  4327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 11 candidate_status { last_received { term: 10 index: 24 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:56.463876  4327 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 10.
I20260812 06:36:56.463958  5476 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260812 06:36:56.464002  5476 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 10 FOLLOWER]: Starting leader election (detected failure of leader d8c36d75aa834cca96b042eb3bf8a6d2)
I20260812 06:36:56.464037  5476 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 10 FOLLOWER]: Advancing to term 11
I20260812 06:36:56.465137  5476 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:56.465370  4032 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:56.465565  5364 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260812 06:36:56.465611  5364 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader d8c36d75aa834cca96b042eb3bf8a6d2)
I20260812 06:36:56.465646  5364 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 10 FOLLOWER]: Advancing to term 11
I20260812 06:36:56.465709  4114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 11 candidate_status { last_received { term: 10 index: 24 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:56.465984  5476 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 11 election: Requested vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:56.466609  5364 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:56.466934  5364 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 11 election: Requested vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:56.467056  4114 raft_consensus.cc:2393] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate df4e0103d6e942fb850d4f18b1cd2b85 in current term 11: Already voted for candidate c8af5f280df94129ba27e02fff9356a3 in this term.
W20260812 06:36:56.467401  5484 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:56.463094 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:52672 (local address 127.3.193.129:40455)
0812 06:36:56.467040 (+  3946us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:56.467045 (+     5us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:56.467060 (+    15us) server_negotiation.cc:408] Connection header received
0812 06:36:56.467104 (+    44us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:56.467107 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:56.467154 (+    47us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:56.467300 (+   146us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52672: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":3850,"thread_start_us":45,"threads_started":1}
I20260812 06:36:56.468204  4335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 11 candidate_status { last_received { term: 10 index: 24 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:36:56.468336  4335 raft_consensus.cc:2393] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c8af5f280df94129ba27e02fff9356a3 in current term 11: Already voted for candidate df4e0103d6e942fb850d4f18b1cd2b85 in this term.
I20260812 06:36:56.468477  4342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 7 candidate_status { last_received { term: 5 index: 21 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:56.468798  4327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 7 candidate_status { last_received { term: 5 index: 21 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
W20260812 06:36:56.470840  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:55.970083 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.130:35779 (local address 127.0.0.1:37066)
0812 06:36:55.970265 (+   182us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:55.970280 (+    15us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:55.970378 (+    98us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:56.459013 (+488635us) client_negotiation.cc:281] Received NEGOTIATE NegotiatePB response
0812 06:36:56.459018 (+     5us) client_negotiation.cc:404] Received NEGOTIATE response from server
0812 06:36:56.459030 (+    12us) client_negotiation.cc:190] Negotiated authn=TOKEN
0812 06:36:56.459285 (+   255us) client_negotiation.cc:530] Sending TLS_HANDSHAKE message to server
0812 06:36:56.459299 (+    14us) client_negotiation.cc:262] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:36:56.469904 (+ 10605us) client_negotiation.cc:281] Received TLS_HANDSHAKE NegotiatePB response
0812 06:36:56.469911 (+     7us) client_negotiation.cc:543] Received TLS_HANDSHAKE response from server
0812 06:36:56.470560 (+   649us) client_negotiation.cc:530] Sending TLS_HANDSHAKE message to server
0812 06:36:56.470572 (+    12us) client_negotiation.cc:262] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:36:56.470715 (+   143us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: sent 0 of 16 requested bytes
Metrics: {"client-negotiator.queue_time_us":62}
W20260812 06:36:56.471485  5477 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:56.458297 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:37066 (local address 127.3.193.130:35779)
0812 06:36:56.458546 (+   249us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:56.458551 (+     5us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:56.458567 (+    16us) server_negotiation.cc:408] Connection header received
0812 06:36:56.458624 (+    57us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:56.458627 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:56.458691 (+    64us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:56.459445 (+   754us) server_negotiation.cc:227] Negotiated authn=TOKEN
0812 06:36:56.459543 (+    98us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0812 06:36:56.469741 (+ 10198us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:36:56.471360 (+  1619us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37066: BlockingRecv error: recv got EOF from 127.0.0.1:37066 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":64,"threads_started":1}
W20260812 06:36:56.613346  4245 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503): Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.076s (SENT)
W20260812 06:36:56.627382  4245 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503): Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.079s (SENT)
I20260812 06:36:56.627511  4245 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df4e0103d6e942fb850d4f18b1cd2b85; no voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2
I20260812 06:36:56.627748  5478 raft_consensus.cc:2749] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260812 06:36:56.645921  4034 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503): Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.091s (SENT)
W20260812 06:36:56.684883  4034 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503): Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.108s (SENT)
I20260812 06:36:56.684985  4034 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3; no voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:36:56.685194  5481 raft_consensus.cc:2749] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260812 06:36:56.718457  5478 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:56.718648  5478 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:56.719197  5478 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:56.719615  4114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 12 candidate_status { last_received { term: 10 index: 24 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:56.719813  4114 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 11.
I20260812 06:36:56.720103  4244 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:56.720296  5478 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260812 06:36:56.720372  5478 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:36:56.720402  5478 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 11 FOLLOWER]: Advancing to term 12
I20260812 06:36:56.721343  5478 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:56.721714  5478 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 12 election: Requested vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:56.722090  4114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 12 candidate_status { last_received { term: 10 index: 24 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:56.722205  4114 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 11 FOLLOWER]: Advancing to term 12
I20260812 06:36:56.723033  4114 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 12.
I20260812 06:36:56.723341  4244 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:56.723572  5478 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 12 FOLLOWER]: Leader election won for term 12
I20260812 06:36:56.723735  5478 raft_consensus.cc:697] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 12 LEADER]: Becoming Leader. State: Replica: df4e0103d6e942fb850d4f18b1cd2b85, State: Running, Role: LEADER
I20260812 06:36:56.723958  5478 consensus_queue.cc:237] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 10.24, Last appended by leader: 24, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:56.725688  3905 catalog_manager.cc:5719] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 reported cstate change: term changed from 10 to 12, leader changed from d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131) to df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130). New cstate: current_term: 12 leader_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: UNKNOWN } } }
I20260812 06:36:56.750296  4114 raft_consensus.cc:1275] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 12 FOLLOWER]: Refusing update from remote peer df4e0103d6e942fb850d4f18b1cd2b85: Log matching property violated. Preceding OpId in replica: term: 10 index: 24. Preceding OpId from leader: term: 12 index: 25. (index mismatch)
I20260812 06:36:56.750799  5478 consensus_queue.cc:1048] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
W20260812 06:36:56.792362  4245 leader_election.cc:336] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503): Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.166s (SENT)
I20260812 06:36:56.792549  4245 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df4e0103d6e942fb850d4f18b1cd2b85; no voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2
I20260812 06:36:56.792740  5478 raft_consensus.cc:2749] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260812 06:36:56.955289  4245 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503): Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.117s (SENT)
W20260812 06:36:56.973753  4245 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503): Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.125s (SENT)
W20260812 06:36:56.988104  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:56.475756 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.131:38503 (local address 127.0.0.1:58416)
0812 06:36:56.475919 (+   163us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:56.475936 (+    17us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:56.476019 (+    83us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:56.987990 (+511971us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.131:38503: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 06:36:56.988279  4683 meta_cache.cc:302] tablet 3cf285c378e04319a5acdcae46870291: replica d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.131:38503: received 0 of 4 requested bytes
I20260812 06:36:57.022526  4583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 11 candidate_status { last_received { term: 10 index: 24 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:57.022917  4583 raft_consensus.cc:3055] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 10 LEADER]: Stepping down as leader of term 10
I20260812 06:36:57.022994  4583 raft_consensus.cc:740] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d8c36d75aa834cca96b042eb3bf8a6d2, State: Running, Role: LEADER
I20260812 06:36:57.023130  4567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 11 candidate_status { last_received { term: 10 index: 24 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:57.023162  4583 consensus_queue.cc:260] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 10.24, Last appended by leader: 24, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:57.023172  4566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 11 candidate_status { last_received { term: 10 index: 24 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:57.023329  4583 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 10 FOLLOWER]: Advancing to term 12
I20260812 06:36:57.023375  4574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 11 candidate_status { last_received { term: 10 index: 24 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:57.023401  4590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 12 candidate_status { last_received { term: 10 index: 24 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:57.023619  4593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 11 candidate_status { last_received { term: 10 index: 24 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:57.023885  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 12 candidate_status { last_received { term: 10 index: 24 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:57.024118  4587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 11 candidate_status { last_received { term: 10 index: 24 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:57.024183  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 5 index: 20 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:57.024345  4588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 11 candidate_status { last_received { term: 10 index: 24 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:57.024569  4595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 12 candidate_status { last_received { term: 10 index: 24 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:57.024673  4583 raft_consensus.cc:1275] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 12 FOLLOWER]: Refusing update from remote peer df4e0103d6e942fb850d4f18b1cd2b85: Log matching property violated. Preceding OpId in replica: term: 10 index: 24. Preceding OpId from leader: term: 12 index: 25. (index mismatch)
I20260812 06:36:57.024781  5437 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:57.024860  5437 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:57.025259  5437 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:36:57.024125  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 12 candidate_status { last_received { term: 10 index: 24 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:57.025991  4597 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 12.
I20260812 06:36:57.023633  4578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 11 candidate_status { last_received { term: 10 index: 24 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:57.026295  4578 raft_consensus.cc:2368] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate df4e0103d6e942fb850d4f18b1cd2b85 for earlier term 11. Current term is 12.
I20260812 06:36:57.027478  5437 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:57.027566  5437 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:57.027935  5437 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:36:57.029042  4354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 22 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:57.023259  4581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 5 index: 20 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:57.029726  5478 consensus_queue.cc:1048] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
W20260812 06:36:57.029913  5527 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:57.029289 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:58416 (local address 127.3.193.131:38503)
0812 06:36:57.029547 (+   258us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:57.029551 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:57.029566 (+    15us) server_negotiation.cc:408] Connection header received
0812 06:36:57.029611 (+    45us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:57.029616 (+     5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:57.029664 (+    48us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:57.029798 (+   134us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58416: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":113,"threads_started":1}
I20260812 06:36:57.030828  4354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
W20260812 06:36:57.564142  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:57.054752 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.129:40455 (local address 127.0.0.1:52724)
0812 06:36:57.054960 (+   208us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:57.054979 (+    19us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:57.055086 (+   107us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:57.564021 (+508935us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":77}
W20260812 06:36:58.055163  4114 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.3.193.131:43093 (request call id 89) took 1026 ms (client timeout 104 ms). Trace:
W20260812 06:36:58.055258  4114 rpcz_store.cc:269] 0812 06:36:57.028366 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:58.055089 (+1026723us) service_pool.cc:211] Skipping call since client already timed out
0812 06:36:58.055145 (+    56us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:36:58.055409  5363 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:58.055537  5363 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
W20260812 06:36:58.055740  4133 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.3.193.131:43093 (request call id 90) took 1027 ms (client timeout 110 ms). Trace:
W20260812 06:36:58.055787  4133 rpcz_store.cc:269] 0812 06:36:58.054895 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:58.055695 (+   800us) service_pool.cc:211] Skipping call since client already timed out
0812 06:36:58.055732 (+    37us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:36:58.056057  5363 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
W20260812 06:36:58.057394  4490 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.104s (SENT) [suppressed 7 similar messages]
I20260812 06:36:58.058243  5517 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:58.058344  5517 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:58.058665  5517 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:36:58.059252  4595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 13 candidate_status { last_received { term: 12 index: 25 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:58.059253  4133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 22 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:58.059253  4118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 25 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:58.059376  4595 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 12.
I20260812 06:36:58.059410  4118 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8c36d75aa834cca96b042eb3bf8a6d2 in term 12.
I20260812 06:36:58.059852  4114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:58.060129  4034 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2; no voters: 
I20260812 06:36:58.060173  4490 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2; no voters: 
I20260812 06:36:58.060333  4490 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:36:58.060431  5540 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260812 06:36:58.060525  5540 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:58.060606  5540 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 12 FOLLOWER]: Advancing to term 13
I20260812 06:36:58.060789  4490 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:36:58.061587  5437 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260812 06:36:58.061653  5437 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:36:58.061713  5437 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 12 FOLLOWER]: Advancing to term 13
I20260812 06:36:58.061743  5540 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:58.061777  5546 raft_consensus.cc:2749] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 06:36:58.061915  5546 raft_consensus.cc:2749] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 06:36:58.062104  5540 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 13 election: Requested vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
W20260812 06:36:58.062760  5542 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:58.058501 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:52724 (local address 127.3.193.129:40455)
0812 06:36:58.062359 (+  3858us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:58.062388 (+    29us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:58.062405 (+    17us) server_negotiation.cc:408] Connection header received
0812 06:36:58.062456 (+    51us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:58.062459 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:58.062510 (+    51us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:58.062656 (+   146us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52724: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":3767,"thread_start_us":54,"threads_started":1}
I20260812 06:36:58.062780  5437 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:58.063115  5437 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 13 election: Requested vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:36:58.063325  4581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 13 candidate_status { last_received { term: 12 index: 25 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:58.063437  4581 raft_consensus.cc:2393] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c8af5f280df94129ba27e02fff9356a3 in current term 13: Already voted for candidate d8c36d75aa834cca96b042eb3bf8a6d2 in this term.
I20260812 06:36:58.063926  4114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 25 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:58.064065  4114 raft_consensus.cc:2393] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d8c36d75aa834cca96b042eb3bf8a6d2 in current term 13: Already voted for candidate c8af5f280df94129ba27e02fff9356a3 in this term.
W20260812 06:36:58.075973  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:57.569243 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.130:35779 (local address 127.0.0.1:37074)
0812 06:36:57.569469 (+   226us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:57.569485 (+    16us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:57.569607 (+   122us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:58.075836 (+506229us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":104}
W20260812 06:36:58.076185  4680 meta_cache.cc:302] tablet 27b60a3088e74447b13844f7b49da4ff: replica df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
W20260812 06:36:58.082777  4530 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58426: Illegal state: replica d8c36d75aa834cca96b042eb3bf8a6d2 is not leader of this config: current role FOLLOWER
I20260812 06:36:58.114959  4346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 25 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:36:58.115375  4354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 25 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:58.115681  4335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 13 candidate_status { last_received { term: 12 index: 25 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:36:58.116739  5476 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:58.116829  5476 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:58.117055  4583 raft_consensus.cc:1240] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 13 FOLLOWER]: Rejecting Update request from peer df4e0103d6e942fb850d4f18b1cd2b85 for earlier term 12. Current term is 13. Ops: []
I20260812 06:36:58.117146  5476 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:58.117669  5476 consensus_queue.cc:1059] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 }, Status: INVALID_TERM, Last received: 12.25, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
I20260812 06:36:58.117761  4489 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:36:58.117825  5476 raft_consensus.cc:3055] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 12 LEADER]: Stepping down as leader of term 12
I20260812 06:36:58.117870  5476 raft_consensus.cc:740] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: df4e0103d6e942fb850d4f18b1cd2b85, State: Running, Role: LEADER
I20260812 06:36:58.118002  5476 consensus_queue.cc:260] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 12.25, Last appended by leader: 25, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:58.118094  5437 raft_consensus.cc:2749] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260812 06:36:58.118153  5476 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 12 FOLLOWER]: Advancing to term 13
I20260812 06:36:58.118292  4346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 13 candidate_status { last_received { term: 12 index: 25 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
W20260812 06:36:58.118438  5562 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:58.117802 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:37074 (local address 127.3.193.130:35779)
0812 06:36:58.118089 (+   287us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:58.118093 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:58.118106 (+    13us) server_negotiation.cc:408] Connection header received
0812 06:36:58.118146 (+    40us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:58.118149 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:58.118197 (+    48us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:58.118328 (+   131us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37074: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":47,"threads_started":1}
I20260812 06:36:58.119370  4346 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 13.
I20260812 06:36:58.119570  4583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:58.119788  5560 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:58.119987  5560 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:58.120522  5560 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:36:58.120922  4583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
W20260812 06:36:58.652034  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:58.147319 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.129:40455 (local address 127.0.0.1:52770)
0812 06:36:58.147503 (+   184us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:58.147520 (+    17us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:58.147613 (+    93us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:58.651916 (+504303us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":69}
W20260812 06:36:59.147312  4244 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260812 06:36:59.148077  5560 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:59.148185  5560 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:59.148607  5560 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
W20260812 06:36:59.150051  4032 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.193.130:35779 timed out after 0.093s (SENT) [suppressed 5 similar messages]
I20260812 06:36:59.150722  4110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:59.150782  4104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:59.151039  4118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:59.151085  4114 raft_consensus.cc:1240] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 13 FOLLOWER]: Rejecting Update request from peer df4e0103d6e942fb850d4f18b1cd2b85 for earlier term 12. Current term is 13. Ops: []
I20260812 06:36:59.151294  4126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 14 candidate_status { last_received { term: 12 index: 25 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:59.151348  4133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:36:59.151407  4126 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 13.
W20260812 06:36:59.151715  5542 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:59.150342 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:52770 (local address 127.3.193.129:40455)
0812 06:36:59.151315 (+   973us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:59.151321 (+     6us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:59.151338 (+    17us) server_negotiation.cc:408] Connection header received
0812 06:36:59.151382 (+    44us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:59.151385 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:59.151439 (+    54us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:59.151566 (+   127us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52770: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":895}
I20260812 06:36:59.151957  4244 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df4e0103d6e942fb850d4f18b1cd2b85; no voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2
I20260812 06:36:59.152174  4244 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df4e0103d6e942fb850d4f18b1cd2b85; no voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2
I20260812 06:36:59.152465  4244 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:59.152653  5560 raft_consensus.cc:2749] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 06:36:59.152734  5560 raft_consensus.cc:2749] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 06:36:59.152803  5560 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260812 06:36:59.152835  5560 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:36:59.152869  5560 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 13 FOLLOWER]: Advancing to term 14
I20260812 06:36:59.153939  5560 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:59.154273  5560 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 14 election: Requested vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:36:59.155331  4346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 13 candidate_status { last_received { term: 12 index: 25 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:36:59.155452  4346 raft_consensus.cc:2368] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c8af5f280df94129ba27e02fff9356a3 for earlier term 13. Current term is 14.
I20260812 06:36:59.155556  4133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 14 candidate_status { last_received { term: 12 index: 25 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:36:59.155675  4133 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 13 FOLLOWER]: Advancing to term 14
W20260812 06:36:59.155862  4285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:37082: Illegal state: replica df4e0103d6e942fb850d4f18b1cd2b85 is not leader of this config: current role FOLLOWER
I20260812 06:36:59.156638  4032 leader_election.cc:400] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 13 election: Vote denied by peer df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779) with higher term. Message: Invalid argument: T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c8af5f280df94129ba27e02fff9356a3 for earlier term 13. Current term is 14.
I20260812 06:36:59.156692  4032 leader_election.cc:403] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260812 06:36:59.156690  4346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 13 candidate_status { last_received { term: 12 index: 25 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:36:59.156788  4346 raft_consensus.cc:2368] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c8af5f280df94129ba27e02fff9356a3 for earlier term 13. Current term is 14.
I20260812 06:36:59.156809  4133 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 14.
I20260812 06:36:59.156919  5538 raft_consensus.cc:2749] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779) with higher term. Message: Invalid argument: T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c8af5f280df94129ba27e02fff9356a3 for earlier term 13. Current term is 14.
I20260812 06:36:59.157117  4244 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:36:59.157184  4032 leader_election.cc:400] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 13 pre-election: Vote denied by peer df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779) with higher term. Message: Invalid argument: T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c8af5f280df94129ba27e02fff9356a3 for earlier term 13. Current term is 14.
I20260812 06:36:59.157948  5576 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 14 FOLLOWER]: Leader election won for term 14
I20260812 06:36:59.158079  5576 raft_consensus.cc:697] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 14 LEADER]: Becoming Leader. State: Replica: df4e0103d6e942fb850d4f18b1cd2b85, State: Running, Role: LEADER
I20260812 06:36:59.158255  5576 consensus_queue.cc:237] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 12.25, Last appended by leader: 25, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:59.159789  3905 catalog_manager.cc:5719] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: UNKNOWN } } }
I20260812 06:36:59.182024  4133 raft_consensus.cc:1275] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 14 FOLLOWER]: Refusing update from remote peer df4e0103d6e942fb850d4f18b1cd2b85: Log matching property violated. Preceding OpId in replica: term: 12 index: 25. Preceding OpId from leader: term: 14 index: 26. (index mismatch)
I20260812 06:36:59.182634  5576 consensus_queue.cc:1048] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
W20260812 06:36:59.364360  4245 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503): Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.107s (SENT)
W20260812 06:36:59.369692  4245 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503): Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.107s (SENT)
W20260812 06:36:59.676060  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:59.161937 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.131:38503 (local address 127.0.0.1:58430)
0812 06:36:59.162123 (+   186us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:59.162137 (+    14us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:59.162209 (+    72us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:36:59.675929 (+513720us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.131:38503: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":94}
W20260812 06:36:59.676254  4683 meta_cache.cc:302] tablet 27b60a3088e74447b13844f7b49da4ff: replica d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.131:38503: received 0 of 4 requested bytes
I20260812 06:36:59.710884  5546 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:59.710989  5546 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:59.711458  5546 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:36:59.711560  5546 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:59.711656  5546 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:36:59.712041  5546 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:36:59.712127  5546 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:36:59.712198  5546 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:36:59.712531  5546 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:36:59.713186  4583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 14 candidate_status { last_received { term: 12 index: 25 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:59.713241  4581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 14 candidate_status { last_received { term: 12 index: 25 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:59.713297  4583 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 13.
I20260812 06:36:59.713474  4578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 14 candidate_status { last_received { term: 12 index: 25 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:36:59.713549  4578 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 13.
I20260812 06:36:59.713690  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 14 candidate_status { last_received { term: 12 index: 25 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:36:59.713757  4597 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 13 FOLLOWER]: Advancing to term 14
W20260812 06:36:59.714092  5608 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:59.713366 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:58430 (local address 127.3.193.131:38503)
0812 06:36:59.713770 (+   404us) server_negotiation.cc:207] Beginning negotiation
0812 06:36:59.713776 (+     6us) server_negotiation.cc:400] Waiting for connection header
0812 06:36:59.713793 (+    17us) server_negotiation.cc:408] Connection header received
0812 06:36:59.713840 (+    47us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:36:59.713843 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:36:59.713896 (+    53us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:36:59.713998 (+   102us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58430: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":78,"threads_started":1}
I20260812 06:36:59.714844  4597 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 14.
I20260812 06:36:59.715042  4595 raft_consensus.cc:1275] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 14 FOLLOWER]: Refusing update from remote peer df4e0103d6e942fb850d4f18b1cd2b85: Log matching property violated. Preceding OpId in replica: term: 12 index: 25. Preceding OpId from leader: term: 14 index: 26. (index mismatch)
W20260812 06:37:00.252067  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:36:59.732446 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.129:40455 (local address 127.0.0.1:52802)
0812 06:36:59.732613 (+   167us) negotiation.cc:107] Waiting for socket to connect
0812 06:36:59.732630 (+    17us) client_negotiation.cc:175] Beginning negotiation
0812 06:36:59.732720 (+    90us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:37:00.251936 (+519216us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":65}
I20260812 06:37:00.732941  4346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 27 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:37:00.733109  4342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:37:00.733197  4346 raft_consensus.cc:2410] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8c36d75aa834cca96b042eb3bf8a6d2 for term 8 because replica has last-logged OpId of term: 7 index: 29, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 27.
I20260812 06:37:00.733191  4342 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8c36d75aa834cca96b042eb3bf8a6d2 in term 7.
I20260812 06:37:00.733528  4327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 25 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:37:00.733626  5607 consensus_queue.cc:1048] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
I20260812 06:37:00.733801  5576 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:37:00.733916  5576 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:37:00.734314  5576 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:37:00.733801  5613 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:37:00.734521  5613 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:37:00.734855  5613 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:37:00.736460  4126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 29 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:37:00.736541  4104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 27 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:37:00.736691  4118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 25 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:37:00.736756  4114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:37:00.736887  4110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
W20260812 06:37:00.740245  4285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:37098: Illegal state: replica df4e0103d6e942fb850d4f18b1cd2b85 is not leader of this config: current role FOLLOWER
W20260812 06:37:00.741362  5619 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:00.739190 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:52802 (local address 127.3.193.129:40455)
0812 06:37:00.741036 (+  1846us) server_negotiation.cc:207] Beginning negotiation
0812 06:37:00.741040 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 06:37:00.741054 (+    14us) server_negotiation.cc:408] Connection header received
0812 06:37:00.741094 (+    40us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:37:00.741098 (+     4us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:37:00.741143 (+    45us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:37:00.741261 (+   118us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52802: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":1744,"thread_start_us":70,"threads_started":1}
W20260812 06:37:00.828915  4245 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.093s (SENT) [suppressed 11 similar messages]
W20260812 06:37:00.923662  4245 leader_election.cc:336] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503): Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.093s (SENT)
I20260812 06:37:00.923804  4245 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df4e0103d6e942fb850d4f18b1cd2b85; no voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2
I20260812 06:37:00.924047  5613 raft_consensus.cc:2749] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260812 06:37:00.924093  4245 leader_election.cc:336] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503): Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.094s (SENT)
I20260812 06:37:00.924153  4245 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df4e0103d6e942fb850d4f18b1cd2b85; no voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2
I20260812 06:37:00.924798  5613 raft_consensus.cc:2749] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260812 06:37:01.275957  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:00.744311 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.131:38503 (local address 127.0.0.1:58442)
0812 06:37:00.744463 (+   152us) negotiation.cc:107] Waiting for socket to connect
0812 06:37:00.744478 (+    15us) client_negotiation.cc:175] Beginning negotiation
0812 06:37:00.744556 (+    78us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:37:01.275849 (+531293us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.131:38503: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 06:37:01.276206  4683 meta_cache.cc:302] tablet 27b60a3088e74447b13844f7b49da4ff: replica d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.131:38503: received 0 of 4 requested bytes
I20260812 06:37:01.313267  4581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 29 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:37:01.313567  4583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:37:01.313663  4583 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 7.
I20260812 06:37:01.313825  4588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:37:01.313902  4588 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 7.
W20260812 06:37:01.315279  5659 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:01.314710 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:58442 (local address 127.3.193.131:38503)
0812 06:37:01.314915 (+   205us) server_negotiation.cc:207] Beginning negotiation
0812 06:37:01.314920 (+     5us) server_negotiation.cc:400] Waiting for connection header
0812 06:37:01.314936 (+    16us) server_negotiation.cc:408] Connection header received
0812 06:37:01.314980 (+    44us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:37:01.314983 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:37:01.315034 (+    51us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:37:01.315176 (+   142us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58442: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":52,"threads_started":1}
I20260812 06:37:01.313578  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 29 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:37:01.317495  4342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:37:01.317589  4327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 27 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:37:01.317605  4346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 25 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
W20260812 06:37:01.856025  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:01.344689 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.129:40455 (local address 127.0.0.1:52836)
0812 06:37:01.344891 (+   202us) negotiation.cc:107] Waiting for socket to connect
0812 06:37:01.344906 (+    15us) client_negotiation.cc:175] Beginning negotiation
0812 06:37:01.344985 (+    79us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:37:01.855907 (+510922us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.129:40455: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":78}
W20260812 06:37:02.345233  4490 leader_election.cc:336] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.088s (SENT)
I20260812 06:37:02.345356  4490 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:37:02.345526  5517 raft_consensus.cc:2749] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260812 06:37:02.345593  4490 leader_election.cc:336] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.096s (SENT)
I20260812 06:37:02.345643  4490 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
W20260812 06:37:02.345849  4490 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.140s (SENT)
I20260812 06:37:02.345913  4490 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:37:02.346953  5517 raft_consensus.cc:2749] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 06:37:02.347097  5517 raft_consensus.cc:2749] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260812 06:37:02.348538  5640 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:37:02.348645  5640 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:37:02.349381  4578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:37:02.349498  4578 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 7.
I20260812 06:37:02.349049  5640 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:37:02.349855  4245 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:37:02.350346  5640 raft_consensus.cc:2804] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 06:37:02.350391  5640 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Starting leader election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:37:02.350425  5640 raft_consensus.cc:3060] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Advancing to term 8
I20260812 06:37:02.351506  5640 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:37:02.351882  5640 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 election: Requested vote from peers d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:37:02.352421  4578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:37:02.352519  4578 raft_consensus.cc:3060] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Advancing to term 8
I20260812 06:37:02.353443  4578 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 8.
I20260812 06:37:02.353727  4245 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
W20260812 06:37:02.353758  4285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:37122: Illegal state: replica df4e0103d6e942fb850d4f18b1cd2b85 is not leader of this config: current role FOLLOWER
I20260812 06:37:02.353885  5640 raft_consensus.cc:2804] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 8 FOLLOWER]: Leader election won for term 8
I20260812 06:37:02.353963  5640 raft_consensus.cc:697] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 8 LEADER]: Becoming Leader. State: Replica: df4e0103d6e942fb850d4f18b1cd2b85, State: Running, Role: LEADER
I20260812 06:37:02.348973  5667 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:37:02.354074  5640 consensus_queue.cc:237] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 7.24, Last appended by leader: 24, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:37:02.354139  5667 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:37:02.354452  5667 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:37:02.355159  4578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 31 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:37:02.355258  4578 raft_consensus.cc:2468] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 7.
I20260812 06:37:02.355558  4245 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:37:02.355613  3906 catalog_manager.cc:5719] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 reported cstate change: term changed from 7 to 8, leader changed from c8af5f280df94129ba27e02fff9356a3 (127.3.193.129) to df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130). New cstate: current_term: 8 leader_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: UNKNOWN } } }
I20260812 06:37:02.355793  5670 raft_consensus.cc:2804] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 06:37:02.355849  5670 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Starting leader election (detected failure of leader c8af5f280df94129ba27e02fff9356a3)
I20260812 06:37:02.355886  5670 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 7 FOLLOWER]: Advancing to term 8
I20260812 06:37:02.356820  5670 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:37:02.357549  4578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 31 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:37:02.357649  4578 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 7 FOLLOWER]: Advancing to term 8
I20260812 06:37:02.357965  5670 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 election: Requested vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:37:02.358515  4578 raft_consensus.cc:2468] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 8.
I20260812 06:37:02.358852  4245 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85; no voters: 
I20260812 06:37:02.359055  5667 raft_consensus.cc:2804] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 8 FOLLOWER]: Leader election won for term 8
I20260812 06:37:02.359160  5667 raft_consensus.cc:697] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 8 LEADER]: Becoming Leader. State: Replica: df4e0103d6e942fb850d4f18b1cd2b85, State: Running, Role: LEADER
I20260812 06:37:02.359373  5667 consensus_queue.cc:237] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 7.31, Last appended by leader: 31, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
W20260812 06:37:02.361899  4530 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58458: Illegal state: replica d8c36d75aa834cca96b042eb3bf8a6d2 is not leader of this config: current role FOLLOWER
I20260812 06:37:02.362215  3906 catalog_manager.cc:5719] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 reported cstate change: term changed from 7 to 8, leader changed from c8af5f280df94129ba27e02fff9356a3 (127.3.193.129) to df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130). New cstate: current_term: 8 leader_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: UNKNOWN } } }
I20260812 06:37:02.375734  4578 raft_consensus.cc:1275] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 8 FOLLOWER]: Refusing update from remote peer df4e0103d6e942fb850d4f18b1cd2b85: Log matching property violated. Preceding OpId in replica: term: 7 index: 29. Preceding OpId from leader: term: 8 index: 33. (index mismatch)
I20260812 06:37:02.376139  5667 consensus_queue.cc:1048] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 29, Time since last communication: 0.000s
I20260812 06:37:02.383287  4578 raft_consensus.cc:1275] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 8 FOLLOWER]: Refusing update from remote peer df4e0103d6e942fb850d4f18b1cd2b85: Log matching property violated. Preceding OpId in replica: term: 7 index: 24. Preceding OpId from leader: term: 8 index: 25. (index mismatch)
I20260812 06:37:02.383719  5670 consensus_queue.cc:1048] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
W20260812 06:37:02.940023  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:02.427263 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.130:35779 (local address 127.0.0.1:37170)
0812 06:37:02.427451 (+   188us) negotiation.cc:107] Waiting for socket to connect
0812 06:37:02.427467 (+    16us) client_negotiation.cc:175] Beginning negotiation
0812 06:37:02.427547 (+    80us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:37:02.939916 (+512369us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":69}
W20260812 06:37:02.940307  4680 meta_cache.cc:302] tablet 27b60a3088e74447b13844f7b49da4ff: replica df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
W20260812 06:37:03.429237  5671 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:03.428539 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:37170 (local address 127.3.193.130:35779)
0812 06:37:03.428817 (+   278us) server_negotiation.cc:207] Beginning negotiation
0812 06:37:03.428822 (+     5us) server_negotiation.cc:400] Waiting for connection header
0812 06:37:03.428840 (+    18us) server_negotiation.cc:408] Connection header received
0812 06:37:03.428894 (+    54us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:37:03.428898 (+     4us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:37:03.428967 (+    69us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:37:03.429102 (+   135us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37170: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":192}
I20260812 06:37:03.430887  4118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 25 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:37:03.431077  4118 raft_consensus.cc:2393] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8c36d75aa834cca96b042eb3bf8a6d2 in current term 14: Already voted for candidate df4e0103d6e942fb850d4f18b1cd2b85 in this term.
I20260812 06:37:03.431192  4110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:37:03.431332  4127 raft_consensus.cc:3055] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 7 LEADER]: Stepping down as leader of term 7
I20260812 06:37:03.431377  4127 raft_consensus.cc:740] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c8af5f280df94129ba27e02fff9356a3, State: Running, Role: LEADER
I20260812 06:37:03.431448  4111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 31 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:37:03.431522  4127 consensus_queue.cc:260] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 7.31, Last appended by leader: 31, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:37:03.431604  4126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:37:03.431703  4127 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 7 FOLLOWER]: Advancing to term 8
I20260812 06:37:03.432072  5616 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:37:03.432147  5616 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:37:03.432488  5616 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:37:03.432829  4127 raft_consensus.cc:1275] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Refusing update from remote peer df4e0103d6e942fb850d4f18b1cd2b85: Log matching property violated. Preceding OpId in replica: term: 7 index: 31. Preceding OpId from leader: term: 8 index: 33. (index mismatch)
I20260812 06:37:03.433094  4133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:37:03.433113  4125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 31 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:37:03.434073  4125 raft_consensus.cc:2468] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 8.
I20260812 06:37:03.434303  4112 raft_consensus.cc:3055] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 7 LEADER]: Stepping down as leader of term 7
I20260812 06:37:03.434355  4112 raft_consensus.cc:740] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c8af5f280df94129ba27e02fff9356a3, State: Running, Role: LEADER
I20260812 06:37:03.434482  4112 consensus_queue.cc:260] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 7.24, Last appended by leader: 24, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:37:03.434603  4112 raft_consensus.cc:3060] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 7 FOLLOWER]: Advancing to term 8
I20260812 06:37:03.435490  4112 raft_consensus.cc:1275] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Refusing update from remote peer df4e0103d6e942fb850d4f18b1cd2b85: Log matching property violated. Preceding OpId in replica: term: 7 index: 24. Preceding OpId from leader: term: 8 index: 25. (index mismatch)
I20260812 06:37:03.435979  4112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
I20260812 06:37:03.436192  4133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 31 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:37:03.436290  4133 raft_consensus.cc:2376] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 8. Re-sending same reply.
I20260812 06:37:03.436312  4114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 27 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:37:03.436429  4114 raft_consensus.cc:2393] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8c36d75aa834cca96b042eb3bf8a6d2 in current term 8: Already voted for candidate df4e0103d6e942fb850d4f18b1cd2b85 in this term.
I20260812 06:37:03.436637  5640 consensus_queue.cc:1048] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20260812 06:37:03.436803  5664 consensus_queue.cc:1048] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
I20260812 06:37:03.436904  4112 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 8.
I20260812 06:37:03.437076  4125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:37:03.437173  4125 raft_consensus.cc:2376] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate df4e0103d6e942fb850d4f18b1cd2b85 in term 8. Re-sending same reply.
I20260812 06:37:03.438177  4346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 15 candidate_status { last_received { term: 14 index: 28 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:37:03.438504  4327 raft_consensus.cc:1240] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 8 LEADER]: Rejecting Update request from peer c8af5f280df94129ba27e02fff9356a3 for earlier term 7. Current term is 8. Ops: []
I20260812 06:37:03.438858  4127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" candidate_term: 8 candidate_status { last_received { term: 7 index: 31 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3"
W20260812 06:37:03.439862  5705 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:03.437438 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:52836 (local address 127.3.193.129:40455)
0812 06:37:03.439498 (+  2060us) server_negotiation.cc:207] Beginning negotiation
0812 06:37:03.439502 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 06:37:03.439515 (+    13us) server_negotiation.cc:408] Connection header received
0812 06:37:03.439553 (+    38us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:37:03.439556 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:37:03.439597 (+    41us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:37:03.439751 (+   154us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52836: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":1998,"thread_start_us":55,"threads_started":1}
W20260812 06:37:03.443159  4071 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:52840: Illegal state: replica c8af5f280df94129ba27e02fff9356a3 is not leader of this config: current role FOLLOWER
W20260812 06:37:03.643347  4034 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503): Timed out: RequestConsensusVote RPC to 127.3.193.131:38503 timed out after 0.104s (SENT)
I20260812 06:37:03.643508  4034 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3; no voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:37:03.643760  5701 raft_consensus.cc:2749] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260812 06:37:03.964149  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:03.448354 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.131:38503 (local address 127.0.0.1:58462)
0812 06:37:03.448510 (+   156us) negotiation.cc:107] Waiting for socket to connect
0812 06:37:03.448524 (+    14us) client_negotiation.cc:175] Beginning negotiation
0812 06:37:03.448589 (+    65us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:37:03.964048 (+515459us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.131:38503: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 06:37:03.964407  4683 meta_cache.cc:302] tablet 27b60a3088e74447b13844f7b49da4ff: replica d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.131:38503: received 0 of 4 requested bytes
W20260812 06:37:03.984009  4071 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:52846: Illegal state: replica c8af5f280df94129ba27e02fff9356a3 is not leader of this config: current role FOLLOWER
I20260812 06:37:04.000679  4578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 15 candidate_status { last_received { term: 14 index: 28 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:37:04.000738  5740 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:37:04.000880  4578 raft_consensus.cc:2410] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c8af5f280df94129ba27e02fff9356a3 for term 15 because replica has last-logged OpId of term: 14 index: 30, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 28.
I20260812 06:37:04.000849  5740 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:37:04.001088  4595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 15 candidate_status { last_received { term: 14 index: 28 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:37:04.001206  4595 raft_consensus.cc:2410] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c8af5f280df94129ba27e02fff9356a3 for term 15 because replica has last-logged OpId of term: 14 index: 30, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 28.
I20260812 06:37:04.001380  5740 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:37:04.001461  5740 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:37:04.001533  5740 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:37:04.001849  5740 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455)
I20260812 06:37:04.001879  5517 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:37:04.001943  5517 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:37:04.002184  5517 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455), df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779)
I20260812 06:37:04.003254  4125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 25 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:37:04.003495  4104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 30 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:37:04.004866  4327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 33 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:37:04.005158  4327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 30 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:37:04.005398  4346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 25 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
W20260812 06:37:04.012182  5741 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:04.004665 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:58462 (local address 127.3.193.131:38503)
0812 06:37:04.011871 (+  7206us) server_negotiation.cc:207] Beginning negotiation
0812 06:37:04.011874 (+     3us) server_negotiation.cc:400] Waiting for connection header
0812 06:37:04.011887 (+    13us) server_negotiation.cc:408] Connection header received
0812 06:37:04.011925 (+    38us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:37:04.011927 (+     2us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:37:04.011973 (+    46us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:37:04.012081 (+   108us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58462: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":7125,"thread_start_us":98,"threads_started":1}
W20260812 06:37:04.540040  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:04.035384 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.130:35779 (local address 127.0.0.1:37198)
0812 06:37:04.035569 (+   185us) negotiation.cc:107] Waiting for socket to connect
0812 06:37:04.035584 (+    15us) client_negotiation.cc:175] Beginning negotiation
0812 06:37:04.035723 (+   139us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:37:04.539915 (+504192us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":82}
W20260812 06:37:05.037338  4104 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.3.193.131:43093 (request call id 103) took 1034 ms (client timeout 103 ms). Trace:
W20260812 06:37:05.037339  4112 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.3.193.131:43093 (request call id 101) took 1034 ms (client timeout 110 ms). Trace:
W20260812 06:37:05.037477  4112 rpcz_store.cc:269] 0812 06:37:04.003057 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:05.037283 (+1034226us) service_pool.cc:211] Skipping call since client already timed out
0812 06:37:05.037326 (+    43us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:37:05.038094  5701 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:37:05.038194  5701 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
W20260812 06:37:05.038498  5758 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:05.036572 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:37198 (local address 127.3.193.130:35779)
0812 06:37:05.037688 (+  1116us) server_negotiation.cc:207] Beginning negotiation
0812 06:37:05.037693 (+     5us) server_negotiation.cc:400] Waiting for connection header
0812 06:37:05.037709 (+    16us) server_negotiation.cc:408] Connection header received
0812 06:37:05.037758 (+    49us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:37:05.037761 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:37:05.038211 (+   450us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:37:05.038359 (+   148us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37198: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":77,"threads_started":1}
I20260812 06:37:05.038640  5701 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:37:05.038771  5701 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:37:05.038852  5701 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:37:05.039227  5701 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:37:05.039324  5701 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:37:05.039407  5701 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:37:05.039757  5701 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:37:05.040107  4346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 15 candidate_status { last_received { term: 14 index: 30 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:37:05.041371  4327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 9 candidate_status { last_received { term: 8 index: 27 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:37:05.042016  4342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 9 candidate_status { last_received { term: 8 index: 34 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
W20260812 06:37:05.037454  4104 rpcz_store.cc:269] 0812 06:37:04.003173 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:04.003462 (+   289us) service_pool.cc:224] Handling call
0812 06:37:05.037304 (+1033842us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260812 06:37:05.051975  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:04.546021 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.131:38503 (local address 127.0.0.1:58468)
0812 06:37:04.546216 (+   195us) negotiation.cc:107] Waiting for socket to connect
0812 06:37:04.546235 (+    19us) client_negotiation.cc:175] Beginning negotiation
0812 06:37:04.546336 (+   101us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:37:05.051867 (+505531us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.131:38503: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":67}
W20260812 06:37:05.052156  4683 meta_cache.cc:302] tablet 3cf285c378e04319a5acdcae46870291: replica d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.131:38503: received 0 of 4 requested bytes
W20260812 06:37:05.059378  4071 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:52854: Illegal state: replica c8af5f280df94129ba27e02fff9356a3 is not leader of this config: current role FOLLOWER
W20260812 06:37:05.089435  4490 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.094s (SENT) [suppressed 7 similar messages]
I20260812 06:37:05.090231  4578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 15 candidate_status { last_received { term: 14 index: 30 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:37:05.090395  4578 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 14.
I20260812 06:37:05.090793  4034 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2; no voters: df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:37:05.090967  4588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 9 candidate_status { last_received { term: 8 index: 27 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:37:05.091239  5764 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260812 06:37:05.091223  4583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 9 candidate_status { last_received { term: 8 index: 34 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:37:05.091293  5764 raft_consensus.cc:493] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:37:05.091339  5764 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 14 FOLLOWER]: Advancing to term 15
I20260812 06:37:05.091683  4034 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3; no voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:37:05.091888  4034 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3; no voters: d8c36d75aa834cca96b042eb3bf8a6d2, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:37:05.092062  5701 raft_consensus.cc:2749] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260812 06:37:05.092157  5701 raft_consensus.cc:2749] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260812 06:37:05.096473  5787 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:05.092644 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:58468 (local address 127.3.193.131:38503)
0812 06:37:05.095855 (+  3211us) server_negotiation.cc:207] Beginning negotiation
0812 06:37:05.095859 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 06:37:05.095884 (+    25us) server_negotiation.cc:408] Connection header received
0812 06:37:05.095931 (+    47us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:37:05.095936 (+     5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:37:05.095987 (+    51us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:37:05.096373 (+   386us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58468: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":3133,"thread_start_us":76,"threads_started":1}
W20260812 06:37:05.628074  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:05.121539 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.130:35779 (local address 127.0.0.1:37240)
0812 06:37:05.121719 (+   180us) negotiation.cc:107] Waiting for socket to connect
0812 06:37:05.121734 (+    15us) client_negotiation.cc:175] Beginning negotiation
0812 06:37:05.121813 (+    79us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:37:05.627965 (+506152us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":61}
W20260812 06:37:06.121670  5764 consensus_meta.cc:287] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3: Time spent flushing consensus metadata: real 1.030s	user 0.001s	sys 0.000s
I20260812 06:37:06.121804  5764 raft_consensus.cc:515] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:37:06.122356  5764 leader_election.cc:290] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 15 election: Requested vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:37:06.122665  4104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 25 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:37:06.122859  4104 raft_consensus.cc:2410] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8c36d75aa834cca96b042eb3bf8a6d2 for term 9 because replica has last-logged OpId of term: 8 index: 28, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 25.
I20260812 06:37:06.123114  4127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 33 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:37:06.123250  4127 raft_consensus.cc:2410] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8c36d75aa834cca96b042eb3bf8a6d2 for term 9 because replica has last-logged OpId of term: 8 index: 34, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 33.
I20260812 06:37:06.123440  4125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 30 } } ignore_live_leader: false dest_uuid: "c8af5f280df94129ba27e02fff9356a3" is_pre_election: true
I20260812 06:37:06.123540  4125 raft_consensus.cc:2393] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8c36d75aa834cca96b042eb3bf8a6d2 in current term 15: Already voted for candidate c8af5f280df94129ba27e02fff9356a3 in this term.
W20260812 06:37:06.125468  4490 leader_election.cc:336] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.094s (SENT)
I20260812 06:37:06.125553  4490 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
W20260812 06:37:06.125813  4490 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.103s (SENT)
I20260812 06:37:06.125886  4490 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
W20260812 06:37:06.126111  4490 leader_election.cc:336] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer c8af5f280df94129ba27e02fff9356a3 (127.3.193.129:40455): Timed out: RequestConsensusVote RPC to 127.3.193.129:40455 timed out after 0.110s (SENT)
I20260812 06:37:06.126163  4490 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8c36d75aa834cca96b042eb3bf8a6d2; no voters: c8af5f280df94129ba27e02fff9356a3, df4e0103d6e942fb850d4f18b1cd2b85
I20260812 06:37:06.126606  4583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 15 candidate_status { last_received { term: 14 index: 32 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:37:06.126713  4583 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 14 FOLLOWER]: Advancing to term 15
I20260812 06:37:06.127907  4583 raft_consensus.cc:2468] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 15.
I20260812 06:37:06.128882  5803 raft_consensus.cc:2749] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260812 06:37:06.129011  5803 raft_consensus.cc:2749] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260812 06:37:06.129109  5803 raft_consensus.cc:2749] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260812 06:37:06.129455  4071 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42440: Illegal state: replica c8af5f280df94129ba27e02fff9356a3 is not leader of this config: current role FOLLOWER
I20260812 06:37:06.129494  4034 leader_election.cc:304] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2; no voters: 
I20260812 06:37:06.129655  5764 raft_consensus.cc:2804] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 15 FOLLOWER]: Leader election won for term 15
I20260812 06:37:06.129741  5764 raft_consensus.cc:697] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 15 LEADER]: Becoming Leader. State: Replica: c8af5f280df94129ba27e02fff9356a3, State: Running, Role: LEADER
I20260812 06:37:06.129868  5764 consensus_queue.cc:237] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 14.32, Last appended by leader: 32, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:37:06.131490  3905 catalog_manager.cc:5719] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 reported cstate change: term changed from 14 to 15, leader changed from df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130) to c8af5f280df94129ba27e02fff9356a3 (127.3.193.129). New cstate: current_term: 15 leader_uuid: "c8af5f280df94129ba27e02fff9356a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: UNKNOWN } } }
W20260812 06:37:06.138885  4530 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50890: Illegal state: replica d8c36d75aa834cca96b042eb3bf8a6d2 is not leader of this config: current role FOLLOWER
I20260812 06:37:06.157549  4583 raft_consensus.cc:1275] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 15 FOLLOWER]: Refusing update from remote peer c8af5f280df94129ba27e02fff9356a3: Log matching property violated. Preceding OpId in replica: term: 14 index: 32. Preceding OpId from leader: term: 15 index: 33. (index mismatch)
I20260812 06:37:06.158092  5763 consensus_queue.cc:1048] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20260812 06:37:06.178601  5763 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:37:06.178758  5763 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:37:06.179322  5763 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:37:06.179844  4583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 9 candidate_status { last_received { term: 8 index: 28 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:37:06.180025  4583 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 8.
I20260812 06:37:06.180372  4034 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2; no voters: 
I20260812 06:37:06.180552  5763 raft_consensus.cc:2804] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260812 06:37:06.180639  5763 raft_consensus.cc:493] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:37:06.180688  5763 raft_consensus.cc:3060] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Advancing to term 9
I20260812 06:37:06.181914  5763 raft_consensus.cc:515] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:37:06.182300  5763 leader_election.cc:290] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 9 election: Requested vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:37:06.182650  4583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 9 candidate_status { last_received { term: 8 index: 28 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:37:06.182745  4583 raft_consensus.cc:3060] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 8 FOLLOWER]: Advancing to term 9
I20260812 06:37:06.183490  4583 raft_consensus.cc:2468] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 9.
I20260812 06:37:06.183758  4034 leader_election.cc:304] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2; no voters: 
I20260812 06:37:06.183916  5763 raft_consensus.cc:2804] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 9 FOLLOWER]: Leader election won for term 9
I20260812 06:37:06.184026  5763 raft_consensus.cc:697] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 9 LEADER]: Becoming Leader. State: Replica: c8af5f280df94129ba27e02fff9356a3, State: Running, Role: LEADER
I20260812 06:37:06.184168  5763 consensus_queue.cc:237] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 8.28, Last appended by leader: 28, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:37:06.185900  3906 catalog_manager.cc:5719] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 reported cstate change: term changed from 8 to 9, leader changed from df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130) to c8af5f280df94129ba27e02fff9356a3 (127.3.193.129). New cstate: current_term: 9 leader_uuid: "c8af5f280df94129ba27e02fff9356a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: HEALTHY } } }
I20260812 06:37:06.198995  5763 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:37:06.199110  5763 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:37:06.199537  5763 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:37:06.199920  4583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 9 candidate_status { last_received { term: 8 index: 34 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" is_pre_election: true
I20260812 06:37:06.200035  4583 raft_consensus.cc:2468] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 8.
I20260812 06:37:06.200379  4034 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2; no voters: 
I20260812 06:37:06.200594  5763 raft_consensus.cc:2804] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260812 06:37:06.200676  5763 raft_consensus.cc:493] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader df4e0103d6e942fb850d4f18b1cd2b85)
I20260812 06:37:06.200732  5763 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 8 FOLLOWER]: Advancing to term 9
I20260812 06:37:06.201579  5763 raft_consensus.cc:515] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:37:06.201951  5763 leader_election.cc:290] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 9 election: Requested vote from peers df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779), d8c36d75aa834cca96b042eb3bf8a6d2 (127.3.193.131:38503)
I20260812 06:37:06.202293  4583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 9 candidate_status { last_received { term: 8 index: 34 } } ignore_live_leader: false dest_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2"
I20260812 06:37:06.202423  4583 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 8 FOLLOWER]: Advancing to term 9
I20260812 06:37:06.203284  4583 raft_consensus.cc:2468] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 9.
I20260812 06:37:06.203627  4034 leader_election.cc:304] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8af5f280df94129ba27e02fff9356a3, d8c36d75aa834cca96b042eb3bf8a6d2; no voters: 
I20260812 06:37:06.203845  5763 raft_consensus.cc:2804] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 9 FOLLOWER]: Leader election won for term 9
I20260812 06:37:06.203969  5763 raft_consensus.cc:697] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [term 9 LEADER]: Becoming Leader. State: Replica: c8af5f280df94129ba27e02fff9356a3, State: Running, Role: LEADER
I20260812 06:37:06.204114  5763 consensus_queue.cc:237] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 8.34, Last appended by leader: 34, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:37:06.204941  4588 raft_consensus.cc:1275] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 9 FOLLOWER]: Refusing update from remote peer c8af5f280df94129ba27e02fff9356a3: Log matching property violated. Preceding OpId in replica: term: 8 index: 28. Preceding OpId from leader: term: 9 index: 29. (index mismatch)
I20260812 06:37:06.205701  5764 consensus_queue.cc:1048] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20260812 06:37:06.205840  3906 catalog_manager.cc:5719] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 reported cstate change: term changed from 8 to 9, leader changed from df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130) to c8af5f280df94129ba27e02fff9356a3 (127.3.193.129). New cstate: current_term: 9 leader_uuid: "c8af5f280df94129ba27e02fff9356a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } health_report { overall_health: UNKNOWN } } }
W20260812 06:37:06.217357  4032 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.193.130:35779 timed out after 0.094s (SENT) [suppressed 2 similar messages]
I20260812 06:37:06.230651  4588 raft_consensus.cc:1275] T 27b60a3088e74447b13844f7b49da4ff P d8c36d75aa834cca96b042eb3bf8a6d2 [term 9 FOLLOWER]: Refusing update from remote peer c8af5f280df94129ba27e02fff9356a3: Log matching property violated. Preceding OpId in replica: term: 8 index: 34. Preceding OpId from leader: term: 9 index: 35. (index mismatch)
I20260812 06:37:06.231079  5820 consensus_queue.cc:1048] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
W20260812 06:37:06.312021  4032 leader_election.cc:336] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779): Timed out: RequestConsensusVote RPC to 127.3.193.130:35779 timed out after 0.094s (SENT)
W20260812 06:37:06.361536  4032 leader_election.cc:336] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779): Timed out: RequestConsensusVote RPC to 127.3.193.130:35779 timed out after 0.089s (SENT)
W20260812 06:37:06.425196  4032 leader_election.cc:336] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779): Timed out: RequestConsensusVote RPC to 127.3.193.130:35779 timed out after 0.112s (SENT)
W20260812 06:37:06.425478  4032 leader_election.cc:336] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779): Timed out: RequestConsensusVote RPC to 127.3.193.130:35779 timed out after 0.122s (SENT)
W20260812 06:37:06.427749  4032 leader_election.cc:336] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779): Timed out: RequestConsensusVote RPC to 127.3.193.130:35779 timed out after 0.112s (SENT)
W20260812 06:37:06.656057  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:06.148149 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.130:35779 (local address 127.0.0.1:46680)
0812 06:37:06.148299 (+   150us) negotiation.cc:107] Waiting for socket to connect
0812 06:37:06.148314 (+    15us) client_negotiation.cc:175] Beginning negotiation
0812 06:37:06.148383 (+    69us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:37:06.655926 (+507543us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 06:37:06.656225  4680 meta_cache.cc:302] tablet 27b60a3088e74447b13844f7b49da4ff: replica df4e0103d6e942fb850d4f18b1cd2b85 (127.3.193.130:35779) has failed: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
W20260812 06:37:07.195966  4687 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:06.677665 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.130:35779 (local address 127.0.0.1:46688)
0812 06:37:06.677799 (+   134us) negotiation.cc:107] Waiting for socket to connect
0812 06:37:06.677813 (+    14us) client_negotiation.cc:175] Beginning negotiation
0812 06:37:06.677877 (+    64us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:37:07.195852 (+517975us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.130:35779: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 06:37:07.203117  4530 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50900: Illegal state: replica d8c36d75aa834cca96b042eb3bf8a6d2 is not leader of this config: current role FOLLOWER
I20260812 06:37:07.216666  4342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 15 candidate_status { last_received { term: 14 index: 32 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:37:07.216774  4354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 9 candidate_status { last_received { term: 8 index: 34 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:37:07.217028  4329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30fb1aacee684c4eacf9e756aa9927fc" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 15 candidate_status { last_received { term: 14 index: 32 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:37:07.217154  4361 raft_consensus.cc:3055] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 8 LEADER]: Stepping down as leader of term 8
I20260812 06:37:07.217204  4361 raft_consensus.cc:740] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: df4e0103d6e942fb850d4f18b1cd2b85, State: Running, Role: LEADER
I20260812 06:37:07.217294  4360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 9 candidate_status { last_received { term: 8 index: 34 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:37:07.217321  4327 raft_consensus.cc:3055] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 14 LEADER]: Stepping down as leader of term 14
I20260812 06:37:07.217368  4361 consensus_queue.cc:260] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 8.34, Last appended by leader: 34, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:37:07.217507  4346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 9 candidate_status { last_received { term: 8 index: 28 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:37:07.217543  4361 raft_consensus.cc:3060] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 8 FOLLOWER]: Advancing to term 9
I20260812 06:37:07.218091  4354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 9 candidate_status { last_received { term: 8 index: 34 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
I20260812 06:37:07.218688  4315 raft_consensus.cc:3055] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 8 LEADER]: Stepping down as leader of term 8
I20260812 06:37:07.218755  4315 raft_consensus.cc:740] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: df4e0103d6e942fb850d4f18b1cd2b85, State: Running, Role: LEADER
I20260812 06:37:07.218812  4361 raft_consensus.cc:1275] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 9 FOLLOWER]: Refusing update from remote peer c8af5f280df94129ba27e02fff9356a3: Log matching property violated. Preceding OpId in replica: term: 8 index: 34. Preceding OpId from leader: term: 9 index: 35. (index mismatch)
W20260812 06:37:07.218855  5758 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:07.218070 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:37240 (local address 127.3.193.130:35779)
0812 06:37:07.218469 (+   399us) server_negotiation.cc:207] Beginning negotiation
0812 06:37:07.218476 (+     7us) server_negotiation.cc:400] Waiting for connection header
0812 06:37:07.218494 (+    18us) server_negotiation.cc:408] Connection header received
0812 06:37:07.218547 (+    53us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:37:07.218553 (+     6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:37:07.218625 (+    72us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:37:07.218753 (+   128us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37240: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":291}
I20260812 06:37:07.218892  4315 consensus_queue.cc:260] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 8.28, Last appended by leader: 28, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } } peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } }
I20260812 06:37:07.219012  4315 raft_consensus.cc:3060] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 8 FOLLOWER]: Advancing to term 9
I20260812 06:37:07.217404  4327 raft_consensus.cc:740] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: df4e0103d6e942fb850d4f18b1cd2b85, State: Running, Role: LEADER
I20260812 06:37:07.219188  4327 consensus_queue.cc:260] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 14.32, Last appended by leader: 32, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8af5f280df94129ba27e02fff9356a3" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 40455 } } peers { permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 } } peers { permanent_uuid: "d8c36d75aa834cca96b042eb3bf8a6d2" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 38503 } }
I20260812 06:37:07.219327  4327 raft_consensus.cc:3060] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 14 FOLLOWER]: Advancing to term 15
I20260812 06:37:07.219333  4125 raft_consensus.cc:1240] T 3cf285c378e04319a5acdcae46870291 P c8af5f280df94129ba27e02fff9356a3 [term 9 LEADER]: Rejecting Update request from peer df4e0103d6e942fb850d4f18b1cd2b85 for earlier term 8. Current term is 9. Ops: []
I20260812 06:37:07.219663  4588 raft_consensus.cc:1240] T 3cf285c378e04319a5acdcae46870291 P d8c36d75aa834cca96b042eb3bf8a6d2 [term 9 FOLLOWER]: Rejecting Update request from peer df4e0103d6e942fb850d4f18b1cd2b85 for earlier term 8. Current term is 9. Ops: []
I20260812 06:37:07.219789  4331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 9 candidate_status { last_received { term: 8 index: 28 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:37:07.219988  4112 raft_consensus.cc:1240] T 30fb1aacee684c4eacf9e756aa9927fc P c8af5f280df94129ba27e02fff9356a3 [term 15 LEADER]: Rejecting Update request from peer df4e0103d6e942fb850d4f18b1cd2b85 for earlier term 14. Current term is 15. Ops: []
I20260812 06:37:07.220057  4335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 9 candidate_status { last_received { term: 8 index: 28 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:37:07.220266  4353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b60a3088e74447b13844f7b49da4ff" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 9 candidate_status { last_received { term: 8 index: 34 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85"
I20260812 06:37:07.220469  4583 raft_consensus.cc:1240] T 30fb1aacee684c4eacf9e756aa9927fc P d8c36d75aa834cca96b042eb3bf8a6d2 [term 15 FOLLOWER]: Rejecting Update request from peer df4e0103d6e942fb850d4f18b1cd2b85 for earlier term 14. Current term is 15. Ops: []
I20260812 06:37:07.220409  4327 raft_consensus.cc:1275] T 30fb1aacee684c4eacf9e756aa9927fc P df4e0103d6e942fb850d4f18b1cd2b85 [term 15 FOLLOWER]: Refusing update from remote peer c8af5f280df94129ba27e02fff9356a3: Log matching property violated. Preceding OpId in replica: term: 14 index: 32. Preceding OpId from leader: term: 15 index: 33. (index mismatch)
I20260812 06:37:07.221247  4353 raft_consensus.cc:2468] T 27b60a3088e74447b13844f7b49da4ff P df4e0103d6e942fb850d4f18b1cd2b85 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8af5f280df94129ba27e02fff9356a3 in term 9.
I20260812 06:37:07.221211  5763 consensus_queue.cc:1048] T 27b60a3088e74447b13844f7b49da4ff P c8af5f280df94129ba27e02fff9356a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
W20260812 06:37:07.221309  5850 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:07.218389 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:46688 (local address 127.3.193.130:35779)
0812 06:37:07.220938 (+  2549us) server_negotiation.cc:207] Beginning negotiation
0812 06:37:07.220942 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 06:37:07.220957 (+    15us) server_negotiation.cc:408] Connection header received
0812 06:37:07.221000 (+    43us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:37:07.221005 (+     5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:37:07.221048 (+    43us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:37:07.221177 (+   129us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46688: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":2466,"thread_start_us":98,"threads_started":1}
I20260812 06:37:07.221657  3846 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu with pid 4006
I20260812 06:37:07.228598  4342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf285c378e04319a5acdcae46870291" candidate_uuid: "c8af5f280df94129ba27e02fff9356a3" candidate_term: 9 candidate_status { last_received { term: 8 index: 28 } } ignore_live_leader: false dest_uuid: "df4e0103d6e942fb850d4f18b1cd2b85" is_pre_election: true
W20260812 06:37:07.220418  5758 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:07.218113 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:46680 (local address 127.3.193.130:35779)
0812 06:37:07.218988 (+   875us) server_negotiation.cc:207] Beginning negotiation
0812 06:37:07.218991 (+     3us) server_negotiation.cc:400] Waiting for connection header
0812 06:37:07.219005 (+    14us) server_negotiation.cc:408] Connection header received
0812 06:37:07.219046 (+    41us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:37:07.219049 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:37:07.219102 (+    53us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:37:07.219208 (+   106us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46680: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":815,"thread_start_us":107,"threads_started":1}
I20260812 06:37:07.231230  4315 raft_consensus.cc:1275] T 3cf285c378e04319a5acdcae46870291 P df4e0103d6e942fb850d4f18b1cd2b85 [term 9 FOLLOWER]: Refusing update from remote peer c8af5f280df94129ba27e02fff9356a3: Log matching property violated. Preceding OpId in replica: term: 8 index: 28. Preceding OpId from leader: term: 9 index: 29. (index mismatch)
W20260812 06:37:07.239352  4246 connection.cc:573] server connection from 127.3.193.129:58353 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 06:37:07.239372  4490 connection.cc:573] server connection from 127.3.193.129:53925 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 06:37:07.239836  3883 connection.cc:573] server connection from 127.3.193.129:60325 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 06:37:07.240406  3846 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu with pid 4219
I20260812 06:37:07.255901  3846 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu with pid 4461
I20260812 06:37:07.268884  3846 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu with pid 3859
2026-08-12T06:37:07Z chronyd exiting
[       OK ] ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer (35032 ms)
[ RUN      ] ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause
2026-08-12T06:37:07Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:37:07Z Disabled control of system clock
I20260812 06:37:07.312326  3846 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
/tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.193.190:45671
--webserver_interface=127.3.193.190
--webserver_port=0
--builtin_ntp_servers=127.3.193.148:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.193.190:45671
--raft_heartbeat_interval_ms=25
--leader_failure_exp_backoff_max_delta_ms=1000 with env {}
W20260812 06:37:07.417090  5861 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:07.417303  5861 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:07.417339  5861 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:07.420646  5861 flags.cc:432] Enabled experimental flag: --leader_failure_exp_backoff_max_delta_ms=1000
W20260812 06:37:07.420714  5861 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:37:07.420735  5861 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:07.420753  5861 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:37:07.420770  5861 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:37:07.424520  5861 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.193.148:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_exp_backoff_max_delta_ms=1000
--raft_heartbeat_interval_ms=25
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.193.190:45671
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.193.190:45671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.193.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.5861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:07.425596  5861 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:07.426703  5861 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:07.432340  5871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:07.432328  5868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:07.432469  5861 server_base.cc:1061] running on GCE node
W20260812 06:37:07.432328  5869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:07.433156  5861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:07.434144  5861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:07.435339  5861 hybrid_clock.cc:648] HybridClock initialized: now 1786516627435329 us; error 31 us; skew 500 ppm
I20260812 06:37:07.437219  5861 webserver.cc:533] Webserver started at http://127.3.193.190:42105/ using document root <none> and password file <none>
I20260812 06:37:07.437810  5861 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:07.437901  5861 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:07.438108  5861 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:07.439774  5861 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/master-0/data/instance:
uuid: "d4cd6db425194865beb21a9c69ab6d3c"
format_stamp: "Formatted at 2026-08-12 06:37:07 on dist-test-slave-mjjr"
I20260812 06:37:07.440263  5861 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/master-0/wal/instance:
uuid: "d4cd6db425194865beb21a9c69ab6d3c"
format_stamp: "Formatted at 2026-08-12 06:37:07 on dist-test-slave-mjjr"
I20260812 06:37:07.443620  5861 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 06:37:07.445916  5878 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:07.446920  5861 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260812 06:37:07.447072  5861 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/master-0/data,/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/master-0/wal
uuid: "d4cd6db425194865beb21a9c69ab6d3c"
format_stamp: "Formatted at 2026-08-12 06:37:07 on dist-test-slave-mjjr"
I20260812 06:37:07.447180  5861 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:07.466650  5861 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:07.467423  5861 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:07.467626  5861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:07.475661  5970 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.193.190:45671 every 8 connection(s)
I20260812 06:37:07.475682  5861 rpc_server.cc:307] RPC server started. Bound to: 127.3.193.190:45671
I20260812 06:37:07.476758  5861 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/master-0/data/info.pb
I20260812 06:37:07.477627  3846 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu as pid 5861
I20260812 06:37:07.477736  3846 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/master-0/wal/instance
I20260812 06:37:07.480325  5972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:07.485015  5972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4cd6db425194865beb21a9c69ab6d3c: Bootstrap starting.
I20260812 06:37:07.487157  5972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4cd6db425194865beb21a9c69ab6d3c: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:07.487964  5972 log.cc:826] T 00000000000000000000000000000000 P d4cd6db425194865beb21a9c69ab6d3c: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:07.489859  5972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4cd6db425194865beb21a9c69ab6d3c: No bootstrap required, opened a new log
I20260812 06:37:07.493244  5972 raft_consensus.cc:359] T 00000000000000000000000000000000 P d4cd6db425194865beb21a9c69ab6d3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4cd6db425194865beb21a9c69ab6d3c" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 45671 } }
I20260812 06:37:07.493432  5972 raft_consensus.cc:385] T 00000000000000000000000000000000 P d4cd6db425194865beb21a9c69ab6d3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:07.493513  5972 raft_consensus.cc:740] T 00000000000000000000000000000000 P d4cd6db425194865beb21a9c69ab6d3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4cd6db425194865beb21a9c69ab6d3c, State: Initialized, Role: FOLLOWER
I20260812 06:37:07.493911  5972 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4cd6db425194865beb21a9c69ab6d3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4cd6db425194865beb21a9c69ab6d3c" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 45671 } }
I20260812 06:37:07.494056  5972 raft_consensus.cc:399] T 00000000000000000000000000000000 P d4cd6db425194865beb21a9c69ab6d3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:37:07.494124  5972 raft_consensus.cc:493] T 00000000000000000000000000000000 P d4cd6db425194865beb21a9c69ab6d3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:37:07.494244  5972 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d4cd6db425194865beb21a9c69ab6d3c [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:07.495164  5972 raft_consensus.cc:515] T 00000000000000000000000000000000 P d4cd6db425194865beb21a9c69ab6d3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4cd6db425194865beb21a9c69ab6d3c" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 45671 } }
I20260812 06:37:07.495544  5972 leader_election.cc:304] T 00000000000000000000000000000000 P d4cd6db425194865beb21a9c69ab6d3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4cd6db425194865beb21a9c69ab6d3c; no voters: 
I20260812 06:37:07.495930  5972 leader_election.cc:290] T 00000000000000000000000000000000 P d4cd6db425194865beb21a9c69ab6d3c [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 06:37:07.495990  5978 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d4cd6db425194865beb21a9c69ab6d3c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:37:07.496248  5978 raft_consensus.cc:697] T 00000000000000000000000000000000 P d4cd6db425194865beb21a9c69ab6d3c [term 1 LEADER]: Becoming Leader. State: Replica: d4cd6db425194865beb21a9c69ab6d3c, State: Running, Role: LEADER
I20260812 06:37:07.496580  5978 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4cd6db425194865beb21a9c69ab6d3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4cd6db425194865beb21a9c69ab6d3c" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 45671 } }
I20260812 06:37:07.497169  5972 sys_catalog.cc:565] T 00000000000000000000000000000000 P d4cd6db425194865beb21a9c69ab6d3c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:37:07.498276  5984 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4cd6db425194865beb21a9c69ab6d3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4cd6db425194865beb21a9c69ab6d3c. Latest consensus state: current_term: 1 leader_uuid: "d4cd6db425194865beb21a9c69ab6d3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4cd6db425194865beb21a9c69ab6d3c" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 45671 } } }
I20260812 06:37:07.498414  5984 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4cd6db425194865beb21a9c69ab6d3c [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:07.498766  5981 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4cd6db425194865beb21a9c69ab6d3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4cd6db425194865beb21a9c69ab6d3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4cd6db425194865beb21a9c69ab6d3c" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 45671 } } }
I20260812 06:37:07.498852  5981 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4cd6db425194865beb21a9c69ab6d3c [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:07.498958  5997 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:37:07.502362  5997 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:37:07.508487  5997 catalog_manager.cc:1383] Generated new cluster ID: 5190b30a8e204117b61bd61f3e7cfb62
I20260812 06:37:07.508558  5997 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:37:07.534621  5997 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 06:37:07.535869  5997 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:37:07.547539  5997 catalog_manager.cc:6092] T 00000000000000000000000000000000 P d4cd6db425194865beb21a9c69ab6d3c: Generated new TSK 0
I20260812 06:37:07.548420  5997 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:37:07.553242  3846 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
/tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.193.129:0
--local_ip_for_outbound_sockets=127.3.193.129
--webserver_interface=127.3.193.129
--webserver_port=0
--tserver_master_addrs=127.3.193.190:45671
--builtin_ntp_servers=127.3.193.148:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=25
--leader_failure_exp_backoff_max_delta_ms=1000
--heartbeat_interval_ms=25
--enable_txn_system_client_init=true with env {}
W20260812 06:37:07.661273  6013 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:07.661523  6013 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:07.661605  6013 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:07.664973  6013 flags.cc:432] Enabled experimental flag: --leader_failure_exp_backoff_max_delta_ms=1000
W20260812 06:37:07.665081  6013 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:07.665154  6013 flags.cc:432] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:37:07.665251  6013 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.193.129
I20260812 06:37:07.669173  6013 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.193.148:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_exp_backoff_max_delta_ms=1000
--raft_heartbeat_interval_ms=25
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.193.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.193.129
--webserver_port=0
--enable_txn_system_client_init=true
--heartbeat_interval_ms=25
--tserver_master_addrs=127.3.193.190:45671
--never_fsync=true
--heap_profile_path=/tmp/kudu.6013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.193.129

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:07.670302  6013 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:07.671461  6013 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 06:37:07.674280  6013 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 12
W20260812 06:37:07.678071  6026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:07.678083  6021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:07.678083  6023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:07.678344  6013 server_base.cc:1061] running on GCE node
I20260812 06:37:07.678731  6013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:07.679297  6013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:07.680476  6013 hybrid_clock.cc:648] HybridClock initialized: now 1786516627680460 us; error 40 us; skew 500 ppm
I20260812 06:37:07.682291  6013 webserver.cc:533] Webserver started at http://127.3.193.129:37885/ using document root <none> and password file <none>
I20260812 06:37:07.682883  6013 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:07.682981  6013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:07.683209  6013 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:07.684818  6013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-0/data/instance:
uuid: "1f207c465502419f815a70d299cfb76f"
format_stamp: "Formatted at 2026-08-12 06:37:07 on dist-test-slave-mjjr"
I20260812 06:37:07.685319  6013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-0/wal/instance:
uuid: "1f207c465502419f815a70d299cfb76f"
format_stamp: "Formatted at 2026-08-12 06:37:07 on dist-test-slave-mjjr"
I20260812 06:37:07.688928  6013 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 06:37:07.691134  6035 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:07.692274  6013 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:07.692414  6013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-0/wal
uuid: "1f207c465502419f815a70d299cfb76f"
format_stamp: "Formatted at 2026-08-12 06:37:07 on dist-test-slave-mjjr"
I20260812 06:37:07.692524  6013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:07.706650  6013 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:07.707351  6013 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:07.707562  6013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:07.709285  6013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:07.709362  6013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:07.709435  6013 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:07.709479  6013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:07.721916  6013 rpc_server.cc:307] RPC server started. Bound to: 127.3.193.129:38663
I20260812 06:37:07.722234  6233 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.193.129:38663 every 8 connection(s)
I20260812 06:37:07.723102  6013 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-0/data/info.pb
I20260812 06:37:07.724706  6235 heartbeater.cc:344] Connected to a master server at 127.3.193.190:45671
I20260812 06:37:07.724893  6235 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:07.725347  6235 heartbeater.cc:507] Master 127.3.193.190:45671 requested a full tablet report, sending...
I20260812 06:37:07.726923  5905 ts_manager.cc:194] Registered new tserver with Master: 1f207c465502419f815a70d299cfb76f (127.3.193.129:38663)
I20260812 06:37:07.728214  5905 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.3.193.129:56991
I20260812 06:37:07.730346  3846 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu as pid 6013
I20260812 06:37:07.730484  3846 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-0/wal/instance
I20260812 06:37:07.735795  3846 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
/tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.193.130:0
--local_ip_for_outbound_sockets=127.3.193.130
--webserver_interface=127.3.193.130
--webserver_port=0
--tserver_master_addrs=127.3.193.190:45671
--builtin_ntp_servers=127.3.193.148:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=25
--leader_failure_exp_backoff_max_delta_ms=1000
--heartbeat_interval_ms=25
--enable_txn_system_client_init=true with env {}
I20260812 06:37:07.755210  6235 heartbeater.cc:499] Master 127.3.193.190:45671 was elected leader, sending a full tablet report...
W20260812 06:37:07.845960  6237 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:07.846203  6237 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:07.846287  6237 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:07.849615  6237 flags.cc:432] Enabled experimental flag: --leader_failure_exp_backoff_max_delta_ms=1000
W20260812 06:37:07.849718  6237 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:07.849813  6237 flags.cc:432] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:37:07.849910  6237 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.193.130
I20260812 06:37:07.853830  6237 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.193.148:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_exp_backoff_max_delta_ms=1000
--raft_heartbeat_interval_ms=25
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.193.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.193.130
--webserver_port=0
--enable_txn_system_client_init=true
--heartbeat_interval_ms=25
--tserver_master_addrs=127.3.193.190:45671
--never_fsync=true
--heap_profile_path=/tmp/kudu.6237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.193.130

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:07.854940  6237 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:07.856141  6237 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 06:37:07.858760  6237 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 12
W20260812 06:37:07.862658  6247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:07.862694  6249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:07.862905  6237 server_base.cc:1061] running on GCE node
W20260812 06:37:07.863061  6246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:07.863759  6237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:07.864373  6237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:07.865561  6237 hybrid_clock.cc:648] HybridClock initialized: now 1786516627865539 us; error 32 us; skew 500 ppm
I20260812 06:37:07.868229  6237 webserver.cc:533] Webserver started at http://127.3.193.130:38767/ using document root <none> and password file <none>
I20260812 06:37:07.868957  6237 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:07.869037  6237 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:07.869292  6237 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:07.871178  6237 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-1/data/instance:
uuid: "54df2bd09c9c4eadb7fac1b09ab64831"
format_stamp: "Formatted at 2026-08-12 06:37:07 on dist-test-slave-mjjr"
I20260812 06:37:07.871797  6237 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-1/wal/instance:
uuid: "54df2bd09c9c4eadb7fac1b09ab64831"
format_stamp: "Formatted at 2026-08-12 06:37:07 on dist-test-slave-mjjr"
I20260812 06:37:07.875483  6237 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 06:37:07.877667  6258 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:07.878713  6237 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:07.878844  6237 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-1/wal
uuid: "54df2bd09c9c4eadb7fac1b09ab64831"
format_stamp: "Formatted at 2026-08-12 06:37:07 on dist-test-slave-mjjr"
I20260812 06:37:07.878958  6237 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:07.908687  6237 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:07.909466  6237 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:07.909685  6237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:07.911310  6237 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:07.911388  6237 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:07.911450  6237 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:07.911490  6237 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:07.923239  6237 rpc_server.cc:307] RPC server started. Bound to: 127.3.193.130:35957
I20260812 06:37:07.923288  6445 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.193.130:35957 every 8 connection(s)
I20260812 06:37:07.924314  6237 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-1/data/info.pb
I20260812 06:37:07.925745  6446 heartbeater.cc:344] Connected to a master server at 127.3.193.190:45671
I20260812 06:37:07.925927  6446 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:07.926419  6446 heartbeater.cc:507] Master 127.3.193.190:45671 requested a full tablet report, sending...
I20260812 06:37:07.927546  5905 ts_manager.cc:194] Registered new tserver with Master: 54df2bd09c9c4eadb7fac1b09ab64831 (127.3.193.130:35957)
I20260812 06:37:07.928304  5905 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.3.193.130:35675
I20260812 06:37:07.933151  3846 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu as pid 6237
I20260812 06:37:07.933254  3846 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-1/wal/instance
I20260812 06:37:07.938678  3846 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
/tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.193.131:0
--local_ip_for_outbound_sockets=127.3.193.131
--webserver_interface=127.3.193.131
--webserver_port=0
--tserver_master_addrs=127.3.193.190:45671
--builtin_ntp_servers=127.3.193.148:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=25
--leader_failure_exp_backoff_max_delta_ms=1000
--heartbeat_interval_ms=25
--enable_txn_system_client_init=true with env {}
I20260812 06:37:07.955909  6446 heartbeater.cc:499] Master 127.3.193.190:45671 was elected leader, sending a full tablet report...
W20260812 06:37:08.048796  6450 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:08.049027  6450 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:08.049077  6450 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:08.052958  6450 flags.cc:432] Enabled experimental flag: --leader_failure_exp_backoff_max_delta_ms=1000
W20260812 06:37:08.053035  6450 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:08.053079  6450 flags.cc:432] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:37:08.053123  6450 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.193.131
I20260812 06:37:08.057029  6450 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.193.148:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_exp_backoff_max_delta_ms=1000
--raft_heartbeat_interval_ms=25
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.193.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.193.131
--webserver_port=0
--enable_txn_system_client_init=true
--heartbeat_interval_ms=25
--tserver_master_addrs=127.3.193.190:45671
--never_fsync=true
--heap_profile_path=/tmp/kudu.6450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.193.131

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:08.058108  6450 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:08.059199  6450 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 06:37:08.062130  6450 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 12
W20260812 06:37:08.066363  6463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:08.066362  6459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:08.066368  6461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:08.066735  6450 server_base.cc:1061] running on GCE node
I20260812 06:37:08.067145  6450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:08.067814  6450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:08.068984  6450 hybrid_clock.cc:648] HybridClock initialized: now 1786516628068966 us; error 30 us; skew 500 ppm
I20260812 06:37:08.070909  6450 webserver.cc:533] Webserver started at http://127.3.193.131:46343/ using document root <none> and password file <none>
I20260812 06:37:08.071535  6450 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:08.071594  6450 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:08.071868  6450 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:08.073486  6450 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-2/data/instance:
uuid: "c8e8979bfd344b4392136c4b84aa2f85"
format_stamp: "Formatted at 2026-08-12 06:37:08 on dist-test-slave-mjjr"
I20260812 06:37:08.073982  6450 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-2/wal/instance:
uuid: "c8e8979bfd344b4392136c4b84aa2f85"
format_stamp: "Formatted at 2026-08-12 06:37:08 on dist-test-slave-mjjr"
I20260812 06:37:08.077894  6450 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.002s
I20260812 06:37:08.080229  6471 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:08.081418  6450 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 06:37:08.081570  6450 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-2/wal
uuid: "c8e8979bfd344b4392136c4b84aa2f85"
format_stamp: "Formatted at 2026-08-12 06:37:08 on dist-test-slave-mjjr"
I20260812 06:37:08.081683  6450 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:08.105896  6450 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:08.106640  6450 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:08.106848  6450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:08.108625  6450 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:08.108701  6450 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:08.108775  6450 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:08.108816  6450 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:08.120368  6450 rpc_server.cc:307] RPC server started. Bound to: 127.3.193.131:41109
I20260812 06:37:08.120421  6679 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.193.131:41109 every 8 connection(s)
I20260812 06:37:08.121763  6450 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-2/data/info.pb
I20260812 06:37:08.123117  6680 heartbeater.cc:344] Connected to a master server at 127.3.193.190:45671
I20260812 06:37:08.123288  6680 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:08.123852  6680 heartbeater.cc:507] Master 127.3.193.190:45671 requested a full tablet report, sending...
I20260812 06:37:08.124836  5905 ts_manager.cc:194] Registered new tserver with Master: c8e8979bfd344b4392136c4b84aa2f85 (127.3.193.131:41109)
I20260812 06:37:08.125551  5905 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.3.193.131:46781
I20260812 06:37:08.125900  3846 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu as pid 6450
I20260812 06:37:08.126048  3846 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause.1786516592259368-3846-0/minicluster-data/ts-2/wal/instance
I20260812 06:37:08.134640  3846 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 06:37:08.145787  5905 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48846:
name: "kudu_system.kudu_transactions"
schema {
  columns {
    name: "txn_id"
    type: INT64
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "entry_type"
    type: INT8
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "identifier"
    type: STRING
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "metadata"
    type: STRING
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001d\000\000\000\000\000\000\000"
  indirect_data: "<redacted>"""
}
partition_schema {
  range_schema {
    columns {
      name: "txn_id"
    }
  }
}
table_type: TXN_STATUS_TABLE
W20260812 06:37:08.148674  5905 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:37:08.153023  6680 heartbeater.cc:499] Master 127.3.193.190:45671 was elected leader, sending a full tablet report...
I20260812 06:37:08.167174  6338 tablet_service.cc:1511] Processing CreateTablet for tablet 7a2ebb9b7faf41e0ab82650b540ff353 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=55e66e388b4e472e8a308b0b06d32227]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 100
I20260812 06:37:08.168668  6338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a2ebb9b7faf41e0ab82650b540ff353. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:08.169895  6119 tablet_service.cc:1511] Processing CreateTablet for tablet 7a2ebb9b7faf41e0ab82650b540ff353 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=55e66e388b4e472e8a308b0b06d32227]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 100
I20260812 06:37:08.171049  6119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a2ebb9b7faf41e0ab82650b540ff353. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:08.175818  6708 tablet_bootstrap.cc:492] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831: Bootstrap starting.
I20260812 06:37:08.167174  6557 tablet_service.cc:1511] Processing CreateTablet for tablet 7a2ebb9b7faf41e0ab82650b540ff353 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=55e66e388b4e472e8a308b0b06d32227]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 100
I20260812 06:37:08.177305  6557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a2ebb9b7faf41e0ab82650b540ff353. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:08.178366  6708 tablet_bootstrap.cc:654] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:08.179260  6708 log.cc:826] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:08.181582  6712 tablet_bootstrap.cc:492] T 7a2ebb9b7faf41e0ab82650b540ff353 P 1f207c465502419f815a70d299cfb76f: Bootstrap starting.
I20260812 06:37:08.183368  6708 tablet_bootstrap.cc:492] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831: No bootstrap required, opened a new log
I20260812 06:37:08.183620  6708 ts_tablet_manager.cc:1403] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831: Time spent bootstrapping tablet: real 0.008s	user 0.003s	sys 0.003s
I20260812 06:37:08.184441  6712 tablet_bootstrap.cc:654] T 7a2ebb9b7faf41e0ab82650b540ff353 P 1f207c465502419f815a70d299cfb76f: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:08.185963  6712 log.cc:826] T 7a2ebb9b7faf41e0ab82650b540ff353 P 1f207c465502419f815a70d299cfb76f: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:08.186014  6713 tablet_bootstrap.cc:492] T 7a2ebb9b7faf41e0ab82650b540ff353 P c8e8979bfd344b4392136c4b84aa2f85: Bootstrap starting.
I20260812 06:37:08.187403  6708 raft_consensus.cc:359] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35957 } } peers { permanent_uuid: "c8e8979bfd344b4392136c4b84aa2f85" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 41109 } } peers { permanent_uuid: "1f207c465502419f815a70d299cfb76f" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 38663 } }
I20260812 06:37:08.187602  6708 raft_consensus.cc:385] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:08.187696  6708 raft_consensus.cc:740] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54df2bd09c9c4eadb7fac1b09ab64831, State: Initialized, Role: FOLLOWER
I20260812 06:37:08.187867  6712 tablet_bootstrap.cc:492] T 7a2ebb9b7faf41e0ab82650b540ff353 P 1f207c465502419f815a70d299cfb76f: No bootstrap required, opened a new log
I20260812 06:37:08.188063  6712 ts_tablet_manager.cc:1403] T 7a2ebb9b7faf41e0ab82650b540ff353 P 1f207c465502419f815a70d299cfb76f: Time spent bootstrapping tablet: real 0.007s	user 0.005s	sys 0.000s
I20260812 06:37:08.188148  6708 consensus_queue.cc:260] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35957 } } peers { permanent_uuid: "c8e8979bfd344b4392136c4b84aa2f85" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 41109 } } peers { permanent_uuid: "1f207c465502419f815a70d299cfb76f" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 38663 } }
I20260812 06:37:08.188325  6713 tablet_bootstrap.cc:654] T 7a2ebb9b7faf41e0ab82650b540ff353 P c8e8979bfd344b4392136c4b84aa2f85: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:08.189033  6708 ts_tablet_manager.cc:1434] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.005s
I20260812 06:37:08.189090  6713 log.cc:826] T 7a2ebb9b7faf41e0ab82650b540ff353 P c8e8979bfd344b4392136c4b84aa2f85: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:08.190590  6717 tablet_replica.cc:442] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35957 } } peers { permanent_uuid: "c8e8979bfd344b4392136c4b84aa2f85" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 41109 } } peers { permanent_uuid: "1f207c465502419f815a70d299cfb76f" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 38663 } } }
I20260812 06:37:08.190694  6717 tablet_replica.cc:445] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:37:08.190793  6717 raft_consensus.cc:493] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:37:08.190901  6717 raft_consensus.cc:515] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35957 } } peers { permanent_uuid: "c8e8979bfd344b4392136c4b84aa2f85" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 41109 } } peers { permanent_uuid: "1f207c465502419f815a70d299cfb76f" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 38663 } }
I20260812 06:37:08.192039  6717 leader_election.cc:290] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8e8979bfd344b4392136c4b84aa2f85 (127.3.193.131:41109), 1f207c465502419f815a70d299cfb76f (127.3.193.129:38663)
I20260812 06:37:08.192937  6712 raft_consensus.cc:359] T 7a2ebb9b7faf41e0ab82650b540ff353 P 1f207c465502419f815a70d299cfb76f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35957 } } peers { permanent_uuid: "c8e8979bfd344b4392136c4b84aa2f85" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 41109 } } peers { permanent_uuid: "1f207c465502419f815a70d299cfb76f" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 38663 } }
I20260812 06:37:08.193194  6712 raft_consensus.cc:385] T 7a2ebb9b7faf41e0ab82650b540ff353 P 1f207c465502419f815a70d299cfb76f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:08.193303  6712 raft_consensus.cc:740] T 7a2ebb9b7faf41e0ab82650b540ff353 P 1f207c465502419f815a70d299cfb76f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f207c465502419f815a70d299cfb76f, State: Initialized, Role: FOLLOWER
I20260812 06:37:08.193320  6713 tablet_bootstrap.cc:492] T 7a2ebb9b7faf41e0ab82650b540ff353 P c8e8979bfd344b4392136c4b84aa2f85: No bootstrap required, opened a new log
I20260812 06:37:08.193512  6713 ts_tablet_manager.cc:1403] T 7a2ebb9b7faf41e0ab82650b540ff353 P c8e8979bfd344b4392136c4b84aa2f85: Time spent bootstrapping tablet: real 0.008s	user 0.004s	sys 0.000s
I20260812 06:37:08.193821  6712 consensus_queue.cc:260] T 7a2ebb9b7faf41e0ab82650b540ff353 P 1f207c465502419f815a70d299cfb76f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35957 } } peers { permanent_uuid: "c8e8979bfd344b4392136c4b84aa2f85" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 41109 } } peers { permanent_uuid: "1f207c465502419f815a70d299cfb76f" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 38663 } }
I20260812 06:37:08.194695  6712 ts_tablet_manager.cc:1434] T 7a2ebb9b7faf41e0ab82650b540ff353 P 1f207c465502419f815a70d299cfb76f: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260812 06:37:08.197162  6600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a2ebb9b7faf41e0ab82650b540ff353" candidate_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8e8979bfd344b4392136c4b84aa2f85" is_pre_election: true
I20260812 06:37:08.197240  6713 raft_consensus.cc:359] T 7a2ebb9b7faf41e0ab82650b540ff353 P c8e8979bfd344b4392136c4b84aa2f85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35957 } } peers { permanent_uuid: "c8e8979bfd344b4392136c4b84aa2f85" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 41109 } } peers { permanent_uuid: "1f207c465502419f815a70d299cfb76f" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 38663 } }
I20260812 06:37:08.197454  6713 raft_consensus.cc:385] T 7a2ebb9b7faf41e0ab82650b540ff353 P c8e8979bfd344b4392136c4b84aa2f85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:08.197561  6713 raft_consensus.cc:740] T 7a2ebb9b7faf41e0ab82650b540ff353 P c8e8979bfd344b4392136c4b84aa2f85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8e8979bfd344b4392136c4b84aa2f85, State: Initialized, Role: FOLLOWER
I20260812 06:37:08.197980  6713 consensus_queue.cc:260] T 7a2ebb9b7faf41e0ab82650b540ff353 P c8e8979bfd344b4392136c4b84aa2f85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35957 } } peers { permanent_uuid: "c8e8979bfd344b4392136c4b84aa2f85" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 41109 } } peers { permanent_uuid: "1f207c465502419f815a70d299cfb76f" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 38663 } }
I20260812 06:37:08.198237  6600 raft_consensus.cc:2468] T 7a2ebb9b7faf41e0ab82650b540ff353 P c8e8979bfd344b4392136c4b84aa2f85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54df2bd09c9c4eadb7fac1b09ab64831 in term 0.
I20260812 06:37:08.198297  6143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a2ebb9b7faf41e0ab82650b540ff353" candidate_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f207c465502419f815a70d299cfb76f" is_pre_election: true
I20260812 06:37:08.198859  6713 ts_tablet_manager.cc:1434] T 7a2ebb9b7faf41e0ab82650b540ff353 P c8e8979bfd344b4392136c4b84aa2f85: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 06:37:08.198865  6260 leader_election.cc:304] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54df2bd09c9c4eadb7fac1b09ab64831, c8e8979bfd344b4392136c4b84aa2f85; no voters: 
I20260812 06:37:08.199191  6719 raft_consensus.cc:2804] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:37:08.199276  6719 raft_consensus.cc:493] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:37:08.199347  6719 raft_consensus.cc:3060] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:08.199213  6722 tablet_replica.cc:442] T 7a2ebb9b7faf41e0ab82650b540ff353 P c8e8979bfd344b4392136c4b84aa2f85: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35957 } } peers { permanent_uuid: "c8e8979bfd344b4392136c4b84aa2f85" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 41109 } } peers { permanent_uuid: "1f207c465502419f815a70d299cfb76f" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 38663 } } }
I20260812 06:37:08.199504  6722 tablet_replica.cc:445] T 7a2ebb9b7faf41e0ab82650b540ff353 P c8e8979bfd344b4392136c4b84aa2f85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:37:08.199782  6143 raft_consensus.cc:2468] T 7a2ebb9b7faf41e0ab82650b540ff353 P 1f207c465502419f815a70d299cfb76f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54df2bd09c9c4eadb7fac1b09ab64831 in term 0.
I20260812 06:37:08.200544  6719 raft_consensus.cc:515] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35957 } } peers { permanent_uuid: "c8e8979bfd344b4392136c4b84aa2f85" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 41109 } } peers { permanent_uuid: "1f207c465502419f815a70d299cfb76f" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 38663 } }
I20260812 06:37:08.200559  6721 tablet_replica.cc:442] T 7a2ebb9b7faf41e0ab82650b540ff353 P 1f207c465502419f815a70d299cfb76f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35957 } } peers { permanent_uuid: "c8e8979bfd344b4392136c4b84aa2f85" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 41109 } } peers { permanent_uuid: "1f207c465502419f815a70d299cfb76f" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 38663 } } }
I20260812 06:37:08.200990  6721 tablet_replica.cc:445] T 7a2ebb9b7faf41e0ab82650b540ff353 P 1f207c465502419f815a70d299cfb76f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:37:08.201123  6719 leader_election.cc:290] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831 [CANDIDATE]: Term 1 election: Requested vote from peers c8e8979bfd344b4392136c4b84aa2f85 (127.3.193.131:41109), 1f207c465502419f815a70d299cfb76f (127.3.193.129:38663)
I20260812 06:37:08.201373  6600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a2ebb9b7faf41e0ab82650b540ff353" candidate_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8e8979bfd344b4392136c4b84aa2f85"
I20260812 06:37:08.201514  6600 raft_consensus.cc:3060] T 7a2ebb9b7faf41e0ab82650b540ff353 P c8e8979bfd344b4392136c4b84aa2f85 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:08.201798  6143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a2ebb9b7faf41e0ab82650b540ff353" candidate_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f207c465502419f815a70d299cfb76f"
I20260812 06:37:08.201915  6143 raft_consensus.cc:3060] T 7a2ebb9b7faf41e0ab82650b540ff353 P 1f207c465502419f815a70d299cfb76f [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:08.202535  6600 raft_consensus.cc:2468] T 7a2ebb9b7faf41e0ab82650b540ff353 P c8e8979bfd344b4392136c4b84aa2f85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54df2bd09c9c4eadb7fac1b09ab64831 in term 1.
I20260812 06:37:08.202864  6260 leader_election.cc:304] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54df2bd09c9c4eadb7fac1b09ab64831, c8e8979bfd344b4392136c4b84aa2f85; no voters: 
I20260812 06:37:08.203088  6143 raft_consensus.cc:2468] T 7a2ebb9b7faf41e0ab82650b540ff353 P 1f207c465502419f815a70d299cfb76f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54df2bd09c9c4eadb7fac1b09ab64831 in term 1.
I20260812 06:37:08.203486  6719 raft_consensus.cc:2804] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:37:08.203612  6719 raft_consensus.cc:697] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831 [term 1 LEADER]: Becoming Leader. State: Replica: 54df2bd09c9c4eadb7fac1b09ab64831, State: Running, Role: LEADER
I20260812 06:37:08.203899  6719 consensus_queue.cc:237] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35957 } } peers { permanent_uuid: "c8e8979bfd344b4392136c4b84aa2f85" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 41109 } } peers { permanent_uuid: "1f207c465502419f815a70d299cfb76f" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 38663 } }
I20260812 06:37:08.206161  6717 tablet_replica.cc:442] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831: TxnStatusTablet state changed. Reason: New leader 54df2bd09c9c4eadb7fac1b09ab64831. Latest consensus state: current_term: 1 leader_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35957 } } peers { permanent_uuid: "c8e8979bfd344b4392136c4b84aa2f85" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 41109 } } peers { permanent_uuid: "1f207c465502419f815a70d299cfb76f" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 38663 } } }
I20260812 06:37:08.206261  6717 tablet_replica.cc:445] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:37:08.206597  6724 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20260812 06:37:08.206800  6724 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20260812 06:37:08.207044  5905 catalog_manager.cc:5719] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831 reported cstate change: term changed from 0 to 1, leader changed from <none> to 54df2bd09c9c4eadb7fac1b09ab64831 (127.3.193.130). New cstate: current_term: 1 leader_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8e8979bfd344b4392136c4b84aa2f85" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 41109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f207c465502419f815a70d299cfb76f" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 38663 } health_report { overall_health: UNKNOWN } } }
I20260812 06:37:08.226617  6717 consensus_queue.cc:1048] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8e8979bfd344b4392136c4b84aa2f85" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 41109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:08.229485  6722 tablet_replica.cc:442] T 7a2ebb9b7faf41e0ab82650b540ff353 P c8e8979bfd344b4392136c4b84aa2f85: TxnStatusTablet state changed. Reason: New leader 54df2bd09c9c4eadb7fac1b09ab64831. Latest consensus state: current_term: 1 leader_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35957 } } peers { permanent_uuid: "c8e8979bfd344b4392136c4b84aa2f85" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 41109 } } peers { permanent_uuid: "1f207c465502419f815a70d299cfb76f" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 38663 } } }
I20260812 06:37:08.229614  6722 tablet_replica.cc:445] T 7a2ebb9b7faf41e0ab82650b540ff353 P c8e8979bfd344b4392136c4b84aa2f85: This TxnStatusTablet replica's current role is: FOLLOWER
W20260812 06:37:09.252274  6688 tablet.cc:2406] T 7a2ebb9b7faf41e0ab82650b540ff353 P c8e8979bfd344b4392136c4b84aa2f85: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:37:09.253773  6447 tablet.cc:2406] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:37:09.256783  6717 tablet_replica.cc:442] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35957 } } peers { permanent_uuid: "c8e8979bfd344b4392136c4b84aa2f85" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 41109 } } peers { permanent_uuid: "1f207c465502419f815a70d299cfb76f" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 38663 } } }
I20260812 06:37:09.256899  6717 tablet_replica.cc:445] T 7a2ebb9b7faf41e0ab82650b540ff353 P 54df2bd09c9c4eadb7fac1b09ab64831: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:37:09.252303  6722 raft_consensus.cc:493] T 7a2ebb9b7faf41e0ab82650b540ff353 P c8e8979bfd344b4392136c4b84aa2f85 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 54df2bd09c9c4eadb7fac1b09ab64831)
I20260812 06:37:09.259262  6722 raft_consensus.cc:515] T 7a2ebb9b7faf41e0ab82650b540ff353 P c8e8979bfd344b4392136c4b84aa2f85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35957 } } peers { permanent_uuid: "c8e8979bfd344b4392136c4b84aa2f85" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 41109 } } peers { permanent_uuid: "1f207c465502419f815a70d299cfb76f" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 38663 } }
I20260812 06:37:09.260319  6722 leader_election.cc:290] T 7a2ebb9b7faf41e0ab82650b540ff353 P c8e8979bfd344b4392136c4b84aa2f85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 54df2bd09c9c4eadb7fac1b09ab64831 (127.3.193.130:35957), 1f207c465502419f815a70d299cfb76f (127.3.193.129:38663)
I20260812 06:37:09.262923  6722 tablet_replica.cc:442] T 7a2ebb9b7faf41e0ab82650b540ff353 P c8e8979bfd344b4392136c4b84aa2f85: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" member_type: VOTER last_known_addr { host: "127.3.193.130" port: 35957 } } peers { permanent_uuid: "c8e8979bfd344b4392136c4b84aa2f85" member_type: VOTER last_known_addr { host: "127.3.193.131" port: 41109 } } peers { permanent_uuid: "1f207c465502419f815a70d299cfb76f" member_type: VOTER last_known_addr { host: "127.3.193.129" port: 38663 } } }
I20260812 06:37:09.263019  6722 tablet_replica.cc:445] T 7a2ebb9b7faf41e0ab82650b540ff353 P c8e8979bfd344b4392136c4b84aa2f85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:37:09.271301  6363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a2ebb9b7faf41e0ab82650b540ff353" candidate_uuid: "c8e8979bfd344b4392136c4b84aa2f85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "54df2bd09c9c4eadb7fac1b09ab64831" is_pre_election: true
I20260812 06:37:09.331722  3846 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu with pid 6013
W20260812 06:37:09.336374  6474 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: connection negotiation to 127.3.193.129:38663 for RPC RequestConsensusVote timed out after 0.075s (ON_OUTBOUND_QUEUE)
W20260812 06:37:09.342548  6259 connection.cc:573] client connection to 127.3.193.129:38663 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 06:37:09.342700  6259 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 06:37:09.343883  6748 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:09.260528 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.129:38663 (local address 127.3.193.131:38811)
0812 06:37:09.266199 (+  5671us) negotiation.cc:107] Waiting for socket to connect
0812 06:37:09.266215 (+    16us) client_negotiation.cc:175] Beginning negotiation
0812 06:37:09.266305 (+    90us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:37:09.343167 (+ 76862us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.193.129:38663: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":5598,"thread_start_us":74,"threads_started":1}
W20260812 06:37:09.344107  6474 leader_election.cc:336] T 7a2ebb9b7faf41e0ab82650b540ff353 P c8e8979bfd344b4392136c4b84aa2f85 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1f207c465502419f815a70d299cfb76f (127.3.193.129:38663): Network error: Client connection negotiation failed: client connection to 127.3.193.129:38663: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 06:37:09.344214  6474 leader_election.cc:304] T 7a2ebb9b7faf41e0ab82650b540ff353 P c8e8979bfd344b4392136c4b84aa2f85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8e8979bfd344b4392136c4b84aa2f85; no voters: 1f207c465502419f815a70d299cfb76f, 54df2bd09c9c4eadb7fac1b09ab64831
I20260812 06:37:09.344257  3846 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu with pid 6237
I20260812 06:37:09.344774  6722 raft_consensus.cc:2749] T 7a2ebb9b7faf41e0ab82650b540ff353 P c8e8979bfd344b4392136c4b84aa2f85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 06:37:09.356266  3846 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu with pid 6450
I20260812 06:37:09.371969  3846 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu with pid 5861
2026-08-12T06:37:09Z chronyd exiting
[       OK ] ClientFailoverOnNegotiationTimeoutITest.TestTxnSystemClientRetryOnPause (2097 ms)
[ RUN      ] ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster
2026-08-12T06:37:09Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:37:09Z Disabled control of system clock
I20260812 06:37:09.409260  3846 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
/tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.193.190:44409
--webserver_interface=127.3.193.190
--webserver_port=0
--builtin_ntp_servers=127.3.193.148:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.193.190:44409,127.3.193.189:40035,127.3.193.188:36179
--raft_heartbeat_interval_ms=25
--leader_failure_exp_backoff_max_delta_ms=1000 with env {}
W20260812 06:37:09.516701  6771 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:09.516914  6771 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:09.516960  6771 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:09.520121  6771 flags.cc:432] Enabled experimental flag: --leader_failure_exp_backoff_max_delta_ms=1000
W20260812 06:37:09.520181  6771 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:37:09.520202  6771 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:09.520220  6771 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:37:09.520238  6771 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:37:09.524358  6771 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.193.148:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_exp_backoff_max_delta_ms=1000
--raft_heartbeat_interval_ms=25
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.193.190:44409,127.3.193.189:40035,127.3.193.188:36179
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.193.190:44409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.193.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.6771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:09.525761  6771 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:09.527139  6771 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:09.532723  6778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:09.532774  6779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:09.532724  6781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:09.532999  6771 server_base.cc:1061] running on GCE node
I20260812 06:37:09.533463  6771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:09.534370  6771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:09.535564  6771 hybrid_clock.cc:648] HybridClock initialized: now 1786516629535551 us; error 36 us; skew 500 ppm
I20260812 06:37:09.537391  6771 webserver.cc:533] Webserver started at http://127.3.193.190:41213/ using document root <none> and password file <none>
I20260812 06:37:09.537989  6771 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:09.538074  6771 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:09.538295  6771 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:09.539939  6771 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-0/data/instance:
uuid: "525d0091035148378fd1ee151c54db08"
format_stamp: "Formatted at 2026-08-12 06:37:09 on dist-test-slave-mjjr"
I20260812 06:37:09.540439  6771 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-0/wal/instance:
uuid: "525d0091035148378fd1ee151c54db08"
format_stamp: "Formatted at 2026-08-12 06:37:09 on dist-test-slave-mjjr"
I20260812 06:37:09.543794  6771 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 06:37:09.545919  6792 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:09.546962  6771 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:09.547098  6771 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-0/data,/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-0/wal
uuid: "525d0091035148378fd1ee151c54db08"
format_stamp: "Formatted at 2026-08-12 06:37:09 on dist-test-slave-mjjr"
I20260812 06:37:09.547200  6771 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:09.564725  6771 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:09.565469  6771 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:09.565702  6771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:09.573940  6875 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.193.190:44409 every 8 connection(s)
I20260812 06:37:09.573938  6771 rpc_server.cc:307] RPC server started. Bound to: 127.3.193.190:44409
I20260812 06:37:09.575114  6771 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-0/data/info.pb
I20260812 06:37:09.578018  6878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:09.581070  6878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:09.584668  3846 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu as pid 6771
I20260812 06:37:09.584791  3846 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-0/wal/instance
I20260812 06:37:09.585517  3846 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
/tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.193.189:40035
--webserver_interface=127.3.193.189
--webserver_port=0
--builtin_ntp_servers=127.3.193.148:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.193.190:44409,127.3.193.189:40035,127.3.193.188:36179
--raft_heartbeat_interval_ms=25
--leader_failure_exp_backoff_max_delta_ms=1000 with env {}
I20260812 06:37:09.592897  6878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } has no permanent_uuid. Determining permanent_uuid...
W20260812 06:37:09.593816  6795 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.193.189:40035: connect: Connection refused (error 111)
W20260812 06:37:09.594761  6878 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.193.189:40035: Network error: Client connection negotiation failed: client connection to 127.3.193.189:40035: connect: Connection refused (error 111)
I20260812 06:37:09.644070  6878 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } attempt: 1
W20260812 06:37:09.645731  6878 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.193.189:40035: Network error: Client connection negotiation failed: client connection to 127.3.193.189:40035: connect: Connection refused (error 111)
W20260812 06:37:09.691672  6884 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:09.691888  6884 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:09.691928  6884 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:09.695115  6884 flags.cc:432] Enabled experimental flag: --leader_failure_exp_backoff_max_delta_ms=1000
W20260812 06:37:09.695170  6884 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:37:09.695190  6884 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:09.695209  6884 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:37:09.695225  6884 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:37:09.698923  6884 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.193.148:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_exp_backoff_max_delta_ms=1000
--raft_heartbeat_interval_ms=25
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.193.190:44409,127.3.193.189:40035,127.3.193.188:36179
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.193.189:40035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.193.189
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.6884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:09.699921  6884 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:09.700947  6884 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:09.706240  6897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:09.706224  6898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:09.706221  6901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:09.706655  6884 server_base.cc:1061] running on GCE node
I20260812 06:37:09.707155  6884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:09.708091  6884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:09.709250  6884 hybrid_clock.cc:648] HybridClock initialized: now 1786516629709238 us; error 24 us; skew 500 ppm
I20260812 06:37:09.710942  6884 webserver.cc:533] Webserver started at http://127.3.193.189:43027/ using document root <none> and password file <none>
I20260812 06:37:09.711521  6884 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:09.711578  6884 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:09.711807  6884 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:09.713313  6884 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-1/data/instance:
uuid: "3fc150343161429397bb523143a9297a"
format_stamp: "Formatted at 2026-08-12 06:37:09 on dist-test-slave-mjjr"
I20260812 06:37:09.713781  6884 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-1/wal/instance:
uuid: "3fc150343161429397bb523143a9297a"
format_stamp: "Formatted at 2026-08-12 06:37:09 on dist-test-slave-mjjr"
I20260812 06:37:09.713948  6878 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } attempt: 2
W20260812 06:37:09.715391  6878 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.193.189:40035: Network error: Client connection negotiation failed: client connection to 127.3.193.189:40035: connect: Connection refused (error 111)
I20260812 06:37:09.717216  6884 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 06:37:09.719236  6915 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:09.720265  6884 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:09.720388  6884 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-1/data,/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-1/wal
uuid: "3fc150343161429397bb523143a9297a"
format_stamp: "Formatted at 2026-08-12 06:37:09 on dist-test-slave-mjjr"
I20260812 06:37:09.720491  6884 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:09.738008  6884 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:09.738745  6884 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:09.738960  6884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:09.746780  6999 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.193.189:40035 every 8 connection(s)
I20260812 06:37:09.746791  6884 rpc_server.cc:307] RPC server started. Bound to: 127.3.193.189:40035
I20260812 06:37:09.747980  6884 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-1/data/info.pb
I20260812 06:37:09.750800  7000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:09.752667  3846 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu as pid 6884
I20260812 06:37:09.752789  3846 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-1/wal/instance
I20260812 06:37:09.753427  3846 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
/tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.193.188:36179
--webserver_interface=127.3.193.188
--webserver_port=0
--builtin_ntp_servers=127.3.193.148:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.193.190:44409,127.3.193.189:40035,127.3.193.188:36179
--raft_heartbeat_interval_ms=25
--leader_failure_exp_backoff_max_delta_ms=1000 with env {}
I20260812 06:37:09.754073  7000 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:09.763320  7000 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:09.769253  7000 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } has no permanent_uuid. Determining permanent_uuid...
W20260812 06:37:09.770047  6917 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.193.188:36179: connect: Connection refused (error 111)
W20260812 06:37:09.770955  7000 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.193.188:36179: Network error: Client connection negotiation failed: client connection to 127.3.193.188:36179: connect: Connection refused (error 111)
I20260812 06:37:09.806602  6878 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } attempt: 3
I20260812 06:37:09.811385  6878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } has no permanent_uuid. Determining permanent_uuid...
W20260812 06:37:09.812740  6878 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.193.188:36179: Network error: Client connection negotiation failed: client connection to 127.3.193.188:36179: connect: Connection refused (error 111)
I20260812 06:37:09.820191  7000 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } attempt: 1
W20260812 06:37:09.821801  7000 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.193.188:36179: Network error: Client connection negotiation failed: client connection to 127.3.193.188:36179: connect: Connection refused (error 111)
I20260812 06:37:09.843911  6878 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } attempt: 1
W20260812 06:37:09.845454  6878 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.193.188:36179: Network error: Client connection negotiation failed: client connection to 127.3.193.188:36179: connect: Connection refused (error 111)
W20260812 06:37:09.864773  7002 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:09.865167  7002 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:09.865250  7002 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:09.870456  7002 flags.cc:432] Enabled experimental flag: --leader_failure_exp_backoff_max_delta_ms=1000
W20260812 06:37:09.870544  7002 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:37:09.870581  7002 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:09.870612  7002 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:37:09.870654  7002 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:37:09.874627  7002 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.193.148:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_exp_backoff_max_delta_ms=1000
--raft_heartbeat_interval_ms=25
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.193.190:44409,127.3.193.189:40035,127.3.193.188:36179
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.193.188:36179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.193.188
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.7002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:09.875866  7002 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:09.877019  7002 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:09.882622  7018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:09.882624  7015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:09.882642  7016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:09.883078  7002 server_base.cc:1061] running on GCE node
I20260812 06:37:09.883554  7002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:09.884506  7002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:09.885661  7002 hybrid_clock.cc:648] HybridClock initialized: now 1786516629885646 us; error 34 us; skew 500 ppm
I20260812 06:37:09.887408  7002 webserver.cc:533] Webserver started at http://127.3.193.188:39141/ using document root <none> and password file <none>
I20260812 06:37:09.888016  7002 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:09.888074  7002 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:09.888236  7002 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:09.889703  7002 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-2/data/instance:
uuid: "44acc162c38741ae87b154e473ea43e5"
format_stamp: "Formatted at 2026-08-12 06:37:09 on dist-test-slave-mjjr"
I20260812 06:37:09.889990  7000 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } attempt: 2
I20260812 06:37:09.890264  7002 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-2/wal/instance:
uuid: "44acc162c38741ae87b154e473ea43e5"
format_stamp: "Formatted at 2026-08-12 06:37:09 on dist-test-slave-mjjr"
W20260812 06:37:09.891574  7000 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.193.188:36179: Network error: Client connection negotiation failed: client connection to 127.3.193.188:36179: connect: Connection refused (error 111)
I20260812 06:37:09.893838  7002 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 06:37:09.895996  7029 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:09.896978  7002 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:09.897109  7002 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-2/data,/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-2/wal
uuid: "44acc162c38741ae87b154e473ea43e5"
format_stamp: "Formatted at 2026-08-12 06:37:09 on dist-test-slave-mjjr"
I20260812 06:37:09.897205  7002 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:09.916370  7002 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:09.917076  7002 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:09.917284  7002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:09.920639  6878 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } attempt: 2
W20260812 06:37:09.922256  6878 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.193.188:36179: Network error: Client connection negotiation failed: client connection to 127.3.193.188:36179: connect: Connection refused (error 111)
I20260812 06:37:09.925897  7126 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.193.188:36179 every 8 connection(s)
I20260812 06:37:09.925904  7002 rpc_server.cc:307] RPC server started. Bound to: 127.3.193.188:36179
I20260812 06:37:09.927129  7002 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-2/data/info.pb
I20260812 06:37:09.930137  7129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:09.930789  3846 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu as pid 7002
I20260812 06:37:09.930951  3846 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/master-2/wal/instance
I20260812 06:37:09.934558  7129 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:09.943451  7129 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:09.947783  7129 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:09.954880  7129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5: Bootstrap starting.
I20260812 06:37:09.957441  7129 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:09.958173  7129 log.cc:826] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:09.959873  7129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5: No bootstrap required, opened a new log
I20260812 06:37:09.962889  7129 raft_consensus.cc:359] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:09.963104  7129 raft_consensus.cc:385] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:09.963191  7129 raft_consensus.cc:740] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44acc162c38741ae87b154e473ea43e5, State: Initialized, Role: FOLLOWER
I20260812 06:37:09.963712  7129 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:09.964562  7138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:09.964709  7138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:09.965096  7129 sys_catalog.cc:565] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 06:37:09.970491  7156 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:37:09.970572  7156 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:37:09.968989  7002 master_runner.cc:187] Error getting master registration for 127.3.193.190:44409: OK, instance_id { permanent_uuid: "525d0091035148378fd1ee151c54db08" instance_seqno: 1786516629571543 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 06:37:09.971292  7002 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260812 06:37:09.982784  7000 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } attempt: 3
I20260812 06:37:09.985813  7138 raft_consensus.cc:493] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:37:09.986008  7138 raft_consensus.cc:515] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:09.987106  7138 leader_election.cc:290] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 525d0091035148378fd1ee151c54db08 (127.3.193.190:44409), 3fc150343161429397bb523143a9297a (127.3.193.189:40035)
I20260812 06:37:09.990578  7000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a: Bootstrap starting.
I20260812 06:37:09.993393  7000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:09.994283  7000 log.cc:826] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:09.996326  7000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a: No bootstrap required, opened a new log
I20260812 06:37:09.999938  7000 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:10.000146  7000 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:10.000204  7000 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc150343161429397bb523143a9297a, State: Initialized, Role: FOLLOWER
I20260812 06:37:10.000669  7000 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:10.001223  7163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:10.001364  7163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:10.002008  7000 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:37:10.002084  6958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44acc162c38741ae87b154e473ea43e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc150343161429397bb523143a9297a" is_pre_election: true
I20260812 06:37:10.002348  6958 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44acc162c38741ae87b154e473ea43e5 in term 0.
I20260812 06:37:10.002882  7030 leader_election.cc:304] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc150343161429397bb523143a9297a, 44acc162c38741ae87b154e473ea43e5; no voters: 
I20260812 06:37:10.003862  7138 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:37:10.003968  7138 raft_consensus.cc:493] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:37:10.004019  7138 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:10.005443  7138 raft_consensus.cc:515] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:10.005873  7138 leader_election.cc:290] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [CANDIDATE]: Term 1 election: Requested vote from peers 525d0091035148378fd1ee151c54db08 (127.3.193.190:44409), 3fc150343161429397bb523143a9297a (127.3.193.189:40035)
W20260812 06:37:10.008672  7180 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:37:10.008749  7180 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:37:10.007514  6884 master_runner.cc:187] Error getting master registration for 127.3.193.190:44409: OK, instance_id { permanent_uuid: "525d0091035148378fd1ee151c54db08" instance_seqno: 1786516629571543 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 06:37:10.010129  6884 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260812 06:37:10.010663  6958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44acc162c38741ae87b154e473ea43e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc150343161429397bb523143a9297a"
I20260812 06:37:10.010783  6958 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:10.012223  6958 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44acc162c38741ae87b154e473ea43e5 in term 1.
I20260812 06:37:10.012713  7030 leader_election.cc:304] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc150343161429397bb523143a9297a, 44acc162c38741ae87b154e473ea43e5; no voters: 
I20260812 06:37:10.012984  7138 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:37:10.013200  7138 raft_consensus.cc:697] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 1 LEADER]: Becoming Leader. State: Replica: 44acc162c38741ae87b154e473ea43e5, State: Running, Role: LEADER
I20260812 06:37:10.013550  7138 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:10.015998  7138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44acc162c38741ae87b154e473ea43e5. Latest consensus state: current_term: 1 leader_uuid: "44acc162c38741ae87b154e473ea43e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:10.016264  7138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:10.020337  7188 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:37:10.021453  7188 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:37:10.021425  6878 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } attempt: 3
I20260812 06:37:10.027480  6958 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 1 FOLLOWER]: Refusing update from remote peer 44acc162c38741ae87b154e473ea43e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:37:10.028338  7183 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:10.029420  6878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08: Bootstrap starting.
I20260812 06:37:10.033682  7163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44acc162c38741ae87b154e473ea43e5. Latest consensus state: current_term: 1 leader_uuid: "44acc162c38741ae87b154e473ea43e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:10.033806  7163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:10.035223  6878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:10.036181  6878 log.cc:826] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:10.037061  7189 mvcc.cc:204] Tried to move back new op lower bound from 7317572116583870464 to 7317572116541198336. Current Snapshot: MvccSnapshot[applied={T|T < 7317572116583870464}]
I20260812 06:37:10.038017  7138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "44acc162c38741ae87b154e473ea43e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:10.038131  7138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:10.040699  7163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "44acc162c38741ae87b154e473ea43e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:10.040822  7163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:10.046217  7188 catalog_manager.cc:1383] Generated new cluster ID: d91d2b9b4b8346a79f6d8c297e4122e7
I20260812 06:37:10.046288  7188 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:37:10.048305  6878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08: No bootstrap required, opened a new log
I20260812 06:37:10.049368  7190 mvcc.cc:204] Tried to move back new op lower bound from 7317572116583870464 to 7317572116541198336. Current Snapshot: MvccSnapshot[applied={T|T < 7317572116583870464}]
I20260812 06:37:10.052073  6878 raft_consensus.cc:359] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:10.052313  6878 raft_consensus.cc:385] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:10.052431  6878 raft_consensus.cc:740] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 525d0091035148378fd1ee151c54db08, State: Initialized, Role: FOLLOWER
I20260812 06:37:10.052981  6878 consensus_queue.cc:260] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:10.054247  6878 sys_catalog.cc:565] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:37:10.054494  6831 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:10.055953  6831 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 1 FOLLOWER]: Refusing update from remote peer 44acc162c38741ae87b154e473ea43e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:37:10.057096  7199 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:10.058983  6834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44acc162c38741ae87b154e473ea43e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "525d0091035148378fd1ee151c54db08"
I20260812 06:37:10.059297  6837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44acc162c38741ae87b154e473ea43e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "525d0091035148378fd1ee151c54db08" is_pre_election: true
W20260812 06:37:10.059475  6837 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56154 (request call id 2) took 71 ms (client timeout 75 ms). Trace:
W20260812 06:37:10.059543  6837 rpcz_store.cc:269] 0812 06:37:09.987564 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:09.987638 (+    74us) service_pool.cc:224] Handling call
0812 06:37:10.059459 (+ 71821us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260812 06:37:10.067121  6802 tablet.cc:2406] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:37:10.071446  7222 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:37:10.071725  7222 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:37:10.073261  7227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44acc162c38741ae87b154e473ea43e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:10.073447  7227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:10.073618  7227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44acc162c38741ae87b154e473ea43e5. Latest consensus state: current_term: 1 leader_uuid: "44acc162c38741ae87b154e473ea43e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:10.073818  7227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:10.074764  7227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "44acc162c38741ae87b154e473ea43e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:10.074927  7227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:10.076766  7225 mvcc.cc:204] Tried to move back new op lower bound from 7317572116583870464 to 7317572116541198336. Current Snapshot: MvccSnapshot[applied={T|T < 7317572116583870464}]
I20260812 06:37:10.085407  7183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "44acc162c38741ae87b154e473ea43e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:10.085530  7183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:10.098855  7188 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 06:37:10.100124  7188 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:37:10.130350  7188 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5: Generated new TSK 0
I20260812 06:37:10.131247  7188 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:37:10.140895  3846 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
/tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.193.129:0
--local_ip_for_outbound_sockets=127.3.193.129
--webserver_interface=127.3.193.129
--webserver_port=0
--tserver_master_addrs=127.3.193.190:44409,127.3.193.189:40035,127.3.193.188:36179
--builtin_ntp_servers=127.3.193.148:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=25
--leader_failure_exp_backoff_max_delta_ms=1000
--heartbeat_interval_ms=25
--enable_txn_system_client_init=true with env {}
W20260812 06:37:10.290434  7234 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:10.290671  7234 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:10.290766  7234 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:10.294507  7234 flags.cc:432] Enabled experimental flag: --leader_failure_exp_backoff_max_delta_ms=1000
W20260812 06:37:10.294637  7234 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:10.294734  7234 flags.cc:432] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:37:10.294840  7234 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.193.129
I20260812 06:37:10.298806  7234 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.193.148:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_exp_backoff_max_delta_ms=1000
--raft_heartbeat_interval_ms=25
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.193.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.193.129
--webserver_port=0
--enable_txn_system_client_init=true
--heartbeat_interval_ms=25
--tserver_master_addrs=127.3.193.190:44409,127.3.193.189:40035,127.3.193.188:36179
--never_fsync=true
--heap_profile_path=/tmp/kudu.7234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.193.129

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:10.300017  7234 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:10.301146  7234 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 06:37:10.304150  7234 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 12
W20260812 06:37:10.308305  7244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:10.308321  7246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:10.308342  7250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:10.308701  7234 server_base.cc:1061] running on GCE node
I20260812 06:37:10.309135  7234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:10.309702  7234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:10.310868  7234 hybrid_clock.cc:648] HybridClock initialized: now 1786516630310857 us; error 31 us; skew 500 ppm
I20260812 06:37:10.312812  7234 webserver.cc:533] Webserver started at http://127.3.193.129:46021/ using document root <none> and password file <none>
I20260812 06:37:10.313522  7234 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:10.313603  7234 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:10.313822  7234 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:10.315526  7234 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/ts-0/data/instance:
uuid: "d3e477b71c5048d0985e09fc4a9dbe01"
format_stamp: "Formatted at 2026-08-12 06:37:10 on dist-test-slave-mjjr"
I20260812 06:37:10.316087  7234 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/ts-0/wal/instance:
uuid: "d3e477b71c5048d0985e09fc4a9dbe01"
format_stamp: "Formatted at 2026-08-12 06:37:10 on dist-test-slave-mjjr"
I20260812 06:37:10.319747  7234 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 06:37:10.322090  7259 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:10.323416  7234 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:10.323570  7234 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/ts-0/wal
uuid: "d3e477b71c5048d0985e09fc4a9dbe01"
format_stamp: "Formatted at 2026-08-12 06:37:10 on dist-test-slave-mjjr"
I20260812 06:37:10.323740  7234 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:10.335111  7234 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:10.335830  7234 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:10.336024  7234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:10.337989  7234 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:10.338056  7234 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:10.338124  7234 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:10.338163  7234 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:10.350147  7234 rpc_server.cc:307] RPC server started. Bound to: 127.3.193.129:36511
I20260812 06:37:10.350291  7455 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.193.129:36511 every 8 connection(s)
I20260812 06:37:10.351377  7234 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/ts-0/data/info.pb
I20260812 06:37:10.353871  3846 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu as pid 7234
I20260812 06:37:10.354002  3846 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster.1786516592259368-3846-0/minicluster-data/ts-0/wal/instance
I20260812 06:37:10.356446  7463 heartbeater.cc:344] Connected to a master server at 127.3.193.190:44409
I20260812 06:37:10.356673  7463 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:10.358790  7463 heartbeater.cc:507] Master 127.3.193.190:44409 requested a full tablet report, sending...
I20260812 06:37:10.360322  6815 ts_manager.cc:194] Registered new tserver with Master: d3e477b71c5048d0985e09fc4a9dbe01 (127.3.193.129:36511)
I20260812 06:37:10.361902  7464 heartbeater.cc:344] Connected to a master server at 127.3.193.189:40035
I20260812 06:37:10.361990  7464 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:10.362226  7464 heartbeater.cc:507] Master 127.3.193.189:40035 requested a full tablet report, sending...
I20260812 06:37:10.363408  6946 ts_manager.cc:194] Registered new tserver with Master: d3e477b71c5048d0985e09fc4a9dbe01 (127.3.193.129:36511)
I20260812 06:37:10.369037  7461 heartbeater.cc:344] Connected to a master server at 127.3.193.188:36179
I20260812 06:37:10.369160  7461 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:10.369390  7461 heartbeater.cc:507] Master 127.3.193.188:36179 requested a full tablet report, sending...
I20260812 06:37:10.370486  7046 ts_manager.cc:194] Registered new tserver with Master: d3e477b71c5048d0985e09fc4a9dbe01 (127.3.193.129:36511)
I20260812 06:37:10.371958  7046 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.3.193.129:35573
I20260812 06:37:10.383334  3846 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260812 06:37:10.398995  7461 heartbeater.cc:499] Master 127.3.193.188:36179 was elected leader, sending a full tablet report...
I20260812 06:37:10.464767  7202 raft_consensus.cc:493] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 44acc162c38741ae87b154e473ea43e5)
I20260812 06:37:10.465011  7202 raft_consensus.cc:515] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:10.465960  7202 leader_election.cc:290] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fc150343161429397bb523143a9297a (127.3.193.189:40035), 44acc162c38741ae87b154e473ea43e5 (127.3.193.188:36179)
I20260812 06:37:10.466672  6958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "525d0091035148378fd1ee151c54db08" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3fc150343161429397bb523143a9297a" is_pre_election: true
I20260812 06:37:10.469957  7163 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 44acc162c38741ae87b154e473ea43e5)
I20260812 06:37:10.470093  7163 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:10.470877  7163 leader_election.cc:290] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 525d0091035148378fd1ee151c54db08 (127.3.193.190:44409), 44acc162c38741ae87b154e473ea43e5 (127.3.193.188:36179)
I20260812 06:37:10.471319  6831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3fc150343161429397bb523143a9297a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "525d0091035148378fd1ee151c54db08" is_pre_election: true
I20260812 06:37:10.471494  6831 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fc150343161429397bb523143a9297a in term 1.
I20260812 06:37:10.471906  6917 leader_election.cc:304] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc150343161429397bb523143a9297a, 525d0091035148378fd1ee151c54db08; no voters: 
I20260812 06:37:10.472164  7163 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 06:37:10.472240  7163 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 44acc162c38741ae87b154e473ea43e5)
I20260812 06:37:10.472297  7163 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:37:10.473202  7163 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:10.473564  7163 leader_election.cc:290] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [CANDIDATE]: Term 2 election: Requested vote from peers 525d0091035148378fd1ee151c54db08 (127.3.193.190:44409), 44acc162c38741ae87b154e473ea43e5 (127.3.193.188:36179)
I20260812 06:37:10.474080  6831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3fc150343161429397bb523143a9297a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "525d0091035148378fd1ee151c54db08"
I20260812 06:37:10.474210  6831 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:37:10.475065  6831 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc150343161429397bb523143a9297a in term 2.
I20260812 06:37:10.475416  6917 leader_election.cc:304] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc150343161429397bb523143a9297a, 525d0091035148378fd1ee151c54db08; no voters: 
I20260812 06:37:10.475591  7163 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 2 FOLLOWER]: Leader election won for term 2
I20260812 06:37:10.475816  7163 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 2 LEADER]: Becoming Leader. State: Replica: 3fc150343161429397bb523143a9297a, State: Running, Role: LEADER
I20260812 06:37:10.476111  7163 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:10.477128  7479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fc150343161429397bb523143a9297a. Latest consensus state: current_term: 2 leader_uuid: "3fc150343161429397bb523143a9297a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:10.477213  7479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:10.477521  7480 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:37:10.478677  7480 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:37:10.479187  7480 catalog_manager.cc:1295] Loaded cluster ID: d91d2b9b4b8346a79f6d8c297e4122e7
I20260812 06:37:10.479236  7480 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:37:10.480917  7480 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:37:10.481503  7480 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a: Loaded TSK: 0
I20260812 06:37:10.482379  7480 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:37:10.505939  6831 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 2 FOLLOWER]: Refusing update from remote peer 3fc150343161429397bb523143a9297a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 06:37:10.506608  7479 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [LEADER]: Connected to new peer: Peer: permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 06:37:10.508010  7202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fc150343161429397bb523143a9297a. Latest consensus state: current_term: 2 leader_uuid: "3fc150343161429397bb523143a9297a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:10.508153  7202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:10.509019  7163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "3fc150343161429397bb523143a9297a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:10.509145  7163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:10.509902  7202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "3fc150343161429397bb523143a9297a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:10.510051  7202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:10.520771  7464 heartbeater.cc:499] Master 127.3.193.189:40035 was elected leader, sending a full tablet report...
W20260812 06:37:10.653790  6917 leader_election.cc:336] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 44acc162c38741ae87b154e473ea43e5 (127.3.193.188:36179): Timed out: RequestConsensusVote RPC to 127.3.193.188:36179 timed out after 0.089s (SENT)
W20260812 06:37:10.686353  6795 leader_election.cc:336] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 44acc162c38741ae87b154e473ea43e5 (127.3.193.188:36179): Timed out: RequestConsensusVote RPC to 127.3.193.188:36179 timed out after 0.109s (SENT)
I20260812 06:37:10.686491  6795 leader_election.cc:304] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 525d0091035148378fd1ee151c54db08; no voters: 3fc150343161429397bb523143a9297a, 44acc162c38741ae87b154e473ea43e5
I20260812 06:37:10.686745  7202 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 06:37:10.693133  6917 leader_election.cc:336] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 44acc162c38741ae87b154e473ea43e5 (127.3.193.188:36179): Timed out: RequestConsensusVote RPC to 127.3.193.188:36179 timed out after 0.110s (SENT)
W20260812 06:37:10.940075  7475 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:10.418445 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.188:36179 (local address 127.0.0.1:53408)
0812 06:37:10.418592 (+   147us) negotiation.cc:107] Waiting for socket to connect
0812 06:37:10.418607 (+    15us) client_negotiation.cc:175] Beginning negotiation
0812 06:37:10.418674 (+    67us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:37:10.939944 (+521270us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.188:36179: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 06:37:10.940896  7469 master_proxy_rpc.cc:203] Re-attempting ListTables request to leader Master (127.3.193.188:36179)
I20260812 06:37:10.953244  7064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "525d0091035148378fd1ee151c54db08" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "44acc162c38741ae87b154e473ea43e5" is_pre_election: true
I20260812 06:37:10.953701  7069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "525d0091035148378fd1ee151c54db08" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "44acc162c38741ae87b154e473ea43e5" is_pre_election: true
W20260812 06:37:10.953908  7499 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:10.952279 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:53408 (local address 127.3.193.188:36179)
0812 06:37:10.952830 (+   551us) server_negotiation.cc:207] Beginning negotiation
0812 06:37:10.952834 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 06:37:10.952846 (+    12us) server_negotiation.cc:408] Connection header received
0812 06:37:10.952886 (+    40us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:37:10.952889 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:37:10.952936 (+    47us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:37:10.953095 (+   159us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53408: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":73,"threads_started":1}
I20260812 06:37:10.953958  7070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3fc150343161429397bb523143a9297a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "44acc162c38741ae87b154e473ea43e5" is_pre_election: true
I20260812 06:37:10.954176  7073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3fc150343161429397bb523143a9297a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "44acc162c38741ae87b154e473ea43e5"
I20260812 06:37:10.954689  7074 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 06:37:10.954762  7074 raft_consensus.cc:740] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 44acc162c38741ae87b154e473ea43e5, State: Running, Role: LEADER
I20260812 06:37:10.954970  7074 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:10.955247  7074 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:37:10.955741  7076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3fc150343161429397bb523143a9297a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "44acc162c38741ae87b154e473ea43e5"
I20260812 06:37:10.956079  7077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3fc150343161429397bb523143a9297a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "44acc162c38741ae87b154e473ea43e5" is_pre_election: true
I20260812 06:37:10.962580  7074 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 2 FOLLOWER]: Refusing update from remote peer 3fc150343161429397bb523143a9297a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 06:37:10.963165  7163 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [LEADER]: Connected to new peer: Peer: permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 06:37:10.965042  7199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fc150343161429397bb523143a9297a. Latest consensus state: current_term: 2 leader_uuid: "3fc150343161429397bb523143a9297a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:10.965143  7199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:10.965281  7138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "3fc150343161429397bb523143a9297a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:10.965353  7138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:10.971827  7156 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:37:11.037055  7202 raft_consensus.cc:493] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3fc150343161429397bb523143a9297a)
I20260812 06:37:11.037214  7202 raft_consensus.cc:515] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:11.037674  7202 leader_election.cc:290] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3fc150343161429397bb523143a9297a (127.3.193.189:40035), 44acc162c38741ae87b154e473ea43e5 (127.3.193.188:36179)
I20260812 06:37:11.038285  7074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "525d0091035148378fd1ee151c54db08" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "44acc162c38741ae87b154e473ea43e5" is_pre_election: true
I20260812 06:37:11.039870  7138 raft_consensus.cc:493] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3fc150343161429397bb523143a9297a)
I20260812 06:37:11.039974  7138 raft_consensus.cc:515] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:11.040294  7138 leader_election.cc:290] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 525d0091035148378fd1ee151c54db08 (127.3.193.190:44409), 3fc150343161429397bb523143a9297a (127.3.193.189:40035)
I20260812 06:37:11.040728  6831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44acc162c38741ae87b154e473ea43e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "525d0091035148378fd1ee151c54db08" is_pre_election: true
I20260812 06:37:11.040894  6831 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44acc162c38741ae87b154e473ea43e5 in term 2.
I20260812 06:37:11.041208  7030 leader_election.cc:304] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44acc162c38741ae87b154e473ea43e5, 525d0091035148378fd1ee151c54db08; no voters: 
I20260812 06:37:11.041445  7138 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 06:37:11.041510  7138 raft_consensus.cc:493] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3fc150343161429397bb523143a9297a)
I20260812 06:37:11.041559  7138 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:37:11.042575  7138 raft_consensus.cc:515] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:11.042986  7138 leader_election.cc:290] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [CANDIDATE]: Term 3 election: Requested vote from peers 525d0091035148378fd1ee151c54db08 (127.3.193.190:44409), 3fc150343161429397bb523143a9297a (127.3.193.189:40035)
I20260812 06:37:11.043401  6831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44acc162c38741ae87b154e473ea43e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "525d0091035148378fd1ee151c54db08"
I20260812 06:37:11.043545  6831 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:37:11.044405  6831 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44acc162c38741ae87b154e473ea43e5 in term 3.
I20260812 06:37:11.044705  7030 leader_election.cc:304] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44acc162c38741ae87b154e473ea43e5, 525d0091035148378fd1ee151c54db08; no voters: 
I20260812 06:37:11.044879  7138 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 06:37:11.044996  7138 raft_consensus.cc:697] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 3 LEADER]: Becoming Leader. State: Replica: 44acc162c38741ae87b154e473ea43e5, State: Running, Role: LEADER
I20260812 06:37:11.045135  7138 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:11.045683  7199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44acc162c38741ae87b154e473ea43e5. Latest consensus state: current_term: 3 leader_uuid: "44acc162c38741ae87b154e473ea43e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:11.045773  7199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:11.046042  7501 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:37:11.046801  7501 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:37:11.047278  7501 catalog_manager.cc:1295] Loaded cluster ID: d91d2b9b4b8346a79f6d8c297e4122e7
I20260812 06:37:11.047353  7501 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:37:11.048143  7501 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:37:11.048552  7501 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5: Loaded TSK: 0
I20260812 06:37:11.048991  7501 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:37:11.064512  6831 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 3 FOLLOWER]: Refusing update from remote peer 44acc162c38741ae87b154e473ea43e5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 06:37:11.065007  7199 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 06:37:11.066108  7202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44acc162c38741ae87b154e473ea43e5. Latest consensus state: current_term: 3 leader_uuid: "44acc162c38741ae87b154e473ea43e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:11.066248  7202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:11.066847  7138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "44acc162c38741ae87b154e473ea43e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:11.066968  7138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:11.067584  7202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "44acc162c38741ae87b154e473ea43e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:11.067715  7202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:11.072861  7222 catalog_manager.cc:1295] Loaded cluster ID: d91d2b9b4b8346a79f6d8c297e4122e7
I20260812 06:37:11.072960  7222 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08: loading cluster ID for follower catalog manager: success
I20260812 06:37:11.074630  7222 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08: acquiring CA information for follower catalog manager: success
I20260812 06:37:11.075766  7222 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:37:11.095089  7461 heartbeater.cc:499] Master 127.3.193.188:36179 was elected leader, sending a full tablet report...
W20260812 06:37:11.132099  7030 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.193.189:40035 timed out after 0.088s (SENT)
W20260812 06:37:11.192145  6795 leader_election.cc:336] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fc150343161429397bb523143a9297a (127.3.193.189:40035): Timed out: RequestConsensusVote RPC to 127.3.193.189:40035 timed out after 0.076s (SENT)
I20260812 06:37:11.192266  6795 leader_election.cc:304] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 525d0091035148378fd1ee151c54db08; no voters: 3fc150343161429397bb523143a9297a, 44acc162c38741ae87b154e473ea43e5
I20260812 06:37:11.192489  7202 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260812 06:37:11.221436  7030 leader_election.cc:336] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3fc150343161429397bb523143a9297a (127.3.193.189:40035): Timed out: RequestConsensusVote RPC to 127.3.193.189:40035 timed out after 0.088s (SENT)
W20260812 06:37:11.263670  7030 leader_election.cc:336] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fc150343161429397bb523143a9297a (127.3.193.189:40035): Timed out: RequestConsensusVote RPC to 127.3.193.189:40035 timed out after 0.111s (SENT)
W20260812 06:37:11.484040  7490 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:10.978517 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.189:40035 (local address 127.0.0.1:45278)
0812 06:37:10.978679 (+   162us) negotiation.cc:107] Waiting for socket to connect
0812 06:37:10.978694 (+    15us) client_negotiation.cc:175] Beginning negotiation
0812 06:37:10.978767 (+    73us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:37:11.483916 (+505149us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.189:40035: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":68}
I20260812 06:37:11.493795  7163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "3fc150343161429397bb523143a9297a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:11.493979  7163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [sys.catalog]: This master's current role is: LEADER
W20260812 06:37:11.495487  7521 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:11.493659 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:45278 (local address 127.3.193.189:40035)
0812 06:37:11.494587 (+   928us) server_negotiation.cc:207] Beginning negotiation
0812 06:37:11.494591 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 06:37:11.494605 (+    14us) server_negotiation.cc:408] Connection header received
0812 06:37:11.494648 (+    43us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:37:11.494651 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:37:11.494710 (+    59us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:37:11.494860 (+   150us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45278: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":848}
I20260812 06:37:11.495503  6831 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 3 FOLLOWER]: Rejecting Update request from peer 3fc150343161429397bb523143a9297a for earlier term 2. Current term is 3. Ops: []
I20260812 06:37:11.495615  7074 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 3 LEADER]: Rejecting Update request from peer 3fc150343161429397bb523143a9297a for earlier term 2. Current term is 3. Ops: []
I20260812 06:37:11.496014  7479 consensus_queue.cc:1059] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 06:37:11.496176  7479 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 2 LEADER]: Stepping down as leader of term 2
I20260812 06:37:11.496212  6955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44acc162c38741ae87b154e473ea43e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3fc150343161429397bb523143a9297a"
I20260812 06:37:11.496531  6952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "525d0091035148378fd1ee151c54db08" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3fc150343161429397bb523143a9297a" is_pre_election: true
I20260812 06:37:11.496227  7479 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3fc150343161429397bb523143a9297a, State: Running, Role: LEADER
I20260812 06:37:11.496812  7479 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:11.496971  6950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "525d0091035148378fd1ee151c54db08" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3fc150343161429397bb523143a9297a" is_pre_election: true
I20260812 06:37:11.497138  7479 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:37:11.497217  6953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44acc162c38741ae87b154e473ea43e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3fc150343161429397bb523143a9297a" is_pre_election: true
I20260812 06:37:11.497413  6958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44acc162c38741ae87b154e473ea43e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3fc150343161429397bb523143a9297a" is_pre_election: true
I20260812 06:37:11.497665  6957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44acc162c38741ae87b154e473ea43e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3fc150343161429397bb523143a9297a"
I20260812 06:37:11.498504  6956 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 3 FOLLOWER]: Refusing update from remote peer 44acc162c38741ae87b154e473ea43e5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 06:37:11.499326  7199 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 06:37:11.500617  7163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "44acc162c38741ae87b154e473ea43e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:11.500732  7163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:11.500612  7479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44acc162c38741ae87b154e473ea43e5. Latest consensus state: current_term: 3 leader_uuid: "44acc162c38741ae87b154e473ea43e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:11.500938  7479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:11.574067  7202 raft_consensus.cc:493] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 44acc162c38741ae87b154e473ea43e5)
I20260812 06:37:11.574254  7202 raft_consensus.cc:515] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:11.574764  7202 leader_election.cc:290] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3fc150343161429397bb523143a9297a (127.3.193.189:40035), 44acc162c38741ae87b154e473ea43e5 (127.3.193.188:36179)
I20260812 06:37:11.575235  6956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "525d0091035148378fd1ee151c54db08" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3fc150343161429397bb523143a9297a" is_pre_election: true
I20260812 06:37:11.575235  7479 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 44acc162c38741ae87b154e473ea43e5)
I20260812 06:37:11.575431  7479 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:11.575685  6956 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 525d0091035148378fd1ee151c54db08 in term 3.
I20260812 06:37:11.575887  7479 leader_election.cc:290] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 525d0091035148378fd1ee151c54db08 (127.3.193.190:44409), 44acc162c38741ae87b154e473ea43e5 (127.3.193.188:36179)
I20260812 06:37:11.576210  6795 leader_election.cc:304] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc150343161429397bb523143a9297a, 525d0091035148378fd1ee151c54db08; no voters: 
I20260812 06:37:11.576253  6831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3fc150343161429397bb523143a9297a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "525d0091035148378fd1ee151c54db08" is_pre_election: true
I20260812 06:37:11.576354  6831 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fc150343161429397bb523143a9297a in term 3.
I20260812 06:37:11.576416  7202 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 06:37:11.576467  7202 raft_consensus.cc:493] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 44acc162c38741ae87b154e473ea43e5)
I20260812 06:37:11.576510  7202 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 3 FOLLOWER]: Advancing to term 4
I20260812 06:37:11.576655  6917 leader_election.cc:304] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc150343161429397bb523143a9297a, 525d0091035148378fd1ee151c54db08; no voters: 
I20260812 06:37:11.576893  7479 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 06:37:11.576974  7479 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 44acc162c38741ae87b154e473ea43e5)
I20260812 06:37:11.577013  7479 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 3 FOLLOWER]: Advancing to term 4
I20260812 06:37:11.577709  7202 raft_consensus.cc:515] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:11.577982  7479 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:11.578109  7202 leader_election.cc:290] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [CANDIDATE]: Term 4 election: Requested vote from peers 3fc150343161429397bb523143a9297a (127.3.193.189:40035), 44acc162c38741ae87b154e473ea43e5 (127.3.193.188:36179)
I20260812 06:37:11.578434  6956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "525d0091035148378fd1ee151c54db08" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3fc150343161429397bb523143a9297a"
I20260812 06:37:11.578603  6956 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 525d0091035148378fd1ee151c54db08 in current term 4: Already voted for candidate 3fc150343161429397bb523143a9297a in this term.
I20260812 06:37:11.578634  6831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3fc150343161429397bb523143a9297a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "525d0091035148378fd1ee151c54db08"
I20260812 06:37:11.578779  6831 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3fc150343161429397bb523143a9297a in current term 4: Already voted for candidate 525d0091035148378fd1ee151c54db08 in this term.
I20260812 06:37:11.579191  7479 leader_election.cc:290] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [CANDIDATE]: Term 4 election: Requested vote from peers 525d0091035148378fd1ee151c54db08 (127.3.193.190:44409), 44acc162c38741ae87b154e473ea43e5 (127.3.193.188:36179)
W20260812 06:37:11.730999  6795 leader_election.cc:336] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 44acc162c38741ae87b154e473ea43e5 (127.3.193.188:36179): Timed out: RequestConsensusVote RPC to 127.3.193.188:36179 timed out after 0.077s (SENT)
W20260812 06:37:11.777168  6917 leader_election.cc:336] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 44acc162c38741ae87b154e473ea43e5 (127.3.193.188:36179): Timed out: RequestConsensusVote RPC to 127.3.193.188:36179 timed out after 0.098s (SENT)
I20260812 06:37:11.777312  6917 leader_election.cc:304] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fc150343161429397bb523143a9297a; no voters: 44acc162c38741ae87b154e473ea43e5, 525d0091035148378fd1ee151c54db08
W20260812 06:37:11.777417  6795 leader_election.cc:336] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 44acc162c38741ae87b154e473ea43e5 (127.3.193.188:36179): Timed out: RequestConsensusVote RPC to 127.3.193.188:36179 timed out after 0.099s (SENT)
I20260812 06:37:11.777551  6795 leader_election.cc:304] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 525d0091035148378fd1ee151c54db08; no voters: 3fc150343161429397bb523143a9297a, 44acc162c38741ae87b154e473ea43e5
I20260812 06:37:11.777583  7479 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 06:37:11.777765  7202 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260812 06:37:11.782954  6917 leader_election.cc:336] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 44acc162c38741ae87b154e473ea43e5 (127.3.193.188:36179): Timed out: RequestConsensusVote RPC to 127.3.193.188:36179 timed out after 0.103s (SENT)
I20260812 06:37:11.872535  7202 raft_consensus.cc:493] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:37:11.872715  7202 raft_consensus.cc:515] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:11.873172  7202 leader_election.cc:290] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3fc150343161429397bb523143a9297a (127.3.193.189:40035), 44acc162c38741ae87b154e473ea43e5 (127.3.193.188:36179)
I20260812 06:37:11.873662  6956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "525d0091035148378fd1ee151c54db08" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3fc150343161429397bb523143a9297a" is_pre_election: true
I20260812 06:37:11.873833  6956 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 525d0091035148378fd1ee151c54db08 in term 4.
I20260812 06:37:11.874099  6795 leader_election.cc:304] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc150343161429397bb523143a9297a, 525d0091035148378fd1ee151c54db08; no voters: 
I20260812 06:37:11.874284  7202 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 06:37:11.874351  7202 raft_consensus.cc:493] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:37:11.874378  7202 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 4 FOLLOWER]: Advancing to term 5
I20260812 06:37:11.875294  7202 raft_consensus.cc:515] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:11.875654  7202 leader_election.cc:290] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [CANDIDATE]: Term 5 election: Requested vote from peers 3fc150343161429397bb523143a9297a (127.3.193.189:40035), 44acc162c38741ae87b154e473ea43e5 (127.3.193.188:36179)
I20260812 06:37:11.876031  6956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "525d0091035148378fd1ee151c54db08" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3fc150343161429397bb523143a9297a"
I20260812 06:37:11.876169  6956 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 4 FOLLOWER]: Advancing to term 5
I20260812 06:37:11.877054  6956 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 525d0091035148378fd1ee151c54db08 in term 5.
I20260812 06:37:11.877370  6795 leader_election.cc:304] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc150343161429397bb523143a9297a, 525d0091035148378fd1ee151c54db08; no voters: 
I20260812 06:37:11.877571  7202 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 5 FOLLOWER]: Leader election won for term 5
I20260812 06:37:11.877744  7202 raft_consensus.cc:697] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 5 LEADER]: Becoming Leader. State: Replica: 525d0091035148378fd1ee151c54db08, State: Running, Role: LEADER
I20260812 06:37:11.878068  7202 consensus_queue.cc:237] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:11.879225  7532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 525d0091035148378fd1ee151c54db08. Latest consensus state: current_term: 5 leader_uuid: "525d0091035148378fd1ee151c54db08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:11.879350  7532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:11.879720  7533 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:37:11.880549  7533 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:37:11.880875  7533 catalog_manager.cc:1295] Loaded cluster ID: d91d2b9b4b8346a79f6d8c297e4122e7
I20260812 06:37:11.880920  7533 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:37:11.881397  7533 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:37:11.881758  7533 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08: Loaded TSK: 0
I20260812 06:37:11.882318  7533 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:37:11.909809  6956 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 5 FOLLOWER]: Refusing update from remote peer 525d0091035148378fd1ee151c54db08: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260812 06:37:11.910326  7532 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 06:37:11.911729  7479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 525d0091035148378fd1ee151c54db08. Latest consensus state: current_term: 5 leader_uuid: "525d0091035148378fd1ee151c54db08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:11.911861  7479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:11.913043  7202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "525d0091035148378fd1ee151c54db08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:11.913169  7202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:11.913496  7479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "525d0091035148378fd1ee151c54db08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:11.913621  7479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:11.921541  7463 heartbeater.cc:499] Master 127.3.193.190:44409 was elected leader, sending a full tablet report...
W20260812 06:37:12.028007  7517 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:11.519460 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.188:36179 (local address 127.0.0.1:53458)
0812 06:37:11.519610 (+   150us) negotiation.cc:107] Waiting for socket to connect
0812 06:37:11.519624 (+    14us) client_negotiation.cc:175] Beginning negotiation
0812 06:37:11.519762 (+   138us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:37:12.027881 (+508119us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.188:36179: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 06:37:12.028559  7469 master_proxy_rpc.cc:203] Re-attempting ListTables request to leader Master (127.3.193.188:36179)
I20260812 06:37:12.037319  3846 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu with pid 7234
W20260812 06:37:12.045272  6795 leader_election.cc:336] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 44acc162c38741ae87b154e473ea43e5 (127.3.193.188:36179): Timed out: RequestConsensusVote RPC to 127.3.193.188:36179 timed out after 0.085s (SENT)
I20260812 06:37:12.045871  7138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "44acc162c38741ae87b154e473ea43e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:12.046007  7138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:12.045948  7062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "525d0091035148378fd1ee151c54db08" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "44acc162c38741ae87b154e473ea43e5"
I20260812 06:37:12.046380  7062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3fc150343161429397bb523143a9297a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "44acc162c38741ae87b154e473ea43e5"
I20260812 06:37:12.046682  7062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3fc150343161429397bb523143a9297a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "44acc162c38741ae87b154e473ea43e5"
I20260812 06:37:12.046926  7062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3fc150343161429397bb523143a9297a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "44acc162c38741ae87b154e473ea43e5" is_pre_election: true
I20260812 06:37:12.047188  7069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "525d0091035148378fd1ee151c54db08" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "44acc162c38741ae87b154e473ea43e5"
I20260812 06:37:12.047472  7070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "525d0091035148378fd1ee151c54db08" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "44acc162c38741ae87b154e473ea43e5" is_pre_election: true
I20260812 06:37:12.047767  7073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "525d0091035148378fd1ee151c54db08" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "44acc162c38741ae87b154e473ea43e5"
I20260812 06:37:12.048056  7058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3fc150343161429397bb523143a9297a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "44acc162c38741ae87b154e473ea43e5" is_pre_election: true
I20260812 06:37:12.048341  7063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "525d0091035148378fd1ee151c54db08" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "44acc162c38741ae87b154e473ea43e5" is_pre_election: true
I20260812 06:37:12.048720  7064 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 06:37:12.048804  7064 raft_consensus.cc:740] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 44acc162c38741ae87b154e473ea43e5, State: Running, Role: LEADER
I20260812 06:37:12.049005  7064 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } }
I20260812 06:37:12.049175  7064 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 3 FOLLOWER]: Advancing to term 5
I20260812 06:37:12.050905  7064 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 5 FOLLOWER]: Refusing update from remote peer 525d0091035148378fd1ee151c54db08: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260812 06:37:12.051237  7074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "525d0091035148378fd1ee151c54db08" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "44acc162c38741ae87b154e473ea43e5" is_pre_election: true
I20260812 06:37:12.051406  7074 raft_consensus.cc:2368] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 525d0091035148378fd1ee151c54db08 for earlier term 4. Current term is 5.
I20260812 06:37:12.051653  7076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "525d0091035148378fd1ee151c54db08" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "44acc162c38741ae87b154e473ea43e5" is_pre_election: true
I20260812 06:37:12.051802  7076 raft_consensus.cc:2368] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 525d0091035148378fd1ee151c54db08 for earlier term 4. Current term is 5.
I20260812 06:37:12.052011  7077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "525d0091035148378fd1ee151c54db08" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "44acc162c38741ae87b154e473ea43e5"
I20260812 06:37:12.052151  7077 raft_consensus.cc:2368] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 525d0091035148378fd1ee151c54db08 for earlier term 4. Current term is 5.
I20260812 06:37:12.053541  6956 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 3fc150343161429397bb523143a9297a [term 5 FOLLOWER]: Rejecting Update request from peer 44acc162c38741ae87b154e473ea43e5 for earlier term 3. Current term is 5. Ops: []
I20260812 06:37:12.053964  6831 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [term 5 LEADER]: Rejecting Update request from peer 44acc162c38741ae87b154e473ea43e5 for earlier term 3. Current term is 5. Ops: []
I20260812 06:37:12.055176  7202 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 525d0091035148378fd1ee151c54db08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 06:37:12.058902  7138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 525d0091035148378fd1ee151c54db08. Latest consensus state: current_term: 5 leader_uuid: "525d0091035148378fd1ee151c54db08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:12.059056  7138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:12.059183  7138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "525d0091035148378fd1ee151c54db08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
I20260812 06:37:12.059293  7138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44acc162c38741ae87b154e473ea43e5 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 06:37:12.065735  7545 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:12.060194 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:53458 (local address 127.3.193.188:36179)
0812 06:37:12.065394 (+  5200us) server_negotiation.cc:207] Beginning negotiation
0812 06:37:12.065397 (+     3us) server_negotiation.cc:400] Waiting for connection header
0812 06:37:12.065409 (+    12us) server_negotiation.cc:408] Connection header received
0812 06:37:12.065442 (+    33us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:37:12.065445 (+     3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:37:12.065490 (+    45us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:37:12.065613 (+   123us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53458: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":5133,"thread_start_us":198,"threads_started":1}
W20260812 06:37:12.068344  7543 catalog_manager.cc:1445] Failed waiting for node to catch up after master election: Illegal state: Node 44acc162c38741ae87b154e473ea43e5 not leader. Raft Consensus state: current_term: 5 leader_uuid: "525d0091035148378fd1ee151c54db08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525d0091035148378fd1ee151c54db08" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 44409 } } peers { permanent_uuid: "3fc150343161429397bb523143a9297a" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 40035 } } peers { permanent_uuid: "44acc162c38741ae87b154e473ea43e5" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 36179 } } }
W20260812 06:37:12.073765  7037 connection.cc:573] server connection from 127.3.193.129:35573 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 06:37:12.074329  3846 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu with pid 6771
I20260812 06:37:12.086215  3846 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu with pid 6884
I20260812 06:37:12.102190  3846 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu with pid 7002
2026-08-12T06:37:12Z chronyd exiting
[       OK ] ClientFailoverOnNegotiationTimeoutITest.Kudu2021ConnectToMaster (2730 ms)
[ RUN      ] ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster
2026-08-12T06:37:12Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:37:12Z Disabled control of system clock
I20260812 06:37:12.140352  3846 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
/tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.193.190:37113
--webserver_interface=127.3.193.190
--webserver_port=0
--builtin_ntp_servers=127.3.193.148:34345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.193.190:37113,127.3.193.189:42351,127.3.193.188:34287
--raft_heartbeat_interval_ms=25
--leader_failure_exp_backoff_max_delta_ms=1000 with env {}
W20260812 06:37:12.246192  7557 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:12.246416  7557 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:12.246503  7557 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:12.249823  7557 flags.cc:432] Enabled experimental flag: --leader_failure_exp_backoff_max_delta_ms=1000
W20260812 06:37:12.249886  7557 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:37:12.249907  7557 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:12.249965  7557 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:37:12.250005  7557 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:37:12.253887  7557 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.193.148:34345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_exp_backoff_max_delta_ms=1000
--raft_heartbeat_interval_ms=25
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.193.190:37113,127.3.193.189:42351,127.3.193.188:34287
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.193.190:37113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.193.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.7557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:12.255033  7557 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:12.256197  7557 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:12.261901  7573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:12.261921  7570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:12.261921  7568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:12.262111  7557 server_base.cc:1061] running on GCE node
I20260812 06:37:12.262660  7557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:12.263527  7557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:12.264712  7557 hybrid_clock.cc:648] HybridClock initialized: now 1786516632264689 us; error 40 us; skew 500 ppm
I20260812 06:37:12.266345  7557 webserver.cc:533] Webserver started at http://127.3.193.190:35553/ using document root <none> and password file <none>
I20260812 06:37:12.266938  7557 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:12.267026  7557 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:12.267231  7557 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:12.268800  7557 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-0/data/instance:
uuid: "0c41b4ed033d44009007ab44c3ec34af"
format_stamp: "Formatted at 2026-08-12 06:37:12 on dist-test-slave-mjjr"
I20260812 06:37:12.269308  7557 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-0/wal/instance:
uuid: "0c41b4ed033d44009007ab44c3ec34af"
format_stamp: "Formatted at 2026-08-12 06:37:12 on dist-test-slave-mjjr"
I20260812 06:37:12.272639  7557 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 06:37:12.274780  7585 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:12.275941  7557 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:12.276080  7557 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-0/data,/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-0/wal
uuid: "0c41b4ed033d44009007ab44c3ec34af"
format_stamp: "Formatted at 2026-08-12 06:37:12 on dist-test-slave-mjjr"
I20260812 06:37:12.276186  7557 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:12.293666  7557 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:12.294401  7557 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:12.294631  7557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:12.302986  7557 rpc_server.cc:307] RPC server started. Bound to: 127.3.193.190:37113
I20260812 06:37:12.302996  7667 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.193.190:37113 every 8 connection(s)
I20260812 06:37:12.304138  7557 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-0/data/info.pb
I20260812 06:37:12.305781  3846 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu as pid 7557
I20260812 06:37:12.305931  3846 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-0/wal/instance
I20260812 06:37:12.306602  3846 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
/tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.193.189:42351
--webserver_interface=127.3.193.189
--webserver_port=0
--builtin_ntp_servers=127.3.193.148:34345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.193.190:37113,127.3.193.189:42351,127.3.193.188:34287
--raft_heartbeat_interval_ms=25
--leader_failure_exp_backoff_max_delta_ms=1000 with env {}
I20260812 06:37:12.307152  7669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:12.310693  7669 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:12.321873  7669 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } has no permanent_uuid. Determining permanent_uuid...
W20260812 06:37:12.322736  7592 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.193.189:42351: connect: Connection refused (error 111)
W20260812 06:37:12.323689  7669 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.193.189:42351: Network error: Client connection negotiation failed: client connection to 127.3.193.189:42351: connect: Connection refused (error 111)
I20260812 06:37:12.372926  7669 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } attempt: 1
W20260812 06:37:12.374815  7669 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.193.189:42351: Network error: Client connection negotiation failed: client connection to 127.3.193.189:42351: connect: Connection refused (error 111)
W20260812 06:37:12.415151  7672 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:12.415397  7672 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:12.415489  7672 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:12.418716  7672 flags.cc:432] Enabled experimental flag: --leader_failure_exp_backoff_max_delta_ms=1000
W20260812 06:37:12.418813  7672 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:37:12.418859  7672 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:12.418897  7672 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:37:12.418932  7672 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:37:12.422701  7672 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.193.148:34345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_exp_backoff_max_delta_ms=1000
--raft_heartbeat_interval_ms=25
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.193.190:37113,127.3.193.189:42351,127.3.193.188:34287
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.193.189:42351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.193.189
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.7672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:12.423827  7672 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:12.424959  7672 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:12.430449  7693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:12.430510  7697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:12.430449  7695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:12.430833  7672 server_base.cc:1061] running on GCE node
I20260812 06:37:12.431293  7672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:12.432237  7672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:12.433414  7672 hybrid_clock.cc:648] HybridClock initialized: now 1786516632433396 us; error 32 us; skew 500 ppm
I20260812 06:37:12.435060  7672 webserver.cc:533] Webserver started at http://127.3.193.189:34029/ using document root <none> and password file <none>
I20260812 06:37:12.435591  7672 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:12.435709  7672 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:12.435906  7672 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:12.437394  7672 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-1/data/instance:
uuid: "75f2600fa64843ffa192098e4b00d8be"
format_stamp: "Formatted at 2026-08-12 06:37:12 on dist-test-slave-mjjr"
I20260812 06:37:12.437812  7672 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-1/wal/instance:
uuid: "75f2600fa64843ffa192098e4b00d8be"
format_stamp: "Formatted at 2026-08-12 06:37:12 on dist-test-slave-mjjr"
I20260812 06:37:12.441088  7672 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 06:37:12.443041  7669 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } attempt: 2
I20260812 06:37:12.443315  7706 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:12.444613  7672 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
W20260812 06:37:12.444679  7669 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.193.189:42351: Network error: Client connection negotiation failed: client connection to 127.3.193.189:42351: connect: Connection refused (error 111)
I20260812 06:37:12.444802  7672 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-1/data,/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-1/wal
uuid: "75f2600fa64843ffa192098e4b00d8be"
format_stamp: "Formatted at 2026-08-12 06:37:12 on dist-test-slave-mjjr"
I20260812 06:37:12.444937  7672 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:12.463040  7672 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:12.463837  7672 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:12.464053  7672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:12.471534  7672 rpc_server.cc:307] RPC server started. Bound to: 127.3.193.189:42351
I20260812 06:37:12.471556  7806 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.193.189:42351 every 8 connection(s)
I20260812 06:37:12.472597  7672 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-1/data/info.pb
I20260812 06:37:12.473286  3846 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu as pid 7672
I20260812 06:37:12.473433  3846 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-1/wal/instance
I20260812 06:37:12.474037  3846 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
/tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.193.188:34287
--webserver_interface=127.3.193.188
--webserver_port=0
--builtin_ntp_servers=127.3.193.148:34345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.193.190:37113,127.3.193.189:42351,127.3.193.188:34287
--raft_heartbeat_interval_ms=25
--leader_failure_exp_backoff_max_delta_ms=1000 with env {}
I20260812 06:37:12.475713  7807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:12.479336  7807 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:12.488346  7807 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:12.497501  7807 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } has no permanent_uuid. Determining permanent_uuid...
W20260812 06:37:12.498394  7708 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.193.188:34287: connect: Connection refused (error 111)
W20260812 06:37:12.499347  7807 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.193.188:34287: Network error: Client connection negotiation failed: client connection to 127.3.193.188:34287: connect: Connection refused (error 111)
I20260812 06:37:12.535882  7669 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } attempt: 3
I20260812 06:37:12.540968  7669 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } has no permanent_uuid. Determining permanent_uuid...
W20260812 06:37:12.542397  7669 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.193.188:34287: Network error: Client connection negotiation failed: client connection to 127.3.193.188:34287: connect: Connection refused (error 111)
I20260812 06:37:12.548604  7807 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } attempt: 1
W20260812 06:37:12.550119  7807 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.193.188:34287: Network error: Client connection negotiation failed: client connection to 127.3.193.188:34287: connect: Connection refused (error 111)
I20260812 06:37:12.573601  7669 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } attempt: 1
W20260812 06:37:12.575517  7669 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.193.188:34287: Network error: Client connection negotiation failed: client connection to 127.3.193.188:34287: connect: Connection refused (error 111)
W20260812 06:37:12.588514  7810 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:12.588794  7810 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:12.588835  7810 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:12.592509  7810 flags.cc:432] Enabled experimental flag: --leader_failure_exp_backoff_max_delta_ms=1000
W20260812 06:37:12.592595  7810 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:37:12.592617  7810 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:12.592636  7810 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:37:12.592679  7810 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:37:12.596868  7810 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.193.148:34345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_exp_backoff_max_delta_ms=1000
--raft_heartbeat_interval_ms=25
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.193.190:37113,127.3.193.189:42351,127.3.193.188:34287
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.193.188:34287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.193.188
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.7810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:12.598090  7810 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:12.599545  7810 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:12.605993  7821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:12.605980  7827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:12.605984  7823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:12.606060  7810 server_base.cc:1061] running on GCE node
I20260812 06:37:12.606835  7810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:12.607954  7810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:12.609191  7810 hybrid_clock.cc:648] HybridClock initialized: now 1786516632609159 us; error 58 us; skew 500 ppm
I20260812 06:37:12.611358  7810 webserver.cc:533] Webserver started at http://127.3.193.188:42195/ using document root <none> and password file <none>
I20260812 06:37:12.612071  7810 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:12.612161  7810 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:12.612399  7810 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:12.614091  7810 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-2/data/instance:
uuid: "3f15fbab38e34ad981ef77a9d882bc08"
format_stamp: "Formatted at 2026-08-12 06:37:12 on dist-test-slave-mjjr"
I20260812 06:37:12.614602  7810 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-2/wal/instance:
uuid: "3f15fbab38e34ad981ef77a9d882bc08"
format_stamp: "Formatted at 2026-08-12 06:37:12 on dist-test-slave-mjjr"
I20260812 06:37:12.618160  7810 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 06:37:12.618337  7807 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } attempt: 2
W20260812 06:37:12.620085  7807 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.193.188:34287: Network error: Client connection negotiation failed: client connection to 127.3.193.188:34287: connect: Connection refused (error 111)
I20260812 06:37:12.621126  7843 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:12.622156  7810 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 06:37:12.622320  7810 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-2/data,/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-2/wal
uuid: "3f15fbab38e34ad981ef77a9d882bc08"
format_stamp: "Formatted at 2026-08-12 06:37:12 on dist-test-slave-mjjr"
I20260812 06:37:12.622452  7810 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:12.639576  7810 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:12.640470  7810 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:12.640702  7810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:12.649380  7810 rpc_server.cc:307] RPC server started. Bound to: 127.3.193.188:34287
I20260812 06:37:12.649392  7929 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.193.188:34287 every 8 connection(s)
I20260812 06:37:12.650557  7810 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-2/data/info.pb
I20260812 06:37:12.650745  7669 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } attempt: 2
I20260812 06:37:12.650920  3846 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu as pid 7810
I20260812 06:37:12.651052  3846 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/master-2/wal/instance
I20260812 06:37:12.654449  7931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:12.658671  7931 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:12.665606  7669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af: Bootstrap starting.
I20260812 06:37:12.667865  7669 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:12.668769  7669 log.cc:826] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:12.669369  7931 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:12.670681  7669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af: No bootstrap required, opened a new log
I20260812 06:37:12.673971  7931 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:12.674206  7669 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } }
I20260812 06:37:12.674393  7669 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:12.674479  7669 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c41b4ed033d44009007ab44c3ec34af, State: Initialized, Role: FOLLOWER
I20260812 06:37:12.675175  7669 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } }
I20260812 06:37:12.675622  7955 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:37:12.675798  7955 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } }
I20260812 06:37:12.676544  7669 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:37:12.676738  7955 leader_election.cc:290] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75f2600fa64843ffa192098e4b00d8be (127.3.193.189:42351), 3f15fbab38e34ad981ef77a9d882bc08 (127.3.193.188:34287)
I20260812 06:37:12.676899  7955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } } }
I20260812 06:37:12.676996  7955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:12.681409  7931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08: Bootstrap starting.
W20260812 06:37:12.683286  7980 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:37:12.683354  7980 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:37:12.684111  7931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:12.682389  7557 master_runner.cc:187] Error getting master registration for 127.3.193.189:42351: OK, instance_id { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" instance_seqno: 1786516632469590 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 06:37:12.684803  7557 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260812 06:37:12.685039  7931 log.cc:826] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:12.686982  7931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08: No bootstrap required, opened a new log
I20260812 06:37:12.691011  7931 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } }
I20260812 06:37:12.691251  7931 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:12.691329  7931 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f15fbab38e34ad981ef77a9d882bc08, State: Initialized, Role: FOLLOWER
I20260812 06:37:12.691840  7931 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } }
I20260812 06:37:12.692582  7984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } } }
I20260812 06:37:12.692987  7984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:12.693128  7931 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:37:12.693156  7888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0c41b4ed033d44009007ab44c3ec34af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f15fbab38e34ad981ef77a9d882bc08" is_pre_election: true
I20260812 06:37:12.693532  7888 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c41b4ed033d44009007ab44c3ec34af in term 0.
I20260812 06:37:12.693985  7588 leader_election.cc:304] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c41b4ed033d44009007ab44c3ec34af, 3f15fbab38e34ad981ef77a9d882bc08; no voters: 
I20260812 06:37:12.694306  7954 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:37:12.694376  7954 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:37:12.694427  7954 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:12.695564  7954 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } }
I20260812 06:37:12.695999  7954 leader_election.cc:290] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [CANDIDATE]: Term 1 election: Requested vote from peers 75f2600fa64843ffa192098e4b00d8be (127.3.193.189:42351), 3f15fbab38e34ad981ef77a9d882bc08 (127.3.193.188:34287)
I20260812 06:37:12.697320  7888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0c41b4ed033d44009007ab44c3ec34af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f15fbab38e34ad981ef77a9d882bc08"
I20260812 06:37:12.697438  7888 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:12.698658  7888 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c41b4ed033d44009007ab44c3ec34af in term 1.
I20260812 06:37:12.699465  7588 leader_election.cc:304] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c41b4ed033d44009007ab44c3ec34af, 3f15fbab38e34ad981ef77a9d882bc08; no voters: 
I20260812 06:37:12.699767  7954 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:37:12.699859  7954 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [term 1 LEADER]: Becoming Leader. State: Replica: 0c41b4ed033d44009007ab44c3ec34af, State: Running, Role: LEADER
I20260812 06:37:12.700183  7954 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } }
I20260812 06:37:12.702580  7955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c41b4ed033d44009007ab44c3ec34af. Latest consensus state: current_term: 1 leader_uuid: "0c41b4ed033d44009007ab44c3ec34af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } } }
I20260812 06:37:12.702710  7955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:12.702883  7810 master_runner.cc:187] Error getting master registration for 127.3.193.189:42351: OK, instance_id { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" instance_seqno: 1786516632469590 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 06:37:12.704854  7810 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260812 06:37:12.711251  7807 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } attempt: 3
W20260812 06:37:12.712808  8005 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:37:12.715026  8005 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:37:12.718606  7807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be: Bootstrap starting.
I20260812 06:37:12.719069  8007 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:37:12.720199  8007 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:37:12.721354  7807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:12.722546  7807 log.cc:826] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:12.725726  7888 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [term 1 FOLLOWER]: Refusing update from remote peer 0c41b4ed033d44009007ab44c3ec34af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:37:12.727540  7955 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:12.731617  7807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be: No bootstrap required, opened a new log
I20260812 06:37:12.732290  7984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c41b4ed033d44009007ab44c3ec34af. Latest consensus state: current_term: 1 leader_uuid: "0c41b4ed033d44009007ab44c3ec34af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } } }
I20260812 06:37:12.732422  7984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:12.738335  7807 raft_consensus.cc:359] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } }
I20260812 06:37:12.738538  7807 raft_consensus.cc:385] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:12.738610  7807 raft_consensus.cc:740] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f2600fa64843ffa192098e4b00d8be, State: Initialized, Role: FOLLOWER
I20260812 06:37:12.739140  7807 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } }
I20260812 06:37:12.740476  7807 sys_catalog.cc:565] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:37:12.740504  7753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0c41b4ed033d44009007ab44c3ec34af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f2600fa64843ffa192098e4b00d8be" is_pre_election: true
I20260812 06:37:12.740767  7753 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c41b4ed033d44009007ab44c3ec34af in term 0.
I20260812 06:37:12.741250  7750 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:12.742221  7955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c41b4ed033d44009007ab44c3ec34af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } } }
I20260812 06:37:12.742338  7955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:12.742913  7751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0c41b4ed033d44009007ab44c3ec34af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f2600fa64843ffa192098e4b00d8be"
I20260812 06:37:12.743477  7984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c41b4ed033d44009007ab44c3ec34af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } } }
I20260812 06:37:12.743582  7984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:12.743633  8013 mvcc.cc:204] Tried to move back new op lower bound from 7317572127638642688 to 7317572127543267328. Current Snapshot: MvccSnapshot[applied={T|T < 7317572127638642688}]
I20260812 06:37:12.743705  7954 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:12.744011  7751 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c41b4ed033d44009007ab44c3ec34af in term 1.
I20260812 06:37:12.747921  8010 mvcc.cc:204] Tried to move back new op lower bound from 7317572127638642688 to 7317572127543267328. Current Snapshot: MvccSnapshot[applied={T|T < 7317572127638642688 or (T in {7317572127638642688})}]
I20260812 06:37:12.749763  8020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c41b4ed033d44009007ab44c3ec34af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } } }
I20260812 06:37:12.750100  8020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:12.750301  8020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c41b4ed033d44009007ab44c3ec34af. Latest consensus state: current_term: 1 leader_uuid: "0c41b4ed033d44009007ab44c3ec34af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } } }
I20260812 06:37:12.750378  8020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:12.750460  8020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c41b4ed033d44009007ab44c3ec34af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } } }
I20260812 06:37:12.750710  8020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:12.752301  8007 catalog_manager.cc:1383] Generated new cluster ID: 2ece07cc6f734d6198a1227638763d0a
I20260812 06:37:12.752364  8007 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
W20260812 06:37:12.774966  8038 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:37:12.776891  8038 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:37:12.776806  8021 mvcc.cc:204] Tried to move back new op lower bound from 7317572127638642688 to 7317572127543267328. Current Snapshot: MvccSnapshot[applied={T|T < 7317572127638642688}]
I20260812 06:37:12.780879  7955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c41b4ed033d44009007ab44c3ec34af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } } }
I20260812 06:37:12.781005  7955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:12.784852  8007 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 06:37:12.785854  8007 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:37:12.804483  8007 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af: Generated new TSK 0
I20260812 06:37:12.805101  8007 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:37:12.815332  3846 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
/tmp/dist-test-taskNlrYvv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.193.129:0
--local_ip_for_outbound_sockets=127.3.193.129
--webserver_interface=127.3.193.129
--webserver_port=0
--tserver_master_addrs=127.3.193.190:37113,127.3.193.189:42351,127.3.193.188:34287
--builtin_ntp_servers=127.3.193.148:34345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=25
--leader_failure_exp_backoff_max_delta_ms=1000
--heartbeat_interval_ms=25
--enable_txn_system_client_init=true with env {}
W20260812 06:37:12.973421  8045 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:12.973690  8045 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:12.973755  8045 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:12.978523  8045 flags.cc:432] Enabled experimental flag: --leader_failure_exp_backoff_max_delta_ms=1000
W20260812 06:37:12.978631  8045 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:12.978694  8045 flags.cc:432] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:37:12.978765  8045 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.193.129
I20260812 06:37:12.985001  8045 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.193.148:34345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--leader_failure_exp_backoff_max_delta_ms=1000
--raft_heartbeat_interval_ms=25
--fs_data_dirs=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.193.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.193.129
--webserver_port=0
--enable_txn_system_client_init=true
--heartbeat_interval_ms=25
--tserver_master_addrs=127.3.193.190:37113,127.3.193.189:42351,127.3.193.188:34287
--never_fsync=true
--heap_profile_path=/tmp/kudu.8045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.193.129

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:12.986261  8045 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:12.987607  8045 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 06:37:12.990940  8045 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 12
W20260812 06:37:12.995292  8064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:12.995478  8057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:12.995532  8061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:12.996815  8045 server_base.cc:1061] running on GCE node
I20260812 06:37:12.997218  8045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:12.997825  8045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:12.999018  8045 hybrid_clock.cc:648] HybridClock initialized: now 1786516632998998 us; error 35 us; skew 500 ppm
I20260812 06:37:13.001114  8045 webserver.cc:533] Webserver started at http://127.3.193.129:42061/ using document root <none> and password file <none>
I20260812 06:37:13.001726  8045 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:13.001782  8045 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:13.001998  8045 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:13.003592  8045 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/ts-0/data/instance:
uuid: "5f3053ffc7254f1b83117b8bed5815fb"
format_stamp: "Formatted at 2026-08-12 06:37:12 on dist-test-slave-mjjr"
I20260812 06:37:13.004155  8045 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/ts-0/wal/instance:
uuid: "5f3053ffc7254f1b83117b8bed5815fb"
format_stamp: "Formatted at 2026-08-12 06:37:12 on dist-test-slave-mjjr"
I20260812 06:37:13.007606  8045 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 06:37:13.010000  8074 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:13.011190  8045 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:13.011324  8045 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/ts-0/wal
uuid: "5f3053ffc7254f1b83117b8bed5815fb"
format_stamp: "Formatted at 2026-08-12 06:37:12 on dist-test-slave-mjjr"
I20260812 06:37:13.011433  8045 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:13.027987  8045 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:13.028642  8045 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:13.028844  8045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:13.030741  8045 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:13.030791  8045 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:13.030833  8045 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:13.030846  8045 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:13.042778  8045 rpc_server.cc:307] RPC server started. Bound to: 127.3.193.129:41983
I20260812 06:37:13.045291  8276 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.193.129:41983 every 8 connection(s)
I20260812 06:37:13.045650  8045 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/ts-0/data/info.pb
I20260812 06:37:13.048415  8284 heartbeater.cc:344] Connected to a master server at 127.3.193.190:37113
I20260812 06:37:13.048568  8284 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:13.049072  8284 heartbeater.cc:507] Master 127.3.193.190:37113 requested a full tablet report, sending...
I20260812 06:37:13.050642  7614 ts_manager.cc:194] Registered new tserver with Master: 5f3053ffc7254f1b83117b8bed5815fb (127.3.193.129:41983)
I20260812 06:37:13.051343  8283 heartbeater.cc:344] Connected to a master server at 127.3.193.188:34287
I20260812 06:37:13.051491  8283 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:13.051764  8283 heartbeater.cc:507] Master 127.3.193.188:34287 requested a full tablet report, sending...
I20260812 06:37:13.052213  7614 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.3.193.129:42085
I20260812 06:37:13.052779  7871 ts_manager.cc:194] Registered new tserver with Master: 5f3053ffc7254f1b83117b8bed5815fb (127.3.193.129:41983)
I20260812 06:37:13.052983  3846 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu as pid 8045
I20260812 06:37:13.053088  3846 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskNlrYvv/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster.1786516592259368-3846-0/minicluster-data/ts-0/wal/instance
I20260812 06:37:13.055251  8286 heartbeater.cc:344] Connected to a master server at 127.3.193.189:42351
I20260812 06:37:13.055341  8286 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:13.055526  8286 heartbeater.cc:507] Master 127.3.193.189:42351 requested a full tablet report, sending...
I20260812 06:37:13.056618  7736 ts_manager.cc:194] Registered new tserver with Master: 5f3053ffc7254f1b83117b8bed5815fb (127.3.193.129:41983)
I20260812 06:37:13.077999  3846 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260812 06:37:13.079789  8284 heartbeater.cc:499] Master 127.3.193.190:37113 was elected leader, sending a full tablet report...
W20260812 06:37:13.103863  7660 generic_service.cc:139] {username='slave'} at 127.0.0.1:52922 forcing change of non-runtime-safe flag rpc_negotiation_inject_delay_ms
I20260812 06:37:13.103971  7660 generic_service.cc:175] {username='slave'} at 127.0.0.1:52922 changed flags via RPC: rpc_negotiation_inject_delay_ms from '0' to '6000'
W20260812 06:37:13.105013  7678 negotiation.cc:265] Injecting 6000ms delay in negotiation
W20260812 06:37:13.628036  8308 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:13.104849 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.190:37113 (local address 127.0.0.1:52936)
0812 06:37:13.105000 (+   151us) negotiation.cc:107] Waiting for socket to connect
0812 06:37:13.105015 (+    15us) client_negotiation.cc:175] Beginning negotiation
0812 06:37:13.105107 (+    92us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:37:13.627903 (+522796us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.190:37113: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 06:37:13.628720  8302 master_proxy_rpc.cc:203] Re-attempting ListTables request to leader Master (127.3.193.190:37113)
W20260812 06:37:13.629381  8315 negotiation.cc:265] Injecting 6000ms delay in negotiation
I20260812 06:37:13.716239  8005 catalog_manager.cc:1295] Loaded cluster ID: 2ece07cc6f734d6198a1227638763d0a
I20260812 06:37:13.716323  8005 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08: loading cluster ID for follower catalog manager: success
I20260812 06:37:13.718061  8005 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08: acquiring CA information for follower catalog manager: success
I20260812 06:37:13.719048  8005 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:37:13.778159  8038 catalog_manager.cc:1295] Loaded cluster ID: 2ece07cc6f734d6198a1227638763d0a
I20260812 06:37:13.778270  8038 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be: loading cluster ID for follower catalog manager: success
I20260812 06:37:13.780385  8038 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be: acquiring CA information for follower catalog manager: success
I20260812 06:37:13.781495  8038 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 06:37:14.144098  8323 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:13.628976 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.190:37113 (local address 127.0.0.1:52938)
0812 06:37:13.629723 (+   747us) negotiation.cc:107] Waiting for socket to connect
0812 06:37:13.629738 (+    15us) client_negotiation.cc:175] Beginning negotiation
0812 06:37:13.629817 (+    79us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:37:14.143905 (+514088us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.190:37113: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":515,"thread_start_us":66,"threads_started":1}
W20260812 06:37:14.147913  8335 negotiation.cc:265] Injecting 6000ms delay in negotiation
W20260812 06:37:14.652019  8326 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:14.147000 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.190:37113 (local address 127.0.0.1:52940)
0812 06:37:14.147475 (+   475us) negotiation.cc:107] Waiting for socket to connect
0812 06:37:14.147492 (+    17us) client_negotiation.cc:175] Beginning negotiation
0812 06:37:14.147593 (+   101us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:37:14.651870 (+504277us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.190:37113: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":111,"thread_start_us":81,"threads_started":1}
I20260812 06:37:14.672009  8343 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0c41b4ed033d44009007ab44c3ec34af)
I20260812 06:37:14.672029  8346 raft_consensus.cc:493] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0c41b4ed033d44009007ab44c3ec34af)
I20260812 06:37:14.672145  8343 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } }
I20260812 06:37:14.672173  8346 raft_consensus.cc:515] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } }
I20260812 06:37:14.673079  8343 leader_election.cc:290] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0c41b4ed033d44009007ab44c3ec34af (127.3.193.190:37113), 75f2600fa64843ffa192098e4b00d8be (127.3.193.189:42351)
I20260812 06:37:14.673237  8346 leader_election.cc:290] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0c41b4ed033d44009007ab44c3ec34af (127.3.193.190:37113), 3f15fbab38e34ad981ef77a9d882bc08 (127.3.193.188:34287)
I20260812 06:37:14.673437  7751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3f15fbab38e34ad981ef77a9d882bc08" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "75f2600fa64843ffa192098e4b00d8be" is_pre_election: true
I20260812 06:37:14.673571  7751 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f15fbab38e34ad981ef77a9d882bc08 in term 1.
I20260812 06:37:14.673614  7888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75f2600fa64843ffa192098e4b00d8be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f15fbab38e34ad981ef77a9d882bc08" is_pre_election: true
I20260812 06:37:14.673776  7888 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75f2600fa64843ffa192098e4b00d8be in term 1.
I20260812 06:37:14.673961  7850 leader_election.cc:304] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f15fbab38e34ad981ef77a9d882bc08, 75f2600fa64843ffa192098e4b00d8be; no voters: 
I20260812 06:37:14.674136  7708 leader_election.cc:304] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f15fbab38e34ad981ef77a9d882bc08, 75f2600fa64843ffa192098e4b00d8be; no voters: 
I20260812 06:37:14.674192  8343 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 06:37:14.674248  8343 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0c41b4ed033d44009007ab44c3ec34af)
I20260812 06:37:14.674319  8343 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:37:14.674376  8346 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 06:37:14.674436  8346 raft_consensus.cc:493] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0c41b4ed033d44009007ab44c3ec34af)
I20260812 06:37:14.674484  8346 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:37:14.675484  8343 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } }
I20260812 06:37:14.675480  8346 raft_consensus.cc:515] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } }
I20260812 06:37:14.675887  8343 leader_election.cc:290] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [CANDIDATE]: Term 2 election: Requested vote from peers 0c41b4ed033d44009007ab44c3ec34af (127.3.193.190:37113), 75f2600fa64843ffa192098e4b00d8be (127.3.193.189:42351)
I20260812 06:37:14.676033  8346 leader_election.cc:290] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [CANDIDATE]: Term 2 election: Requested vote from peers 0c41b4ed033d44009007ab44c3ec34af (127.3.193.190:37113), 3f15fbab38e34ad981ef77a9d882bc08 (127.3.193.188:34287)
I20260812 06:37:14.676203  7751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3f15fbab38e34ad981ef77a9d882bc08" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "75f2600fa64843ffa192098e4b00d8be"
I20260812 06:37:14.676383  7751 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f15fbab38e34ad981ef77a9d882bc08 in current term 2: Already voted for candidate 75f2600fa64843ffa192098e4b00d8be in this term.
I20260812 06:37:14.676519  7888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75f2600fa64843ffa192098e4b00d8be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f15fbab38e34ad981ef77a9d882bc08"
I20260812 06:37:14.676683  7888 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 75f2600fa64843ffa192098e4b00d8be in current term 2: Already voted for candidate 3f15fbab38e34ad981ef77a9d882bc08 in this term.
W20260812 06:37:14.771396  7849 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.193.190:37113 timed out after 0.095s (SENT)
W20260812 06:37:14.848042  7709 leader_election.cc:336] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0c41b4ed033d44009007ab44c3ec34af (127.3.193.190:37113): Timed out: RequestConsensusVote RPC to 127.3.193.190:37113 timed out after 0.085s (SENT)
I20260812 06:37:14.848165  7709 leader_election.cc:304] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75f2600fa64843ffa192098e4b00d8be; no voters: 0c41b4ed033d44009007ab44c3ec34af, 3f15fbab38e34ad981ef77a9d882bc08
I20260812 06:37:14.848434  8346 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260812 06:37:14.854390  7709 leader_election.cc:336] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0c41b4ed033d44009007ab44c3ec34af (127.3.193.190:37113): Timed out: RequestConsensusVote RPC to 127.3.193.190:37113 timed out after 0.090s (SENT)
W20260812 06:37:14.868206  7849 leader_election.cc:336] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0c41b4ed033d44009007ab44c3ec34af (127.3.193.190:37113): Timed out: RequestConsensusVote RPC to 127.3.193.190:37113 timed out after 0.095s (SENT)
I20260812 06:37:14.868392  7849 leader_election.cc:304] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f15fbab38e34ad981ef77a9d882bc08; no voters: 0c41b4ed033d44009007ab44c3ec34af, 75f2600fa64843ffa192098e4b00d8be
I20260812 06:37:14.868750  8343 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260812 06:37:14.878183  7849 leader_election.cc:336] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0c41b4ed033d44009007ab44c3ec34af (127.3.193.190:37113): Timed out: RequestConsensusVote RPC to 127.3.193.190:37113 timed out after 0.102s (SENT)
I20260812 06:37:14.944182  8346 raft_consensus.cc:493] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:37:14.944357  8346 raft_consensus.cc:515] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } }
I20260812 06:37:14.944862  8346 leader_election.cc:290] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0c41b4ed033d44009007ab44c3ec34af (127.3.193.190:37113), 3f15fbab38e34ad981ef77a9d882bc08 (127.3.193.188:34287)
I20260812 06:37:14.945381  7888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75f2600fa64843ffa192098e4b00d8be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f15fbab38e34ad981ef77a9d882bc08" is_pre_election: true
I20260812 06:37:14.945566  7888 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75f2600fa64843ffa192098e4b00d8be in term 2.
I20260812 06:37:14.945958  7708 leader_election.cc:304] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f15fbab38e34ad981ef77a9d882bc08, 75f2600fa64843ffa192098e4b00d8be; no voters: 
I20260812 06:37:14.946170  8346 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 06:37:14.946233  8346 raft_consensus.cc:493] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:37:14.946281  8346 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:37:14.947217  8346 raft_consensus.cc:515] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } }
I20260812 06:37:14.947675  8346 leader_election.cc:290] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [CANDIDATE]: Term 3 election: Requested vote from peers 0c41b4ed033d44009007ab44c3ec34af (127.3.193.190:37113), 3f15fbab38e34ad981ef77a9d882bc08 (127.3.193.188:34287)
I20260812 06:37:14.948064  7888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75f2600fa64843ffa192098e4b00d8be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f15fbab38e34ad981ef77a9d882bc08"
I20260812 06:37:14.948197  7888 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:37:14.949059  7888 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f2600fa64843ffa192098e4b00d8be in term 3.
I20260812 06:37:14.949424  7708 leader_election.cc:304] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f15fbab38e34ad981ef77a9d882bc08, 75f2600fa64843ffa192098e4b00d8be; no voters: 
I20260812 06:37:14.949648  8346 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [term 3 FOLLOWER]: Leader election won for term 3
I20260812 06:37:14.949826  8346 raft_consensus.cc:697] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [term 3 LEADER]: Becoming Leader. State: Replica: 75f2600fa64843ffa192098e4b00d8be, State: Running, Role: LEADER
I20260812 06:37:14.950150  8346 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } }
I20260812 06:37:14.951275  8351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75f2600fa64843ffa192098e4b00d8be. Latest consensus state: current_term: 3 leader_uuid: "75f2600fa64843ffa192098e4b00d8be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } } }
I20260812 06:37:14.951395  8351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:14.951781  8353 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:37:14.952630  8353 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:37:14.953037  8353 catalog_manager.cc:1295] Loaded cluster ID: 2ece07cc6f734d6198a1227638763d0a
I20260812 06:37:14.953086  8353 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:37:14.953670  8353 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:37:14.954066  8353 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be: Loaded TSK: 0
I20260812 06:37:14.954685  8353 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:37:14.970774  7888 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [term 3 FOLLOWER]: Refusing update from remote peer 75f2600fa64843ffa192098e4b00d8be: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260812 06:37:14.971328  8351 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 06:37:14.972575  8343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75f2600fa64843ffa192098e4b00d8be. Latest consensus state: current_term: 3 leader_uuid: "75f2600fa64843ffa192098e4b00d8be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } } }
I20260812 06:37:14.972714  8343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:14.974062  8351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "75f2600fa64843ffa192098e4b00d8be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } } }
I20260812 06:37:14.974192  8351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:14.974421  8343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "75f2600fa64843ffa192098e4b00d8be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } } }
I20260812 06:37:14.974563  8343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:14.985090  8286 heartbeater.cc:499] Master 127.3.193.189:42351 was elected leader, sending a full tablet report...
W20260812 06:37:15.131877  7709 leader_election.cc:336] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0c41b4ed033d44009007ab44c3ec34af (127.3.193.190:37113): Timed out: RequestConsensusVote RPC to 127.3.193.190:37113 timed out after 0.093s (SENT)
W20260812 06:37:15.164022  8326 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:14.656866 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.190:37113 (local address 127.0.0.1:52948)
0812 06:37:14.657106 (+   240us) negotiation.cc:107] Waiting for socket to connect
0812 06:37:14.657124 (+    18us) client_negotiation.cc:175] Beginning negotiation
0812 06:37:14.657219 (+    95us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:37:15.163891 (+506672us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.193.190:37113: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":117}
I20260812 06:37:15.179967  3846 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu with pid 8045
I20260812 06:37:15.184930  7619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75f2600fa64843ffa192098e4b00d8be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0c41b4ed033d44009007ab44c3ec34af" is_pre_election: true
I20260812 06:37:15.185535  7619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75f2600fa64843ffa192098e4b00d8be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0c41b4ed033d44009007ab44c3ec34af" is_pre_election: true
I20260812 06:37:15.185771  7634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3f15fbab38e34ad981ef77a9d882bc08" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0c41b4ed033d44009007ab44c3ec34af" is_pre_election: true
I20260812 06:37:15.185817  7619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75f2600fa64843ffa192098e4b00d8be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0c41b4ed033d44009007ab44c3ec34af"
I20260812 06:37:15.186110  7629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75f2600fa64843ffa192098e4b00d8be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0c41b4ed033d44009007ab44c3ec34af"
I20260812 06:37:15.186218  7634 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [term 1 LEADER]: Stepping down as leader of term 1
I20260812 06:37:15.186285  7634 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0c41b4ed033d44009007ab44c3ec34af, State: Running, Role: LEADER
I20260812 06:37:15.186398  7615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75f2600fa64843ffa192098e4b00d8be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0c41b4ed033d44009007ab44c3ec34af"
I20260812 06:37:15.186443  7634 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } }
I20260812 06:37:15.186609  7634 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [term 1 FOLLOWER]: Advancing to term 3
I20260812 06:37:15.186733  7624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75f2600fa64843ffa192098e4b00d8be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0c41b4ed033d44009007ab44c3ec34af" is_pre_election: true
I20260812 06:37:15.187023  7627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75f2600fa64843ffa192098e4b00d8be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0c41b4ed033d44009007ab44c3ec34af"
I20260812 06:37:15.187340  7630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75f2600fa64843ffa192098e4b00d8be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0c41b4ed033d44009007ab44c3ec34af" is_pre_election: true
I20260812 06:37:15.187674  7632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3f15fbab38e34ad981ef77a9d882bc08" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0c41b4ed033d44009007ab44c3ec34af"
I20260812 06:37:15.187948  7634 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [term 3 FOLLOWER]: Refusing update from remote peer 75f2600fa64843ffa192098e4b00d8be: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260812 06:37:15.188234  7633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3f15fbab38e34ad981ef77a9d882bc08" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0c41b4ed033d44009007ab44c3ec34af"
I20260812 06:37:15.188345  7633 raft_consensus.cc:2368] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f15fbab38e34ad981ef77a9d882bc08 for earlier term 2. Current term is 3.
I20260812 06:37:15.188452  7631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3f15fbab38e34ad981ef77a9d882bc08" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0c41b4ed033d44009007ab44c3ec34af" is_pre_election: true
I20260812 06:37:15.188606  7631 raft_consensus.cc:2368] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f15fbab38e34ad981ef77a9d882bc08 for earlier term 2. Current term is 3.
I20260812 06:37:15.190078  8351 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260812 06:37:15.190778  8374 negotiation.cc:265] Injecting 6000ms delay in negotiation
W20260812 06:37:15.190915  8371 negotiation.cc:265] Injecting 6000ms delay in negotiation
I20260812 06:37:15.191761  7888 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 3f15fbab38e34ad981ef77a9d882bc08 [term 3 FOLLOWER]: Rejecting Update request from peer 0c41b4ed033d44009007ab44c3ec34af for earlier term 1. Current term is 3. Ops: []
I20260812 06:37:15.192849  8324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75f2600fa64843ffa192098e4b00d8be. Latest consensus state: current_term: 3 leader_uuid: "75f2600fa64843ffa192098e4b00d8be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } } }
I20260812 06:37:15.192950  8324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:15.193048  8324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "75f2600fa64843ffa192098e4b00d8be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } } }
I20260812 06:37:15.193123  8324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c41b4ed033d44009007ab44c3ec34af [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:15.196185  7751 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [term 3 LEADER]: Rejecting Update request from peer 0c41b4ed033d44009007ab44c3ec34af for earlier term 1. Current term is 3. Ops: []
W20260812 06:37:15.199955  7589 connection.cc:573] server connection from 127.3.193.129:42085 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 06:37:15.200481  3846 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu with pid 7557
I20260812 06:37:15.210333  8346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "75f2600fa64843ffa192098e4b00d8be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c41b4ed033d44009007ab44c3ec34af" member_type: VOTER last_known_addr { host: "127.3.193.190" port: 37113 } } peers { permanent_uuid: "75f2600fa64843ffa192098e4b00d8be" member_type: VOTER last_known_addr { host: "127.3.193.189" port: 42351 } } peers { permanent_uuid: "3f15fbab38e34ad981ef77a9d882bc08" member_type: VOTER last_known_addr { host: "127.3.193.188" port: 34287 } } }
I20260812 06:37:15.210525  8346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75f2600fa64843ffa192098e4b00d8be [sys.catalog]: This master's current role is: LEADER
W20260812 06:37:15.214572  8326 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:15.170837 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.193.190:37113 (local address 127.0.0.1:52960)
0812 06:37:15.171049 (+   212us) negotiation.cc:107] Waiting for socket to connect
0812 06:37:15.171066 (+    17us) client_negotiation.cc:175] Beginning negotiation
0812 06:37:15.171158 (+    92us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:37:15.214421 (+ 43263us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.193.190:37113: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":98}
W20260812 06:37:15.214603  7709 connection.cc:573] client connection to 127.3.193.190:37113 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 06:37:15.215039  3846 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu with pid 7672
I20260812 06:37:15.229988  3846 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskNlrYvv/build/debug/bin/kudu with pid 7810
2026-08-12T06:37:15Z chronyd exiting
[       OK ] ClientFailoverOnNegotiationTimeoutITest.Kudu2021NegotiateWithMaster (3129 ms)
[----------] 4 tests from ClientFailoverOnNegotiationTimeoutITest (42989 ms total)

[----------] Global test environment tear-down
[==========] 4 tests from 1 test suite ran. (42989 ms total)
[  PASSED  ] 4 tests.
I20260812 06:37:15.260177  3846 logging.cc:436] LogThrottler /workspace/apache/dev/local/kudu/src/kudu/client/meta_cache.cc:302: suppressed but not reported on 1 messages since previous log ~8 seconds ago
