main.go
. It's better to start tracing from the backend repo's main rather than the core repo's main, since the core repo is mainly intended to be used as a library. Moreover, since backend uses the core repo as a library, we will hit all of the core functionality by starting here anyway.Run()
function defined in the cmd package.init()
function. Some of these flags are initialized in LoadConfig()
at the beginning of Run()
.Run()
function, everything the node does can be traced explicitly. We will be walking through some of the key codepaths belowbitclout-seed-*.io
to see if any valid peers are available. The function that does that is addSeedAddrsFromPrefixes()
and the list of "prefixes" that are scanned is defined in constants.go
.--connect-ips
means a peer will connect to the specified peer and nothing else. --add-ips
means these peers will be added to the list of things that the peer is going to try and connect to. When we spin up new nodes, we often use --connect-ips
with a trusted node because it's easier than bootstrapping from the sea of nodes that are running in the wider internet.Start()
function that is kicked off in main.go. Tracing the code starting from this function is a great way to understand how connections with peers are established and maintained.ConnectPeer()
. If the peer passes this version negotiation, then the peer is passed off to server.go
via a "newPeerChan." server.go is then responsible for doing higher-level interactions with the peer.server.go
is started using a Start()
, which is a good place to start tracing through it. server.go
can be thought of as the "main loop" for the node. It is basically a single for{} loop that all peers and services are adding messages to. See messageHandler()
to see this "main loop" in action. MsgBitCloutNewPeer
control message from the ConnectionManager and handles it in _handleNewPeer()
. This is typically the "starting point" for server.go_startSync()
, and it will send it a GetHeaders or MsgBitCloutGetHeaders
message to start syncing headers and blocks from it.MsgBitCloutNewPeer
message to server.go, which is processed in messageHandler()
.MsgBitCloutGetHeaders
with a MsgBitCloutHeaderBundle
in _handleGetHeaders()
.MsgBitCloutHeaderBundle
at _handleHeaderBundle()
and responds with different messages depending on how synced the peer is.MsgBitCloutGetHeaders
. Note that headers are requested until the number of headers in the latest HeaderBundle is < MaxHeadersPerMsg
. This is how the node knows that it's downloaded all the headers the remote peer has for it.MsgBitCloutGetBlocks
message is sent in GetBlocks()
.ProcessBlock()
, which is a great function to trace through. It calls ConnectBlock()
, which calls ConnectTransaction on each transaction, which we'll discuss later.INV
messages or MsgBitCloutInv
are processed via messageHandler()
just like everything else. INV
messages can be for a block, as mentioned previously OR for a transaction. Below is the case for a transaction INV
:_handlePeerMessages()
function will just enqueue the message for the peer's thread to process it. This is done in order to move processing into another thread for efficiency reasons (not doing this would cause server.go to be *too* single-threaded). Here you can see the _handleInv()
in server.go
delegate the call to peer.go, and here you see peer.go dequeuing it to process it. Note that there are several messages that are delegated in this way, all defined in the StartBitCloutMessageProcessor()
function.MsgBitCloutGetTransactions
message to the peer._handleGetTransactions()
function in server.go, which results in a TransactionBundle
or MsgBitCloutTransactionBundle
being sent back._processTransactions()
in server.go
.server.go
, it is basically just calling processTransaction()
in mempool.go
. If the transaction is valid then it will be added to the mempool, and if not then it will be rejected. In order to validate a transaction, mempool uses the previously mentioned ConnectTransaction()
function defined in block_view.go
.block_producer.go
runs in a continuous loop kicked off via a Start()
function called in main.go. Start()
calls UpdateLatestBlockTemplate()
at regular intervals to create new blocks for miners to mine. This is a great function to trace._getBlockTemplate()
contains the logic for constructing a new block. It basically does the following:block_producer.go
just produces block templates, but it's up to miners to compute winning hashes. That happens via a remote process as follows:GetBlockTemplate()
and SubmitBlock()
. The URL paths for these and all other API functions can be seen here and here (the latter powers the block explorer). remote_miner_main.go
and connect to any node they want via a flag. This can be their own local node or a remote node like api.bitclout.com. remote_miner_main.go
will continuously call GetBlockTemplate()
on the chosen node and hash it until it's found a block. Once it has found a winning hash, it calls SubmitBlock()
, which then causes the node to process it and broadcast it to the rest of the network.get-block-template
, all nodes can be used to mine blocks in this way. Miners generally don't need to do anything other than point to a valid BitClout node somewhere on the network.GetBlockTemplate()
much faster because it won't have to copy a block.ProcessBlock()
notifies core's server.go
that a block was connected by calling _handleBlockMainChainConnectedd()
_handleBlockMainChainConnectedd()
updates the mempool using UpdateAfterConnectBlock()
, which removes transactions from the mempool that have been mined into the blockProcessBlock()
notifies server.go
again by enqueing a MsgBitCloutBlockAccepted
message at the end, triggering a call to _handleBlockAccepted
.INV
for the new block that gets relayed to all the peers who will then request it from this node.GetHeaders
and kicks off a single-threaded main loop with its peer here. ConnectTransaction()
in either ProcessBlock()
or processTransaction()
. A BitcoinExchange transaction is only valid if it has a merkle proof attached to it that has a valid Bitcoin header hash as its root. More on this later.messageHandler
. This happens here.encryptedSeedHex
. Again, this only happens if the node has the FULL access level.encryptedSeedHex
, and then sends it back to bitclout.com. This same process works if you replace bitclout.com with the host of your own third-party node. The difference is that your third-party node will need to ask the user for permission in order to get encryptedSeedHex sent back to it.encryptedSeedHex
, it uses it to sign things. It does this by calling various operations on an iframe of identity.bitclout.com embedded within it.signAndSubmitTransaction
, which sends it to the identity.bitclout.com iframe via a postMessage to call doSign().encryptedSeedHex
and the transaction hex.encryptedSeedHex
with the host-specific encryption key, signs the transaction, and returns the signed transaction back. This all happens here.encryptedSeedHex
back to the host? Wouldn’t it be better to just keep everything in identity.bitclout.com?localStorage
when it’s embedded as an iframe
in bitclout.com. This is due to Apple’s crusade against third-party cookies. However, Apple does allow identity.bitclout.com to access its cookies when its embedded as an iframe on bitclout.com if those cookies are set as first-party cookies.encryptedSeedHex
. When signing is needed, the encryptedSeedHex
is passed to the identity.bitclout.com iframe, which has access to the encryption key in the cookie, which it then uses to decrypt the encryptedSeedHex
and sign the transaction.localStorage
. This means that only iOS devices are subject to this drawback.encryptedSeedHex
. However, this information is useless without the encryption key stored exclusively in identity.bitclout.com.frontend_server.go
is called to construct a transaction. That transaction is then returned unsigned, signed by the identity iframe, and then submitted back to core via SubmitTransaction()
.AddInputsAndChangeToTransaction()
, which is a good function to trace through. I'm not aware of any transaction assembly that does not utilize this function for UTXO fetching.GetSpendableUtxosForPublicKey()
, which generates a universal view that includes txns from the mempool and then returns all UTXO's that are associated with the particular public key. These UTXO's can then be assembled into a transaction.BroadcastTransaction()
, which calls _addNewTxn()
in server.go, which adds the transaction to the mempool calling ProcessTransaction()
.Start()
through _startTransactionRelayer()
.MinerBitcoinMinBurnWorkBlocks
that is utilized by the block producer.MsgBitCloutTxn
and MsgBitCloutBlock
, are defined in network.go
. They all implement the very simple BitCloutMessage
interface.TxnMeta
. More on this laterPublicKey
. In BitClout transactions are very simple and only have one public key that can be deemed to be the "executor" of the transaction. The transaction is generally always signed by this public key.ExtraData
. This is a flexible map that arbitrary data can be added to. It is currently used to support Reclouts via RecloutedPostHash
and IsQuoteReclouted
params. It can be used to augment a transaction without causing a hard fork, which significantly increases the extensibility of BitClout by the community. For example, one can trivially add a "pinned posts" feature using ExtraData
without consulting the core BitClout devs about it.ExtraData
are always sorted when serialized so that consistent serialization across machines is preserved even though we're using a map.BitCloutTxnMetadata
interface. The full list of transaction types can be viewed here. To see descriptions of each one, simply find where that transaction type implements the interface.BitcoinExchangeMetadata
. You can see it contains a full Bitcoin transaction plus a merkle proof into the Bitcoin blockchain. This is how a node verifies that a particular Bitcoin transaction has a sufficient amount of work on it.ConnectBlock
does._setProfileEntryMappings
and _getProfileEntryForUsername
.sDeleted=true
and then writing entries that have isDeleted=false.ConnectBlock()
and then flushes it. The mempool uses a view to validate transactions as well inside of its core processTransaction
function._connectBasicTransfer
, which returns the total input and output of the transaction.isDeleted=true
. This is because the flush needs to propagate this change to the db, and it can only do that if it knows the entry is scheduled to be deleted by leaving it in the view._connect
and a _disconnect
. The _disconnect
restores the view to the state it was in before the transaction was connected. _disconnect
code is rarely used, but it supports reorgs of blocks, which happen from time to time.