Skip to content

My notes for Komodo DeFi API

Jump to bottom
dimxy edited this page Dec 19, 2024 · 39 revisions

Building Trezor emulator hints

How to build and user Trezor emulator (Note that I tried only legacy emulator "One" version):

  • Install Poetry and always work from Poetry shell to build emulator (start with poetry shell)
  • cmd to build legacy emulator on MacOs: make build_unix in the trezor-firmware/legacy dir
  • the executable is trezor-firmware/legacy/firmware/trezor.elf (no params)
  • Note that Emulator requires window GUI (can't run on VPC). Update: you can set env SDL_VIDEODRIVER=dummy and run it nointeractive
  • to configure Emulator - create a wallet you need to use Trezor Suite (web or GUI). Trezor Suite needs the bridge which can be downloaded from trezor.io (as dmg for macos. I installed it and the bridge started automatically. I could not connect to the emilator when the bridge was autostarted with no params, so I copied its exe from the install directory and removed the app on MacOs). I ran bridge as ./trezord -e 21324. Note that you need to run trezord before Trezor Suite (if it is desktop version) as it loads its own version which cannot connect to legacy trezor for some reason (wrong -e default param possibly)
  • When start the Trezor Suite please wait for some time to let it find the trezor emulator.
  • Note that rebuilding emulator overrides emulator flash file emulator.img (with the wallet setting where your wallet seed is stored), so copy it before rebuilding
  • When configure the wallet do not set PIN (not supported in adex yet)
  • Perform wallet backup when you are recovering it after the emulator.img was lost - emulator will show on its screen wallet words - write them down
  • Before running a komodo-defi test for emulator, stop the Trezor Suite. Also be aware emulator may hang sometimes and needs to be restarted.

BTW there is trezor-user-env project which provides a web interface to start all emulators version. It requires Xquartz lib installed. But trezor-user-env is basically not needed, you can run emulator without it just from cmd line.

Yet another concise instruction to build and use Trezor Emulator:
https://github.com/MetaMask/metamask-extension/blob/develop/docs/trezor-emulator.md

How to dump privkey from bitcoin for tests

Note dumpprivkey is supported only for legacy wallets:

./bitcoin-cli -testnet help dumpprivkey 
dumpprivkey "address"

Reveals the private key corresponding to 'address'.
Then the importprivkey can be used with this output
Note: This command is only compatible with legacy wallets.

To run bitcoin testnet node isolated

Stop bitcoind and delete peer files:
~/.bitcoin/testnet3/peers.dat
~/.bitcoin/testnet3/anchors.dat
Run command: ./bitcoind -testnet -port=7777 -dnsseed=0
To clear mempool stop bitcoind and delete ~/.bitcoin/testnet3/mempool.dat

Which coin config fields are responsible for Segwit

We have segwit and non-segwit utxo coins' configurations in the 'coins' configuration file. F.e. we have BTC and BTC-segwit coin configs for the same blockchain BTC. For users BTC or BTC-segwit means that their address in the wallet is either standard ('1...') or native segwit ('bc1..'). For that this coins param is responsible "address_format": {"format": "segwit"}: when a coin is activated the address format in its activation param is checked against the address_format in the coin config, so it would not allow to activate a segwit coin with non-segwit address format.

There is another param in the coin config "segwit" param which actually tells to the code that the segwit feature is enabled on this network. So if "segwit" is set to false for some coin then the API code won't allow to send value to segwit addresses for that coin.

If the HD wallet API feature is used then derivation_path should be set in the coins config which is used to generate user addresses based on BIP32 schema. For non-segwit utxo coins BIP44 note is used to for derivation_path which should have "m/44'" as the purpose field in it. For segwit coins BIP84 is used and derivation_path starts with "m/84'". That means that for segwit coins we must ensure the correct 84 purpose field in the config, otherwise accounts will be created differently from other apps, including trezor firmware, which is not good.
I created an issue that would prevent from mistakes when derivation_path in coins file is set incorrectly to m/44' for segwit coins: https://github.com/KomodoPlatform/komodo-defi-framework/issues/1995

How key policy is created on coin init:

There are 2 options for coin initialisation in Komodo DeFi API: legacy and a new one. We will use the standalone coin as an example of both paths:

Before any coin addition the crypto_ctx in MmCtx is initialised at mm2 startup in lp_init fn. Param "passphrase" must be set in ctx.conf for crypto_ctx init. The crypto_ctx is initialised depending on "enable_hd" ctx.conf param, either as KeyPairPolicy::Iguana or GlobalHDAccount (see CryptoCtx::init_with_global_hd_account and CryptoCtx::init_with_iguana_passphrase fn). So currently MmCtx::crypto_ctx is always initialised with a keypair (Note the Trezor context cannot be initialised at mm2 startup).

In the legacy init option each coin can be initialised and added by "enable" or "electrum" calls. In this case we still cannot add Trezor policy as the legacy case does not allow user interaction needed for hardware wallet. In legacy case lp_coininit fn is called which gets PrivKeyBuildPolicy enum from MmCtx and calls utxo_standard_coin_with_policy fn to build the coin with UtxoArcBuilder::new().build() object. Note that priv_key_policy field of PrivKeyActivationPolicy type in the UtxoActivationParams is ignored here.

The new init option is spawned in a rpc task manager thread: the InitStandaloneCoinTask run fn begins with init_standalone_coin fn then the result is obtained with get_activation_result fn. The init_standalone_coin fn from init_utxo_standard_activation.rs gets PrivKeyBuildPolicy enum from the priv_key_policy field of PrivKeyActivationPolicy type in the UtxoActivationParams, and now it can be already Trezor policy. Then the same UtxoArcBuilder::new().build() object is used here too to create and add the coin object.

BTW maybe there is a bug in the new init option:
For the new init option the initialisation result should be obtained with get_activation_result fn, which calls enable_coin_balance fn, which in turn calls enable_hd_wallet fn for the DerivationMethod::HDWallet path. This enable_hd_wallet fn, if the wallet accounts list is empty, creates a new account with coin.create_new_account fn. This coin.create_new_account creates a new account with INIT_ACCOUNT_ID = 0 and calls coin.extract_extended_pubkey fn which always extracts a pubkey for trezor coin or throws CoinDoesntSupportTrezor error (impl in utxo_common::extract_extended_pubkey fn). So it looks like the new init option will work only from trezor wallet (and would fail if we call it with PrivKeyActivationPolicy::ContextPrivKey (meaning Iguana or HdWallet).
Update: fixed in evm-hd-wallet branch, check the new version of extract_extended_pubkey fn in lp_coins.rs there

MM2 internal private key

Note that there is an internal privkey used in p2p (swap) exchanges and generated for Komodo coin. It is always generated even if a hardware wallet is initialized. (TODO: use hardware wallet key instead). See hw_ctx.rmd160() call.

Do proper merge commit into feature branch

Recently we agreed not to rebase my development feature branch on the dev branch but to do merge 'dev' instead. This is done to more easily see changes that I did when added recent 'dev' changes. First try was not successful: when I did merge dev to my branch (feature-add-script-type-to-address) in SourceTree and fixed changes I reloaded the SourceTree app and when I was about to create the merge commit SourceTree did not offer the commit message and I had to create my own one. I think this led to an improper (unrecognisable?) merge commit created. As a result when I tried to do a pull request of my branch to the dev branch I saw red label 'could not automatically merge'.
When I tired to merge again and got a merge commit message auto-created by SourceTree (git) and again made a pull request the PR label was green already. Plus after creation the second PR I clicked the merge commit and Github showed me a compact 'condensed version' of this commit with only my added changes to the huge amount of recent code pulled from 'dev' branch. I had also an option to see 'full version' of the merge commit.
In the first case (with the merge commit created myself) I did not see the condensed version but only full one.
I guess it is important to create a proper merge commit with a git auto-created commit message?

But it looks like the cause was not the commit message but in that github could not identify the commit as 'merge'. Apparently a commit is a 'merge' when it has more than one parent (https://stackoverflow.com/questions/74478550/is-there-a-way-to-tell-if-a-git-commit-is-a-merge). Indeed, for the first myself created 'merge-commit' it was only one parent:

git show --pretty=%ph --quiet a705919
e5283c317h

But for the second case it was two parents (merge commit indeed):

git show --pretty=%ph --quiet 123e3f8
e5283c317 fc95ef3edh

How could I have lost the parent in the first try? Need to pay attention to the merge commit next time

How Trezor invokes pin

All eth and btc calls to the trezor device is protected with pin, including getting address or pubkey calls (see CHECK_PIN in the firmware code). The device auto locks its storage with default timeout of 10 min and unlocks it by requesting the pin. Few calls like fsm_msgTxAck and fsm_msgEthereumTxAck do not ask pin but check that storage is unlocked (see CHECK_UNLOCKED)

Hardened and non-hardened derivation

Just to memorise how it works.
Here is a sample derivation path: "m/44'/0'/0'/0/0".
The hardened path is path_to_coin: "m/44'/0'/0'" that is derivation from privkey is used to this level.
But address_to_account part is non-hardened (".../0/0") so derivation from pubkey is used to generate accounts.

fix cargo error failed to select a version for wasmparser.

I got an error:

failed to select a version for `wasmparser`.
    ... required by package `wasm-bindgen-cli v0.2.95 (/Users/dimxy/repo/wasm-bindgen/crates/cli)`
versions that meet the requirements `^0.211` are: 0.211.1, 0.211.0

the package `wasm-bindgen-cli` depends on `wasmparser`, with features: `indexmap` but `wasmparser` does not have these features.
 It has an optional dependency with that name, but that dependency uses the "dep:" syntax in the features table, so it does not have an implicit feature with that name.

Fixed by enabling feature 'validate' (which turns on optional dependency 'indexmap') wherever wasmparser is added in Cargo.toml: wasmparser = { version = "0.212", features = ['validate'] }

How to enable logging in MM2 cargo tests

add env_logger init with common::log::UnifiedLoggerBuilder::default().init();

Info how to run zcoin native tests

Added notes in this file

Info about private coins and transactions (relevant for Sapling update)

Some info about private coins ['zcoin' in the kdf]:

  • z_key, z_spending_key - secret key to sign private transactions shielded spends and sign binding signatures
  • Note - like utxo
  • SpendDescription - like utxo tx input
  • Nullifier - part of SpendDescription, nullifies a previous tx note
  • fvk - full viewing key ...
  • viewing_key - key to encrypt/decrypt private transaction's outputs' part where tx's amount and address are encrypted (to inform the receiver)
  • incoming
  • binding signature - signs private tx balance (visible) TODO...

How to use komodefi-wasm-rpc

It needs the path to chrome browser - can't configure, fixed in code:

-    executablePath: '/usr/bin/google-chrome',
+    executablePath: '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',

Build wasm target in atomicDEX (KDF) repo: CC=/usr/local/opt/llvm/bin/clang AR=/usr/local/opt/llvm/bin/llvm-ar wasm-pack build mm2src/mm2_bin_lib --target web --out-dir wasm_build/deps/pkg/

Create wasm zip (recursively):

cd ~/repo/atomicDEX-API/mm2src/mm2_bin_lib/wasm_build/deps/pkg
zip -r kdf_wasm.zip # (better to omit .gitgnore package.json though)

copy kdf_wasm.zip to komodefi-wasm-rpc installation:

cp ~/repo/atomicDEX-API/mm2src/mm2_bin_lib/wasm_build/deps/pkg/kdf_wasm3.zip .

update kdf files in komodefi-wasm-rpc:

./update_wasm_local.sh  /Users/dimxy/repo/komodefi-wasm-rpc/kdf_wasm.zip. # use full path with kdf_wasm.zip

It should finish without copy or other errors.

Then start two modules:

yarn preview
node server.cjs

To call KDF rpc fix the 'rpc' and 'userpass' source and test bash-script using them: add 'endpoint=rpc' var to the rpc files. add the 'endpoint' to the KDF url in bash-scripts fix password in the userpass file

Clone this wiki locally