How to get BIP32 from 12.2 QT with -usehd

camosoul

Grizzled Member
Sep 19, 2014
2,261
1,130
1,183
Yes, I have read this warning and understand (and have compensated for) the ramifications of using the -usehd feature on 12.2
Code:
cat ~/dashmnb/dashlib/config.py
...
default_receiving_address = 'putbip32craphere'
...
Instructions for extracting this data from trezor/keepkey exist in ~/dashmnb/sendto_bip32_path.md
Code:
trezorctl get_public_node -n "44'/5'/1'/0"
But...

How does one extract this from a -usehd enabled 12.2 QT?

@UdjinM6 @flare @chaeplin

??

Thanks in advance...
 

chaeplin

Active Member
Core Developer
Mar 29, 2014
749
356
133
@camosoul

if a wallet uses `usehd`, `dash-cli dumphdinfo` will show
{
"hdseed": "db39ebf414fa9631b1966850cfc49d2ab02661e5459411bb0fe830e0ae0b1e2069fec0089e37410171095e4c8eeae78a0dda95e94d91c579cc7b045baf6dcb69",
"mnemonic": "kingdom another invite curtain deal humor rail swamp sound faculty inflict lesson pulp ring aerobic answer gold blood gym attitude false menu couch park",
"mnemonicpassphrase": ""
}
Or a wallet can be started usehd + mnemonic + mnemonicpassphrase.

Use `mnemonic` to get `BIP32 Extended Public Key`.

https://iancoleman.io/bip39/ ( offline version is here https://github.com/iancoleman/bip39 )

example
mnemonic : kingdom another invite curtain deal humor rail swamp sound faculty inflict lesson pulp ring aerobic answer gold blood gym attitude false menu couch park

https://iancoleman.io/bip39/
BIP39 Mnemonic --> kingdom another invite curtain deal humor rail swamp sound faculty inflict lesson pulp ring aerobic answer gold blood gym attitude false menu couch park
Coin : Dash

BIP32 Extended Public Key : xpub6DkddQ11Eb83wEk4gBsqtBKEiTruTsVBe7zMk3dNmpwx9MofDEEButQ5yg5Yv58kr68x9fQSLN7K6KoFiC432s57AtcGf6MgsSeuEmWLDmC

`BIP32 Extended Public Key` can be used on dashmnb

Select address on "m/44'/5'/0'/0/0" : XoCQvNDZg2DBTsnrks1EG2kTrBevenCPkv

Use Dashcore to validate things ok.

dash-cli validateaddress XoCQvNDZg2DBTsnrks1EG2kTrBevenCPkv
 
  • Like
Reactions: UdjinM6

camosoul

Grizzled Member
Sep 19, 2014
2,261
1,130
1,183
put mnemonic into bip39 Mnemonic field of bip39-standalone.html

watch it do magical things

observe/verify that bip39 seed field matches "hdseed"

click on BIP44 tab

default_receiving_address = 'BIP32 Extended Public Key'

observe first derived address and assure that it matches the bip32 check performed during "dashmnb -x" before approving any sends
 
Last edited:

camosoul

Grizzled Member
Sep 19, 2014
2,261
1,130
1,183
Hrm, I must be doing something wrong. It's not working.

The addresses match up, but the client doesn't actually have any of the predicted addresses... Is it not actually using the seed that dumphdinfo presented? The client shows HD is in use...
 

camosoul

Grizzled Member
Sep 19, 2014
2,261
1,130
1,183
Found the problem...

The client does not select the predicted addresses in a linear fashion.

since I have keypool=250000

When I ask for a new address, it picks one at random from /0 to /249999

So, I had to ask for a reallyyy long list of predicted addresses before it showed up...

The client only looks for addresses in the memory pool if it has been exposed to the user.

So, payments sent to the xpub chain in a linear manner don't show up in the balance because the exposed addresses are /2895 /23671 and /122909 are nowhere near the /1 /2 or /3 where the VINs occurred.

I suppose this would need a keypool=0 to keep it from doing this? Did I create an unexpected circumstance by using a large keypool? Did someone forget that the addresses are selected at random thus excluding payments from being seen in the balance which occurred elsewhere in the predictive chain?

So...

I think keypool=0 is a sloppy workaround.

Maybe HD wallets should ignore the keypool variable? It seems to generate keys in a way that breaks the functionality, even if technically proper.

The real fix is to have the client pick/search the VINs linearly... Or maybe define the client's "max gab" as the depth of the keypool... Or have no keypool. That could get hairy... But, I don't see a way around it if you expect the balance to be real.

If you have no keypool, how do you define the depth of the scan?

Seems keeping it linear is the only way to avoid an infinity scan.

It seems like HD wallets shouldn't have a keypool potential. It seems the point of HD is to not need a keypool... But, how do you determine how deep it scans? The entity sending to the xpub chain is selecting addresses in a different way than the entity scanning it. The only way to be sure seems to be demanding linearity as part of the standard, but that clearly isn't happening.

Examining wallet.dat shows that the addresses were generated in linear compliance with the sequence. But, the keypool selection process is random.

I guess you could fix that several ways, the question is the "most appropriate" fix... I guess keypool=0 works for now.
 
Last edited:

chaeplin

Active Member
Core Developer
Mar 29, 2014
749
356
133
@camosoul

Use `getwalletinfo` and `validateaddress` to check

I have 20K keypool on testnet, and will try 250K(!!) keypool.

example with 1K keypool.

[email protected]:~ $ dash-cli getwalletinfo

{

"walletversion": 120200,

"balance": 0.76536056,

"unconfirmed_balance": 0.00000000,

"immature_balance": 0.00000000,

"txcount": 1220,

"keypoololdest": 1508901799,

"keypoolsize": 1000,

"keypoolsize_hd_internal": 999,

"keys_left": 999,

"paytxfee": 0.00000000,

"hdchainid": "e9c3b0aa46cd67bbc0ebb868b96ea8c014d33befd641cf7f770c37ca28918a60",

"hdaccountcount": 1,

"hdaccounts": [

{

"hdaccountindex": 0,

"hdexternalkeyindex": 6132,

"hdinternalkeyindex": 2215

}

]

}

[email protected]:~ $ dash-cli getnewaddress

yUAfFiVhkL7FHpvQSZQ7GWVV2PwGoRQwhZ

[email protected]:~ $ dash-cli getnewaddress

yeatqMZhpnECKwLhEbFfStgPEirC6gkG4n

[email protected]:~ $ dash-cli getnewaddress

yaDL3HDLPCRY4XoBgQ6vPHsZjvNrqihN9j

[email protected]:~ $ dash-cli validateaddress yUAfFiVhkL7FHpvQSZQ7GWVV2PwGoRQwhZ

{

"isvalid": true,

"address": "yUAfFiVhkL7FHpvQSZQ7GWVV2PwGoRQwhZ",

"scriptPubKey": "76a91456179e3d2323a3f88a472cc5055d3ca0e4707de088ac",

"ismine": true,

"iswatchonly": false,

"isscript": false,

"pubkey": "02119bbc6aeee2cdde74800417085e4bb94bf0185f1a0580bb22754afbb96584fe",

"iscompressed": true,

"account": "",

"hdkeypath": "m/44'/1'/0'/0/5132",

"hdchainid": "e9c3b0aa46cd67bbc0ebb868b96ea8c014d33befd641cf7f770c37ca28918a60"

}

[email protected]:~ $ dash-cli validateaddress yeatqMZhpnECKwLhEbFfStgPEirC6gkG4n

{

"isvalid": true,

"address": "yeatqMZhpnECKwLhEbFfStgPEirC6gkG4n",

"scriptPubKey": "76a914c85e31ee3e70046677c0f6745582e7d25602686988ac",

"ismine": true,

"iswatchonly": false,

"isscript": false,

"pubkey": "02b437b556cf4c7606d340448250c259cc39f5374f017fd69818cdbe3f55abc9ce",

"iscompressed": true,

"account": "",

"hdkeypath": "m/44'/1'/0'/0/5133",

"hdchainid": "e9c3b0aa46cd67bbc0ebb868b96ea8c014d33befd641cf7f770c37ca28918a60"

}

[email protected]:~ $ dash-cli validateaddress yaDL3HDLPCRY4XoBgQ6vPHsZjvNrqihN9j

{

"isvalid": true,

"address": "yaDL3HDLPCRY4XoBgQ6vPHsZjvNrqihN9j",

"scriptPubKey": "76a914986994d1b64f435cd24d6dc821974be688e3709488ac",

"ismine": true,

"iswatchonly": false,

"isscript": false,

"pubkey": "0372d090868575d882e269ae48b04cfd4babdf1c8088dcb97493232ea21ebafe10",

"iscompressed": true,

"account": "",

"hdkeypath": "m/44'/1'/0'/0/5134",

"hdchainid": "e9c3b0aa46cd67bbc0ebb868b96ea8c014d33befd641cf7f770c37ca28918a60"

}

Found the problem...

The client does not select the predicted addresses in a linear fashion.

since I have keypool=250000

When I ask for a new address, it picks one at random from /0 to /249999

So, I had to ask for a reallyyy long list of predicted addresses before it showed up...

The client only looks for addresses in the memory pool if it has been exposed to the user.

So, payments sent to the xpub chain in a linear manner don't show up in the balance because the exposed addresses are /2895 /23671 and /122909 are nowhere near the /1 /2 or /3 where the VINs occurred.

I suppose this would need a keypool=0 to keep it from doing this? Did I create an unexpected circumstance by using a large keypool? Did someone forget that the addresses are selected at random thus excluding payments from being seen in the balance which occurred elsewhere in the predictive chain?

So...

I think keypool=0 is a sloppy workaround.

Maybe HD wallets should ignore the keypool variable? It seems to generate keys in a way that breaks the functionality, even if technically proper.

The real fix is to have the client pick/search the VINs linearly... Or maybe define the client's "max gab" as the depth of the keypool... Or have no keypool. That could get hairy... But, I don't see a way around it if you expect the balance to be real.

If you have no keypool, how do you define the depth of the scan?

Seems keeping it linear is the only way to avoid an infinity scan.

It seems like HD wallets shouldn't have a keypool potential. It seems the point of HD is to not need a keypool... But, how do you determine how deep it scans? The entity sending to the xpub chain is selecting addresses in a different way than the entity scanning it. The only way to be sure seems to be demanding linearity as part of the standard, but that clearly isn't happening.
 
Last edited:

chaeplin

Active Member
Core Developer
Mar 29, 2014
749
356
133
@camosoul
tested, it's linear.

Code:
[email protected]:~ $ dash-cli keypoolrefill 250000
[email protected]:~ $
[email protected]:~ $ dash-cli getwalletinfo
{
  "walletversion": 120200,
  "balance": 0.00000000,
  "unconfirmed_balance": 0.00000000,
  "immature_balance": 0.00000000,
  "txcount": 0,
  "keypoololdest": 1511074654,
  "keypoolsize": 250000,
  "keypoolsize_hd_internal": 250000,
  "keys_left": 999,
  "paytxfee": 0.00000000,
  "hdchainid": "32d9544499010197fa41dad8decfbd98bbef9f19b9fe22e98654fe3ef091f23f",
  "hdaccountcount": 1,
  "hdaccounts": [
    {
      "hdaccountindex": 0,
      "hdexternalkeyindex": 250001,
      "hdinternalkeyindex": 250000
    }
  ]
}
[email protected]:~ $ dash-cli getnewaddress
yUyQ5gdwW9eY2rRzTFGaAgUdvXEsRq1NLM
[email protected]:~ $ dash-cli getnewaddress
ydg8fFWB1UKvdDoSGuqh8sAUgZzyEbAxFB
[email protected]:~ $ dash-cli getnewaddress
yiT6qXdqFpvJzjQaAtApgZjYpPn5koMMMb
[email protected]:~ $ dash-cli getnewaddress
yR2UzDjvyYWYxm6LBwNHWo5nfxjfX55smM
[email protected]:~ $ dash-cli getnewaddress
yLu4wzfY86XFhV8T1Vb7MGXDr2GqxFF4A9
[email protected]:~ $ dash-cli validateaddress yUyQ5gdwW9eY2rRzTFGaAgUdvXEsRq1NLM
{
  "isvalid": true,
  "address": "yUyQ5gdwW9eY2rRzTFGaAgUdvXEsRq1NLM",
  "scriptPubKey": "76a9145eee80a87109379bc861424bfce9ba1b9dac5b6c88ac",
  "ismine": true,
  "iswatchonly": false,
  "isscript": false,
  "pubkey": "03d8892e73e42a128327fdd87a0de589acd6db235a1b8f91c7d63f4c2f9b31ca1d",
  "iscompressed": true,
  "account": "",
  "hdkeypath": "m/44'/1'/0'/0/1",
  "hdchainid": "32d9544499010197fa41dad8decfbd98bbef9f19b9fe22e98654fe3ef091f23f"
}
[email protected]:~ $ dash-cli validateaddress ydg8fFWB1UKvdDoSGuqh8sAUgZzyEbAxFB
{
  "isvalid": true,
  "address": "ydg8fFWB1UKvdDoSGuqh8sAUgZzyEbAxFB",
  "scriptPubKey": "76a914be63b0b440ad056005fb955ac960cf3be745fc4d88ac",
  "ismine": true,
  "iswatchonly": false,
  "isscript": false,
  "pubkey": "020d5f44754e4a3c2934bfef8b9ccf48f6540c25b4e845f6b61660d3b75217e6c5",
  "iscompressed": true,
  "account": "",
  "hdkeypath": "m/44'/1'/0'/0/2",
  "hdchainid": "32d9544499010197fa41dad8decfbd98bbef9f19b9fe22e98654fe3ef091f23f"
}
[email protected]:~ $ dash-cli validateaddress yiT6qXdqFpvJzjQaAtApgZjYpPn5koMMMb
{
  "isvalid": true,
  "address": "yiT6qXdqFpvJzjQaAtApgZjYpPn5koMMMb",
  "scriptPubKey": "76a914f2c563dc163a76d4ab69efaae0e3de73f2c6b89c88ac",
  "ismine": true,
  "iswatchonly": false,
  "isscript": false,
  "pubkey": "02092121d1efbc8a7656af5c4e04fc606d41f50c8954beeb7ac38953f18d032374",
  "iscompressed": true,
  "account": "",
  "hdkeypath": "m/44'/1'/0'/0/3",
  "hdchainid": "32d9544499010197fa41dad8decfbd98bbef9f19b9fe22e98654fe3ef091f23f"
}
 
  • Like
Reactions: UdjinM6

camosoul

Grizzled Member
Sep 19, 2014
2,261
1,130
1,183
Tested, it's not linear...

I'll spare you the entire copy-pasta...

I ask for a new address, run validate, hdkeypath is completely random, but always within the constraints of the keypool depth defined in dash.conf.

m/44'/1'/0'/0/260
m/44'/1'/0'/0/3822
m/44'/1'/0'/0/9274
m/44'/1'/0'/0/477
m/44'/1'/0'/0/184456

If I set keypool=1000 I get

m/44'/1'/0'/0/98
m/44'/1'/0'/0/177
m/44'/1'/0'/0/817
m/44'/1'/0'/0/22
m/44'/1'/0'/0/919

I fooled with it for nearly a day before posting this just to be absolutely sure...

I did this on 6 different platforms with the same results.

The addresses appear to be created linearly, but if already created (keypool=XXX), they seem to be selected randomly from the range XXX. Every single time. 100% repeatable.

I'm only reporting what I observe... Not trying to be a dick.

If I set keypool=0, I get linear results.

I'm using the QT for requesting addresses. Does that make any difference?
 
Last edited:

chaeplin

Active Member
Core Developer
Mar 29, 2014
749
356
133
Which version do you use(download or own build) ?
Which OS/Version do you use ?
How do you run dashd ?
How do you set keypool size first time ?
Is your wallet encrypted ?
Which command do you use to get a new address ?
Do you have any account set up ?
Does fresh wallet (just usehd=1 and no keypool setup) show random address ?



Tested, it's not linear...

I'll spare you the entire copy-pasta...

I ask for a new address, run validate, hdkeypath is completely random, but always within the constraints of the keypool depth defined in dash.conf.

m/44'/1'/0'/0/260
m/44'/1'/0'/0/3822
m/44'/1'/0'/0/9274
m/44'/1'/0'/0/477
m/44'/1'/0'/0/184456

If I set keypool=1000 I get

m/44'/1'/0'/0/98
m/44'/1'/0'/0/177
m/44'/1'/0'/0/817
m/44'/1'/0'/0/22
m/44'/1'/0'/0/919

I fooled with it for nearly a day before posting this just to be absolutely sure...

I did this on 6 different platforms with the same results.

The addresses appear to be created linearly, but if already created (keypool=XXX), they seem to be selected randomly from the range XXX. Every single time. 100% repeatable.

I'm only reporting what I observe... Not trying to be a dick.

If I set keypool=0, I get linear results.

I'm using the QT for requesting addresses. Does that make any difference?
 
Last edited:

camosoul

Grizzled Member
Sep 19, 2014
2,261
1,130
1,183
I've got screen caps.

But, they are of addresses I actually use, on mainnet.

Don't want to post publicly.

GPG?