Dash Core v0.14 on Mainnet

Status
Not open for further replies.

splawik21

Grizzled Member
Dash Core Team
Moderator
Foundation Member
Dash Support Group
Apr 8, 2014
1,917
1,274
1,283
If you have received the PoSe Penalty on your MN and you have updated the dashd to 14.0.1 and have the sentinel on 1.4.0 but still your PoSe score rise be sure you have not put the blsprivkey (operator private key) on both sides, remote server and local.
ONLY blsprivkey (operator private key) must be on dash.conf remote server, on the local side it MUST be a public operator key which always is created in pair when you use "bls generate" command in console.
Please follow the steps precisely when updating/registering the MN especially that this is the FIRST step, to generate the blskeys pair. https://docs.dash.org/en/latest/masternodes/dip3-upgrade.html#generate-a-bls-key-pair

edit: operator public key goes to DMT when using the HW and if using dashcore you do not put in on local dash.conf
 
Last edited:

JGCMiner

Moderator
Moderator
Jun 8, 2014
360
211
113
If you have received the PoSe Penalty on your MN and you have updated the dashd to 14.0.1 and have the sentinel on 1.4.0 but still your PoSe score rise be sure you have not put the blsprivkey (operator private key) on both sides, remote server and local.
ONLY blsprivkey (operator private key) must be on dash.conf remote server, on the local side it MUST be a public operator key which always is created in pair when you use "bls generate" command in console.
Please follow the steps precisely when updating/registering the MN especially that this is the FIRST step, to generate the blskeys pair. https://docs.dash.org/en/latest/masternodes/dip3-upgrade.html#generate-a-bls-key-pair
Thanks for this splawik21.

Guys just to reiterate. If you are having issues double check your configuration carefully, step-by-step. On Discord, if the problem wasn’t a non-updated node it was the above issue (bls operator key and related dash.conf misconfiguration).

I know it seems like that was for v13, but with v14 everything, including things introduced in the past, must be setup correctly or your node will run into PoSe issues.
 

codablock

Active Member
Core Developer
Mar 29, 2017
100
154
93
36
So, just had a private discussion with a MNO that had PoSe problems. Turned out that he was running a second copy of the same MN in a different location as a backup solution, which means he used the same BLS operator key on two instances of dashd. This is NOT supported and will lead to your node being banned. The reason is that running nodes in this way would cause potential conflicts in LLMQ votes.
 

Rick Seeger

New Member
Jun 8, 2019
12
5
3
49
Half my nodes on Vultr are banned even though they've never experienced any down time. All of them have a PoSe penalty > 0. I parsed the debug.log to produce this chart - definitely linear, possibly exponential. I don't think it's worth the effort to re-establish my nodes until this is resolved. There's clearly a major problem here.View attachment 9543
did you proper register your MN (determanistic and all) ?
BLS operator key ?
Yes. I rebuilt all my servers with V14 back in April and registered them as deterministic. I check their status hourly through cron (via RCP to dashd with masternode list command) to verify their status is indeed ENABLED. This is how I am aware that half of them are down. Weirdly, the other half are still humming along fine, which is why it doesn't seem like a configuration issue (they all have identical config).
 

qwizzie

Well-known Member
Aug 6, 2014
1,561
729
183
Half my nodes on Vultr are banned even though they've never experienced any down time. All of them have a PoSe penalty > 0. I parsed the debug.log to produce this chart - definitely linear, possibly exponential. I don't think it's worth the effort to re-establish my nodes until this is resolved. There's clearly a major problem here.View attachment 9543


Yes. I rebuilt all my servers with V14 back in April and registered them as deterministic. I check their status hourly through cron (via RCP to dashd with masternode list command) to verify their status is indeed ENABLED. This is how I am aware that half of them are down. Weirdly, the other half are still humming along fine, which is why it doesn't seem like a configuration issue (they all have identical config).
See : https://www.dash.org/forum/threads/dash-core-v0-14-on-mainnet.45821/page-2#post-212492
 

qwizzie

Well-known Member
Aug 6, 2014
1,561
729
183
If you have received the PoSe Penalty on your MN and you have updated the dashd to 14.0.1 and have the sentinel on 1.4.0 but still your PoSe score rise be sure you have not put the blsprivkey (operator private key) on both sides, remote server and local.
ONLY blsprivkey (operator private key) must be on dash.conf remote server, on the local side it MUST be a public operator key which always is created in pair when you use "bls generate" command in console.
Please follow the steps precisely when updating/registering the MN especially that this is the FIRST step, to generate the blskeys pair. https://docs.dash.org/en/latest/masternodes/dip3-upgrade.html#generate-a-bls-key-pair
This should really be added to that referred docs as it seems to be missing completely.

The docs just mentions the adding of BLS privkey to dash.conf on the remote server, it does not mention anything
about putting a BLS public key in a dash.conf of a local client / server.
 
Last edited:

Rick Seeger

New Member
Jun 8, 2019
12
5
3
49
I appreciate the responsiveness and all the info here in this thread. I'm reviewing my config carefully including the blspriv/operator key pairs and have sent my MN info to @codablock. Meanwhile, I'm watching the bans very closely. Here are the latest stats:

Since activation:
274 total bans
63 revived
2 of the revived were re-banned

Cumulative bans confirmed linear:
Correlation coeff: 0.9866
Slope: 80.48 masternodes bans / day
total-bans-20190608.png
 
  • Like
Reactions: grzem and pin0de

pin0de

New Member
Jan 30, 2016
36
29
18
I also have one out of multiple identical nodes that shows a PoSePenalty and I've yet to figure out why.
DMT has the correct operator public key in the config.
 

strophy

Administrator
Dash Core Team
Moderator
Dash Support Group
Feb 13, 2016
710
408
133
The docs just mentions the adding of BLS privkey to dash.conf on the remote server, it does not mention anything
about putting a BLS public key in a dash.conf of a local client / server.
Operator BLS pubkeys are not stored in a local wallet and should not be entered in any configuration files. This serves no purpose, as far as I know there is no code existing to do anything with this information. The BLS pubkey should exist in one place only - in the registration protx written to the blockchain. The BLS privkey should only appear in the dash.conf file of a single masternode. The BLS privkey must be unique and not appear in any other dash.conf for a registered masternode, or both masternodes will be banned.

If you are registering masternodes from a hardware wallet, it is possible to import the owner privkey in an instance of Dash Core Qt wallet in order to monitor your masternodes from the "My masternodes" tab.

Also, to avoid confusion, I have removed the /latest/ branch of the documentation. The documentation for the currently released version of Dash Core is available under the /stable/ URL, and legacy documentation, including the DIP3 upgrade process, is available (and maintained/updated if necessary) under versioned branches, e.g.: https://docs.dash.org/en/0.13.0/masternodes/dip3-upgrade.html You can change the documentation version you are looking at from the menu in the bottom right corner.
 

qwizzie

Well-known Member
Aug 6, 2014
1,561
729
183
Operator BLS pubkeys are not stored in a local wallet and should not be entered in any configuration files. This serves no purpose, as far as I know there is no code existing to do anything with this information. The BLS pubkey should exist in one place only - in the registration protx written to the blockchain. The BLS privkey should only appear in the dash.conf file of a single masternode. The BLS privkey must be unique and not appear in any other dash.conf for a registered masternode, or both masternodes will be banned.

If you are registering masternodes from a hardware wallet, it is possible to import the owner privkey in an instance of Dash Core Qt wallet in order to monitor your masternodes from the "My masternodes" tab.

Also, to avoid confusion, I have removed the /latest/ branch of the documentation. The documentation for the currently released version of Dash Core is available under the /stable/ URL, and legacy documentation, including the DIP3 upgrade process, is available (and maintained/updated if necessary) under versioned branches, e.g.: https://docs.dash.org/en/0.13.0/masternodes/dip3-upgrade.html You can change the documentation version you are looking at from the menu in the bottom right corner.
See post from pin0de (above yours). Should he not have used the BLS private key ? He seems to indicate he is using the BLS public key ? :confused:

I also have one out of multiple identical nodes that shows a PoSePenalty and I've yet to figure out why.
DMT has the correct operator public key in the config.

#69pin0de, Today at 5:15 AM
If he is actually using the BLS public key there, it could explain his PoSe penalty ?
 
Last edited:

pin0de

New Member
Jan 30, 2016
36
29
18
As DMT generates the key pair, it has both, the public and private key. You can display either one, so yes that's a bit confusing when asking people whether they have correctly entered the key locally.
 

camosoul

Grizzled Member
Sep 19, 2014
2,265
1,130
1,183
@camosoul : i remember you using a restart script / command that lets your masternodes restart every "x" time automatically.
Is that still the case ? I'm wondering if that could have either an impact on your PoSe score or obscure a possible misconfiguration.
1) I no longer run masternodes.
2) I help people set up masternodes.
3) My access to configuration information varies by each individual's willingness to share data. I do't ask for what I don't need, and I don't keep records.

About an hour ago, 50% of observed nodes have been banned by no discernible fault of the owner or operator.

[shrug]

sumthin' ain't right

Most of these nodes ran fine for several days, then got hammered with penalties in rapid succession for no apparent reason. Some are heavily overbuilt bare metal, some are VMs. The interesting part are the VMs... They're running on the same host. Some get pounded off, and other VMs on the same host are flawless... Run by the same owner/operator who isn't likely to be making mistakes in config on one, then not on the other. The bare metal machines only prove that no amount of clocks, resources, or network bandwidth will help you. I'm seeding no evidence of DoS, but my view is also limited so that seems inconclusive. I would note that any evidence of a DoS would be glaringly obvious, so, DoS is unlikely to be in play.

If there is a "mis-configuration" then it is a result of unclear or inaccurate documentation. It shouldn't be possible to register incorrectly. Nobody wants to do this wrong for fun.

edit: could you share the mis-configuration details so that I might suggest what to look for instead of requesting information from those who'd rather not share it?
 
Last edited:
  • Like
Reactions: grzem

camosoul

Grizzled Member
Sep 19, 2014
2,265
1,130
1,183
What's the point in restarting masternodes regularly? Using the stop command or by killing the process with a SIGTERM?
I generally would recommend to use scripts which only start the daemon when it is not running (based on the PID file or by process name in a single node setup).
That was based on old info back when MNs were a fairly new thing and liked to crash a lot. It turned out to be resultant to an upstream issue. I forget the exact details, but it was a block irregularity the fix for which actually got (thanklessly) upstreamed...

I still run my clients this way, but nowhere near as aggressively as I used to. I don't have masternodes anymore; too dangerous given my status as a vocal anti-communist who is still owned by The Communist States of America.
 
Last edited:

camosoul

Grizzled Member
Sep 19, 2014
2,265
1,130
1,183
In other news... Is there an upper limit to DASH denominating? Used to be 2K. Seems to be, uh, not 2K anymore...
 

camosoul

Grizzled Member
Sep 19, 2014
2,265
1,130
1,183
As DMT generates the key pair, it has both, the public and private key. You can display either one, so yes that's a bit confusing when asking people whether they have correctly entered the key locally.
I seem to say this regarding any software project done since the early 90s; really bad documentation.
 

pin0de

New Member
Jan 30, 2016
36
29
18
so the affected node got hit with a penalty again and is banned now ...
would have been paid tomorrow, that is bad
 

camosoul

Grizzled Member
Sep 19, 2014
2,265
1,130
1,183
I've tried to reply multiple times, but the website is acting up again... I'm a bit frustrated.
It's open source software, feel free to contribute to the documentation to improve it.
I can't possibly know unless I wrote the code... It's the same irrational expectation across the board with coders these days.

I'd love to help out, but unless I already know, which I don't, I can't write docs. Docs have to be written by the people who wrote the code. Period. But since this makes coders mad ("How dare you not already understand my genius! Behold, the glory of my massive ego!"), they passive-aggressively create circular self-references written as a brag that you can't understand unless you already know... It's basically "Oh, you're going to make me write docs? So I'll write useless docs! NYA NYA! Now gimmie giant piles of money."

It's not such an extreme case in this instance, but it's become so commonplace that most people, myself included, don't even bother to read anymore... It's often faster and easier to just start fumbling and fix it as you go than deal with the obnoxious, snarky wise-assery laughingly referred to as The eFfing Manual.

So far I gather, "key doesn't mean key." If the owner and operator are two different people, there's still no clear direction for what to do or why. It's blind and arbitrary... If it were ever explained, one might logically deduce what to do for himself... But, it isn't explained, it's just bragged with sarcastic circular references... Directions still unclear, dicks still caught in ceiling fans. Naturally...

This is a bit of a rant becasue I'm also forcing myself to learn Python. Which is a miserable experience sorting through mountains of worthless, bloviating, egotistical, epeen noise to occasionally find a useful fact... My patience for this idiotic behavior was used up decades ago. Can't care anymore. Just keep beating the freaks until they finally give in and stop talking in circles. It's amazing what BS can be dissolved by the application of pain, and how much pain an obnoxious jerk is willing to tolerate merely for the game of continuing to be an obnoxious jerk... But, after hours of going around in torturous nonsense, you reach the breaking point, and suddenly the direct answer to the question you asked emerges. Why not start out that way? Why is it so important to be a deliberately un-useful, uncooperative, childish a-hole at every opportunity? I'll never understand it...

As for helping out... I even offered to fix up the vending machine, but it's held in a communist sanctum into which I am forbidden to enter if I wish to stay living... The games being played by people pretending that they aren't playing games... I find it baffling that anyone with an IQ above 80 can't sort it out for themselves, it's not complicated, but whatevs....
 
Last edited:

qwizzie

Well-known Member
Aug 6, 2014
1,561
729
183
so the affected node got hit with a penalty again and is banned now ...
would have been paid tomorrow, that is bad
See : https://www.dash.org/forum/threads/dash-core-v0-14-on-mainnet.45821/page-2#post-212492

At this point i think you are better served by providing codablock the requested info, so he
can take a look at your debug.log and pinpoint a possible misconfiguration.

Misconfiguration could be a number of things i guess, including but not limited to :

* using an incorrect BLS operator key in the masternode configuration
* using the same BLS operator key on two instances of dashd
* forgetting to update sentinel
* deviating from the upgrade guide
 
Last edited:

codablock

Active Member
Core Developer
Mar 29, 2017
100
154
93
36
@camosoul the docs and instructions are pretty good IMHO, and if details are missing/misleading then it is something that is usually fixed fast. Your babbling about bad documentation is out of place here tbh.

When it comes to misconfiguration, everyone who has issues please run "dash-cli masternode status" on the masternode and it will tell you if your node is configured well. Also make sure to NOT run the same masternode twice, as this is guaranteed to get you banned.
 

qwizzie

Well-known Member
Aug 6, 2014
1,561
729
183
According my calculations only 315 blocks to go before the second window / 4032 blocks has ended and we could have a possible DIP8 lock-in.
Second window should end with block 1,084,608

To check for a possible lock-in : issue a "getblockchaininfo" command
After the lock-in there will be another window of 4032 blocks to activate DIP8
 
Last edited:

masternube

Member
Nov 9, 2017
81
14
48
For DIP8 to activate we need 80% of blocks between 1080577 and 1084608 to be signaling support. That's 3226 blocks or more. Block 1084340 was the 3226th block since 1080577 to signal so DIP8 is now guaranteed to be locked in and will active at 1088640. Since is expected to happen (with 90% probability) on 2019-06-17 between 7:05 and 16:32 UTC.
 
Last edited:

flare

Administrator
Dash Core Team
Moderator
May 18, 2014
2,287
2,406
1,183
Germany
For DIP8 to activate we need 80% of blocks between 1080577 and 1084608 to be signaling support. That's 3226 blocks or more. Block 1084340 was the 3226th block since 1080577 to signal so DIP8 is now guaranteed to be locked in and will active at 1088640. Since is expected to happen (with 90% probability) on 2019-07-17 between 7:05 and 16:32 UTC.
Yup, we have locked in

upload_2019-6-9_17-54-1.png
 

qwizzie

Well-known Member
Aug 6, 2014
1,561
729
183
For DIP8 to activate we need 80% of blocks between 1080577 and 1084608 to be signaling support. That's 3226 blocks or more. Block 1084340 was the 3226th block since 1080577 to signal so DIP8 is now guaranteed to be locked in and will active at 1088640. Since is expected to happen (with 90% probability) on 2019-07-17 between 7:05 and 16:32 UTC.
First window / first 4032 blocks : failure to lock-in DIP8, due to lack of support

Started with block 1,076,544
Ended with block 1,080,576

Second window / second 4032 blocks : will most likely lock-in DIP8, as miners now seems to show enough support

Starting with block : 1,080,577
Ending with block : 1,084,609

Third window / third 4032 blocks : will most likely activate DIP8, depending on lock-in taking place in second window.

Starting with block : 1,084,610
Ending with block : 1,088,642

Edit : strange, thought we still had 10 or 12 hours to go for a lock-in, but according to flare we indeed have already locked in. Dont these windows have an exact number of blocks ? (4032?) :confused:
 
Last edited:

flare

Administrator
Dash Core Team
Moderator
May 18, 2014
2,287
2,406
1,183
Germany
Edit : strange, thought we still had 10 or 12 hours to go for a lock-in, but according to flare we indeed have already locked in.
Technically you are right: The period has 260 blocks left. But as we are beyond the 80% threshold already we are past point of no return and i consider it as locked in :)
 

masternube

Member
Nov 9, 2017
81
14
48
First window / first 4032 blocks : failure to lock-in DIP8, due to lack of support

Started with block 1,076,544
Ended with block 1,080,576
That's 4033 blocks.

Second window / second 4032 blocks : will most likely lock-in DIP8, as miners now seems to show enough support

Starting with block : 1,080,577
Ending with block : 1,084,609
If the first window started at 1,076,544, then the second window must also start at an even number, otherwise you have a window with odd size.
I'm not sure what is the exact cut off, but I assumed every window start with an odd number since I assume the first block was #1. But I might be off by one.
 

qwizzie

Well-known Member
Aug 6, 2014
1,561
729
183
Technically you are right: The period has 260 blocks left. But as we are beyond the 80% threshold already we are past point of no return and i consider it as locked in :)
Any information when spork 19 (ChainLock) will be activated ? I exspected this spork to be activated already as the Dash Core v0.14 Planned Upgrade Phases mentions "The DKG Spork (17) will be activated,
followed shortly by the ChainLock spork (19)"



Are you planning to do this after lock-in of DIP8? or after activation of DIP8 ? (when ChainLocks is enforced as well)
Or a period after that ?
 
Last edited:
Status
Not open for further replies.