Development Update - 5/30/2014 - Fork Causes and Solutions

eduffield

Core Developer
Mar 9, 2014
1,084
5,320
183
Development Update - 5/30/2014

Fork Causes and Solutions
Over the last few days, InternetApe and I have been collecting data to help recreate the problems we experienced with the last hard fork. We reached out to the community and requested that anyone still running a daemon on the wrong fork report back to us. We received a great number of responses, and analysis of these daemons was incredibly useful in our effort to determine the root cause of the issue.

After examining the daemons and their associated blockchains, we've determined that, when approving blocks, in very rare situations the Masternode voting system was looking at the incorrect previous block. The forked clients were receiving an error that could only have been caused by the client comparing the current block to something that wasn't the correct previous block. This error caused the votes to appear manipulated and the client would reject the block.

Later, when receiving the same block again, the client would accept the block. This makes it clear that there wasn't an issue with the block itself, but with the system that checks it.

To solve this problem, we are going to take a multifaceted approach to development:

1.) We are recreating this problem on testnet in order to determine the best fix possible (it's a rare error, so it's taking some time).
2.) We are implementing automatic checkpointing, so that if this does happen again the network will not be able to diverge paths. In the event of a rejected block, the auto-checkpointing system will be able to tell the daemon to retry it later and put the daemon on the correct chain.

Development Schedule
Because we are implementing new technology, it will be necessary to change the schedule a bit:

RC3 (Mid June): This version will include automatic checkpointing, resolution of the forking issue, and fixes for ghost Masternodes (Masternodes which have gone offline but still appear to be online, or which have transferred their 1000DRK out but remain in the election).

RC4 (Mid July): The previous plans for RC3 are being moved to RC4. This will include significant improvements to anonymity and multi ticket support (which allows single daemons to have more than one "ticket" in the election by holding more than 1000DRK).

Programming Team
Last update we asked for anyone interested in helping the program to reach out to us. We've gotten a great number of responses, and we are still organizing and trying to find the best fit for the people that want to help out. Thanks to everyone that wanted to help further the development of Darkcoin.
 

ozziecoin

Member
May 26, 2014
50
14
48
Thanks for all the hardwork Evan. Do realise how difficult this must be.

I have two questions:

1. If there is no way for pools to hardcode/set which MN they vote for - why are 6 "votes" required?

2. Re: auto checkpointing - is this for MNs only or for the entire network of nodes?
 

Scriptiee

Member
Apr 24, 2014
44
20
48
Out of this world
[...] multi ticket support (which allows single daemons to have more than one "ticket" in the election by holding more than 1000DRK) [...]
This multi ticket solution is not a good idea:
1. Reduces the number of MNs, which decreases resiliency.
2. If malicious party infiltrated the large pot MN, you will have a compromised coinjoin service. i.e. This is back to the centralised coinjoin problem again.
 

UdjinM6

Official Dash Dev
Core Developer
Dash Core Team
May 20, 2014
3,639
3,537
1,183
I also think that multi ticket support should be done with great caution. Maybe we have to limit max number of tickets, let's say, to 5 or 10 per MN. That would be nice compromise between centralization and difficulties on maintaining many MNs.
 
  • Like
Reactions: mattmct

ozziecoin

Member
May 26, 2014
50
14
48
I also think that multi ticket support should be done with great caution. Maybe we have to limit max number of tickets, let's say, to 5 or 10 per MN. That would be nice compromise between centralization and difficulties on maintaining many MNs.
Also please see AlexGR suggestion that high resourced people can do their own stack in one MN via virtual servers on the one machine. i.e. There is no need to mess with the protocol to allow multi ticket on one MN. https://bitcointalk.org/index.php?topic=421615.msg7046010#msg7046010
 

Argamas

New Member
May 31, 2014
1
0
1
I don't see any issue with the ticket system itself, since creating and owning multiple MNs account to the same thing (eg. greater probability of processing a transaction). It doesn't really improve decentralization to have more MNs, since a single owner can still get a larger share of the transactions by deploying more nodes. It only distribute the load across more instances.

Although I agree that this feature is probably not really required at this point, and should receive a lower priority, overall.
 

just_mike

Member
Mar 9, 2014
218
20
78
Los Angeles
I also think that multi ticket support should be done with great caution. Maybe we have to limit max number of tickets, let's say, to 5 or 10 per MN. That would be nice compromise between centralization and difficulties on maintaining many MNs.
Agree - Lower number is better.
 

ozziecoin

Member
May 26, 2014
50
14
48
I don't see any issue with the ticket system itself, since creating and owning multiple MNs account to the same thing (eg. greater probability of processing a transaction). It doesn't really improve decentralization to have more MNs, since a single owner can still get a larger share of the transactions by deploying more nodes. It only distribute the load across more instances.

Although I agree that this feature is probably not really required at this point, and should receive a lower priority, overall.
I think the MN processing a transaction is randomly chosen, whereas the MN reward is paid using a 6 vote system. So, I can't assume the stacked MN has a greater probability of processing a transaction, though it has greater probability of winning the MN reward. I don't know if people would like that, tbh.

Moreover, if you're right and there is a greater probability of processing a transaction; if the large stacked MN is infiltrated, this introduces centralised coinjoin issues.
 

pinwc4

Member
May 16, 2014
45
60
58
Agree - Lower number is better.
I have multiple MN's and I agree with this as well, for health of the network encourage people to have multiple masternodes not centralize them. While it would somewhat simplify managing them for anyone actually experienced managing servers managing 1 or 100 identical servers is about the same.

Maybe a compromise would be to make the number of tickets scale less than 1000 to 1 as you increase. Something like tickets=round(dark/1000 * 0.9). With this when you move above 5000 dark you miss out on one ticket. When you get above 10,000 then you miss out on 2 tickets etc.

The more you centralize them the less credit you get. Maybe even scale to a curve instead of a fixed line to further discourage bigger groupings.
 

UdjinM6

Official Dash Dev
Core Developer
Dash Core Team
May 20, 2014
3,639
3,537
1,183
I have multiple MN's and I agree with this as well, for health of the network encourage people to have multiple masternodes not centralize them. While it would somewhat simplify managing them for anyone actually experienced managing servers managing 1 or 100 identical servers is about the same.

Maybe a compromise would be to make the number of tickets scale less than 1000 to 1 as you increase. Something like tickets=round(dark/1000 * 0.9). With this when you move above 5000 dark you miss out on one ticket. When you get above 10,000 then you miss out on 2 tickets etc.

The more you centralize them the less credit you get. Maybe even scale to a curve instead of a fixed line to further discourage bigger groupings.
Like this idea too. Less work - less reward. Proof Of Work MasterNode-Style :)
 

kaene

New Member
Apr 11, 2014
12
20
3
I have multiple MN's and I agree with this as well, for health of the network encourage people to have multiple masternodes not centralize them. While it would somewhat simplify managing them for anyone actually experienced managing servers managing 1 or 100 identical servers is about the same.

Maybe a compromise would be to make the number of tickets scale less than 1000 to 1 as you increase. Something like tickets=round(dark/1000 * 0.9). With this when you move above 5000 dark you miss out on one ticket. When you get above 10,000 then you miss out on 2 tickets etc.

The more you centralize them the less credit you get. Maybe even scale to a curve instead of a fixed line to further discourage bigger groupings.
+1
This solution makes sense to me, but still limit it to a maximum of maybe 10k DRK per MN.
 

fernando

Powered by Dash
Dash Core Team
Moderator
Foundation Member
May 9, 2014
1,527
2,058
283
In my opinion a multi-ticket system doesn't sound like the right move as it is very important to maintain network decentralization and stability.
Yes, but also having well taken care servers is important. To really be able to give an opinion I would need to know how many people run one server vs several ones.
 
  • Like
Reactions: Populandum

donho

Member
Masternode Owner/Operator
Apr 16, 2014
96
20
58
I have multiple MN's and I agree with this as well, for health of the network encourage people to have multiple masternodes not centralize them. While it would somewhat simplify managing them for anyone actually experienced managing servers managing 1 or 100 identical servers is about the same.

Maybe a compromise would be to make the number of tickets scale less than 1000 to 1 as you increase. Something like tickets=round(dark/1000 * 0.9). With this when you move above 5000 dark you miss out on one ticket. When you get above 10,000 then you miss out on 2 tickets etc.

The more you centralize them the less credit you get. Maybe even scale to a curve instead of a fixed line to further discourage bigger groupings.
sounds good at first glance. the factor could be adjusted maybe but I kind of like the idea
Would be nice to hear the Masters opinion on this :)
 

fernando

Powered by Dash
Dash Core Team
Moderator
Foundation Member
May 9, 2014
1,527
2,058
283
Another good thing about the multi ticket support is that it is probably more friendly to a multi investor masternode for people who don't have 1000 drk to start their own.
 

fernando

Powered by Dash
Dash Core Team
Moderator
Foundation Member
May 9, 2014
1,527
2,058
283
No but it would allow a person to setup a Masternode and sell shares to other investors. Easily add or remove 1k DRK.
Exactly. Depending on how this system would work (it doesn't exist yet!) it could make easier to reallocate investments if there were the option to have bigger masternodes.
 

pinestabe

New Member
May 1, 2014
26
9
3
Would it be possible / a good idea to implement checkpoints so that they are accepted for only limited times? Seems like they should only be needed for a short time after hardforks.
 

eltito

Active Member
Apr 21, 2014
157
185
103
Would it be possible / a good idea to implement checkpoints so that they are accepted for only limited times? Seems like they should only be needed for a short time after hardforks.
Think of it as a safety net. In the event of any unforeseen issues, it'll prevent the sort of forking we saw last weekend (well, more precisely, it'll correct it before it becomes a network-wide problem). Better idea to have it running constantly "just in case".
 

pinestabe

New Member
May 1, 2014
26
9
3
It's nice to have a safety-net, but the checkpoint server could also be a source of problems. Seems like it would require considerable effort to continuously make sure it's running properly. If it's running only for limited times there's less chance of problems happening over time. I'm not really familiar with the checkpointing system, though.
 

eltito

Active Member
Apr 21, 2014
157
185
103
It's nice to have a safety-net, but the checkpoint server could also be a source of problems. Seems like it would require considerable effort to continuously make sure it's running properly. If it's running only for limited times there's less chance of problems happening over time. I'm not really familiar with the checkpointing system, though.
I understand your concern, but standard maintenance headaches like those you mention are simply insignificant in comparison to another random forking incident.
 
  • Like
Reactions: pinestabe