v0.10.15.x Testing

[thelonecrouton]

I'm in agreement with the mob here. Maybe I'm just misunderstanding the mechanics of it, but a third layer in the cake doesn't seem a good idea just to achieve even distribution.

Are these ubernodes running the same code as other MNs? What actually differentiates them? Why cannot plain old MNs serve the same function either via mass consensus or via a limited random selection to act as ubernodes for that block or n blocks?

 

[eduffield]

I'm in agreement with the mob here. Maybe I'm just misunderstanding the mechanics of it, but a third layer in the cake doesn't seem a good idea just to achieve even distribution.

Are these ubernodes running the same code as other MNs? What actually differentiates them? Why cannot plain old MNs serve the same function either via mass consensus or via a limited random selection to act as ubernodes for that block or n blocks?

Uber nodes run the exact same software with a special private key that enables them to sign messages that tell the network who to pay. Without the central authority, miners can pay whoever they want as the payee. The masternodes can't function as the authority, because they could exploit the code and fork the network.

It doesn't have to be this way forever, it's just a good solution to support the growth of the network. I think the best overall solution is the voting mechanism that was blockchain based (RC3). To fix that system requires that we update all miners software (CPU, GPU, etc), all stratum pools and all daemons. But it would be completely decentralized and pretty much perfect.

 

[eduffield]

I'm in agreement with the mob here. Maybe I'm just misunderstanding the mechanics of it, but a third layer in the cake doesn't seem a good idea just to achieve even distribution.

Are these ubernodes running the same code as other MNs? What actually differentiates them? Why cannot plain old MNs serve the same function either via mass consensus or via a limited random selection to act as ubernodes for that block or n blocks?

Uber nodes run the exact same software with a special private key that enables them to sign messages that tell the network who to pay. Without the central authority, miners can pay whoever they want as the payee. The masternodes can't function as the authority, because they could exploit the code and fork the network.

It doesn't have to be this way forever, it's just a good solution to support the growth of the network. I think the best overall solution is the voting mechanism that was blockchain based (RC3). To fix that system requires that we update all miners software (CPU, GPU, etc), all stratum pools and all daemons. But it would be completely decentralized and pretty much perfect.

Here's my stab at a possible solution. Please feel free to rip it apart.

1) from blockchain: during daemon startup, walk last 4000 blocks, collect paid masternode addresses.
2) from network: collate all masternode addresses from peers into array 'mnlist'
3) prune masternode addresses having been paid within length(mnlist) blocks
4) prune masternode addresses with 1k deposits newer than length(mnlist) from array. (i.e. 3000 active masternodes requires 3000 confirmations to receive payments.)
5) array now contains nodes due for payment.
6) sort array by address.
7) pay top address
8) GOTO 2

This will create a deterministic payout list all masternodes can confirm with their peers before accepting.

This assumes masternode addresses (mnlist contents) are valid/validated and that we haven't yet grown past 4000 masternodes. (step one would be adjusted accordingly.)

tl;dr: use the historical blockchain payments to prevent dupes, filter out the too-new, alpha sort the to-be-paid and pay the top address.

How does a masternode get paid the first time?

If you use the masternode list, they aren't always in sync between clients. So if a payee is on the list in one client, but not in another the network will fork.

How does that ensure the masternodes port is open and doing the service properly?

 

[eduffield]

There is more than one way to accomplish this is all we're saying.

I'm open to any solution anyone can come up with. I've been working on this for months and have had five different implementations of it. This is the best solution I've come up with yet. Yes, it's a compromise, but it seems like the benefits completely outweigh the problems.

Here's what it solves:

- Non-exploitable masternode payments (Currently miners can pay their own masternode every single time)
- Completely deterministic round-robin
- 100% compatible with proof-of-service

Anything that replaces it must solve all of those and can't be attackable.

 

[moocowmoo]

How does a masternode get paid the first time?

once it reaches the payment age it would eventually get selected for it's first payment from the sorted-by-address (leveraging dispersion) list.

If you use the masternode list, they aren't always in sync between clients. So if a payee is on the list in one client, but not in another the network will fork.

wouldn't one of the blocks just get orphaned due to it not being 'best' -- in bitcoin thats the number of leading zeros -- nothing similar in x11?

 

[oblox]

Uber nodes run the exact same software with a special private key that enables them to sign messages that tell the network who to pay. Without the central authority, miners can pay whoever they want as the payee. The masternodes can't function as the authority, because they could exploit the code and fork the network.

It doesn't have to be this way forever, it's just a good solution to support the growth of the network. I think the best overall solution is the voting mechanism that was blockchain based (RC3). To fix that system requires that we update all miners software (CPU, GPU, etc), all stratum pools and all daemons. But it would be completely decentralized and pretty much perfect.

Why can't the masternodes as a whole function as the authority? Understandably, individually they could exploit, but as a group, the majority shouldn't be rogue thus leading to consensus that is deterministically random.

 

[thelonecrouton]

The root problem seems to be that miners/pools lay outside of direct MN control.

I've suggested before that MNs act as pools. If MN pools were the only network-acceptable pools to mine at, this would place miners under direct MN control, and also solve the potential hash centralisation issue.

Maybe that's too big of a rewrite though, I don't know (assuming it was thought to be a viable idea anyway. No one has yet pointed out to me the suck in this plan. Maybe they are embarrassed by my stupidity? :tongue

If the only difference between MNs and ubernodes is that the latter have this special private key, could the form that key takes be some hashed combination of blockheight and MN identity, along the lines of 2FA? Thus the ubernodes would vary with time.

 

[eduffield]

once it reaches the payment age it would eventually get selected for it's first payment from the sorted-by-address (leveraging dispersion) list.

Sure, but then I'd just make a bunch of 1000DRK inputs and get paid with no nodes running.

wouldn't one of the blocks just get orphaned due to it not being 'best' -- in bitcoin thats the number of leading zeros -- nothing similar in x11?

No, the client must reject the bad blocks with the wrong payee.

 

[eduffield]

Really though, back to the "ubernode". It's just determining who the payee is for a given block in the future. It poses no security risk to the network and is trustless. So I don't think it should be a concern.

Later on, we can revisit this and remove the checkpointer and ubernodes and replace it with something decentralized and is really well thought out. There's 2 good ideas, one the voting in-block chain and a second blockchain for the active masternodes that's used. Both are a lot of work.

 

[flare]

Excellent discussion folks

I must confess that i am 50:50 on this topic: On the one hand i understand the dislike of yet another centralised component (beside checkpoint server and spork switch), on the other hand it doesn't make it worse, as the ubernodes are not crucial for the proper functioning of the network.

Like Evan pointed out, the best solution (out of five!) so far was the vote based system he implemented in RC3, which is has a slight incompatibility with the current mining/stratum protocol, which in turn lead to permanent forks --> https://darkcointalk.org/threads/6-20-rc3-post-mortem.1463

I also like the idea of masternodes mining (either the actual DRK blockchain or a distinct masternode blockchain).

Both solutions are a lot of work, as either the miners and stratum protocol need to be updated or a second blockchain needs to be implemented. Both have pros and cons...

 

[HinnomTX]

If you use the masternode list, they aren't always in sync between clients. So if a payee is on the list in one client, but not in another the network will fork.

This appears to be a critical problem that needs to be solved. If we can share one blockchain among all the clients, surely there is a way to share one masternode list too. Perhaps this list cannot be updated in real time, but I bet it could be updated once per block. Then build in some hysteretic qualifiers to prevent gaming of the system (e.g. have 3000 confirms to make it on a MN list of 3000, plus a few confirms of closed port/inactivity/misbehavior to get kicked off the list).

Edit: I see a second MN blockchain has been proposed. Sorry about the echo chamber.

 

[flare]

This appears to be a critical problem that needs to be solved. If we can share one blockchain among all the clients, surely there is a way to share one masternode list too. Perhaps this list cannot be updated in real time, but I bet it could be updated once per block. Then build in some hysteretic qualifiers to prevent gaming of the system (e.g. have 3000 confirms to make it on a MN list of 3000, plus a few confirms of closed port/inactivity/misbehavior to get kicked off the list).

This is exactly what Evan implemented in RC3 as vote-in-blockchain: The votes had to mature for seven blocks. But there was an issue when two miners found the block almost simultanously with different vote confirmations.... --> fork.

 

[crowning]

Anything that replaces it must solve all of those and can't be attackable.

Is there anything missing or tedious to implement??

1:  Read blockchain and look which Masternode got already a payment

1a: all got payments already: select the one with the oldest payment:
    Goto 1:

1b: not all got payments: select the oldest one (by its initial 1000 DRK transaction)
    and which is at least n-confirmations old
    If all unpaid are too young: apply 1a for the rest (already paid ones) of the Masternodes
    Goto 1:
  
  
Each Masternode broadcasts its result and the Masternode with the most votes is selected

 

[thelonecrouton]

I also like the idea of masternodes mining (either the actual DRK blockchain or a distinct masternode blockchain).

I don't know if you were referring to what I wrote, but to be clear, that isn't my suggestion. I think mining should remain, at least for now, a separate (and secondary, backup blockchain verification mechanism when InstanTX gets introduced) activity, I just think that if all pools had to run on MNs it would solve a variety of problems at once. I'll shut up about it now, promise.

 

[kryptofoo]

Really though, back to the "ubernode". It's just determining who the payee is for a given block in the future. It poses no security risk to the network and is trustless. So I don't think it should be a concern.

Later on, we can revisit this and remove the checkpointer and ubernodes and replace it with something decentralized and is really well thought out. There's 2 good ideas, one the voting in-block chain and a second blockchain for the active masternodes that's used. Both are a lot of work.

It does seem like a reasonable compromise to make sure payments are equal and cannot be gamed. But please be sure to explain it thoroughly up front to head off the uproar. if it gets mentioned only in passing e.g. as a single bullet point in the release notes, the community may latch on to this and create a lot of heat. Let's make sure people understand this is only to ensure fair masternode payments, nothing more.

I suggest not calling them privileged nodes or uber-nodes or anything of the sort. I would stick with something more benign like "reference masternodes" or "payment verification masternodes".

I trust you have looked at this from all angles and have settled on this as the best possible solution that would not consume vast amounts of your valuable time. Let's get it done and roll with the release!

 

[qwizzie]

Really though, back to the "ubernode". It's just determining who the payee is for a given block in the future. It poses no security risk to the network and is trustless. So I don't think it should be a concern.

Later on, we can revisit this and remove the checkpointer and ubernodes and replace it with something decentralized and is really well thought out. There's 2 good ideas, one the voting in-block chain and a second blockchain for the active masternodes that's used. Both are a lot of work.

Good enough for me... i think we should just proceed with bringing v0.15 to mainnet,
including the implementation of those few ubernodes that make the MN payments more reliable
and brainstorm about how we can make this all better in a seperate post here on darkcointalk

edit : assuming we consider v0.15 bugfree at the moment .. was my latest debug.log / wallet any help Evan?
there were some 0.1 fees in there although that specific client was acting a bit weird from the start.

 

[thelonecrouton]

It does seem like a reasonable compromise to make sure payments are equal and cannot be gamed. But please be sure to explain it thoroughly up front to head off the uproar. if it gets mentioned only in passing e.g. as a single bullet point in the release notes, the community may latch on to this and create a lot of heat. Let's make sure people understand this is only to ensure fair masternode payments, nothing more.

I suggest not calling them privileged nodes or uber-nodes or anything of the sort. I would stick with something more benign like "reference masternodes" or "payment verification masternodes".

I trust you have looked at this from all angles and have settled on this as the best possible solution that would not consume vast amounts of your valuable time. Let's get it done and roll with the release!

Agreed, it's not that big of a deal, but how it's presented needs some, shall we say, finesse, there *will* be bitching about this.

If some way could be found though of making the payment distribution nodes a moving bloc, that would be perfect and nobody would have anything to moan about. I'd happily trade a delay in release of Onyx for this to be achieved somehow. Sadly I'm not qualified to help much beyond tossing suggestions around.

 

[oblox]

Since we are essentially forcing mandatory updates, it wouldn't be half bad of an idea to also increase the tx size, much like BTC wants to do to future proof it. Better to get everything done in as few forks as possible than to try to fork a more established ecosystem.

 

[HinnomTX]

It doesn't have to be this way forever, it's just a good solution to support the growth of the network. I think the best overall solution is the voting mechanism that was blockchain based (RC3). To fix that system requires that we update all miners software (CPU, GPU, etc), all stratum pools and all daemons. But it would be completely decentralized and pretty much perfect.

IMHO, a revised blockchain based voting system is what you should strive for. The growth of the network will come from establishing completely decentralized, trustless relationships. A payment system that touts anonymity must be as trust-free as possible.

 

[thelonecrouton]

Is there anything missing or tedious to implement??

1:  Read blockchain and look which Masternode got already a payment

1a: all got payments already: select the one with the oldest payment:
    Goto 1:

1b: not all got payments: select the oldest one (by its initial 1000 DRK transaction)
    and which is at least n-confirmations old
    If all unpaid are too young: apply 1a for the rest (already paid ones) of the Masternodes
    Goto 1:


Each Masternode broadcasts its result and the Masternode with the most votes is selected

Yeah I'm still unsure how a few (how many?) ubernodes are able to enforce network behaviour (as InstanTX will require?) but a similar sized random - or deterministic, eg. the last n MN's to get paid - subset of existing MNs (if whole-network consensus is too unreliable) acting in the same way can not.

ubernodes = [last 8 winners]

if MN in ubernodes:
    gets to act as ubernode, wields the magic key
else:
    does not

edit: random is better, an easily discoverable list paints a target on them

 

[eduffield]

Good enough for me... i think we should just proceed with bringing v0.15 to mainnet,
including the implementation of those few ubernodes that make the MN payments more reliable
and brainstorm about how we can make this all better in a seperate post here on darkcointalk

edit : assuming we consider v0.15 bugfree at the moment .. was my latest debug.log / wallet any help Evan?
there were some 0.1 fees in there although that specific client was acting a bit weird from the start.

Yeah, looks bug free. You were the only one charged and your client froze up but was still in the process of anonymizing. I'm OK with users getting hit with collateral changes rarely, they can actually get them back if they wanted to. Plus they're only 0.1DRK. If you can anonymize 10000DRK and get hit with one, you paid about 50 cents for the whole process, which is more than fair.

 

[moli]

Yeah, looks bug free. You were the only one charged and your client froze up but was still in the process of anonymizing. I'm OK with users getting hit with collateral changes rarely, they can actually get them back if they wanted to. Plus they're only 0.1DRK. If you can anonymize 10000DRK and get hit with one, you paid about 50 cents for the whole process, which is more than fair.

Is this collateral fee in place more likely to protect Darksend from being DoSed?

 

[eduffield]

Is this collateral fee in place more likely to protect Darksend from being DoSed?

Exactly.

 

[moli]

Exactly.

Thanks, Evan

 

[TaoOfSatoshi]

Yeah, looks bug free. You were the only one charged and your client froze up but was still in the process of anonymizing. I'm OK with users getting hit with collateral changes rarely, they can actually get them back if they wanted to. Plus they're only 0.1DRK. If you can anonymize 10000DRK and get hit with one, you paid about 50 cents for the whole process, which is more than fair.

This is great now that 1DRK is $2.10, but what is going to happen when 1DRK is $100.00? $1,000? A person or business that wants to anonymize small amounts and gets hit with a $100.00 collateral charge is not going to be happy... :sad: I'm sure you will revisit those fees as DRK increases in value.

 

[moli]

This is great now that 1DRK is $2.10, but what is going to happen when 1DRK is $100.00? $1,000? A person or business that wants to anonymize small amounts and gets hit with a $100.00 collateral charge is not going to be happy... :sad: I'm sure you will revisit those fees as DRK increases in value.

Yes, i think he will... for now we need to get the system hardened and secure

 

[DrkMiner]

eduffield, The problem is that we need a List that is accessible to all Node and that is tampered proof.

We already have that "list" its the blockchain!

What can't we "mine" the data we need from the blockchain?

1) get all MN address, total number "N" and time "active minutes" (not sure where that info is stored ).
2) compare all addresses to the "N" of last paid MN (blockchain info).
3) payment:

MN is not on that list and "active minutes" > 12 hours (or less??).

I'm not a programmer, just brain storming.

 

[UdjinM6]

Uber nodes run the exact same software with a special private key that enables them to sign messages that tell the network who to pay. Without the central authority, miners can pay whoever they want as the payee. The masternodes can't function as the authority, because they could exploit the code and fork the network.

It doesn't have to be this way forever, it's just a good solution to support the growth of the network. I think the best overall solution is the voting mechanism that was blockchain based (RC3). To fix that system requires that we update all miners software (CPU, GPU, etc), all stratum pools and all daemons. But it would be completely decentralized and pretty much perfect.

I have few questions:
1. Is that "special private key" shared by uber nodes or is it generated for each one independently like "masternode genkey"?
2. Where is that functionality in code, can your give a code line link pls?

 

[Cofresí]

Uber nodes run the exact same software with a special private key that enables them to sign messages that tell the network who to pay. Without the central authority, miners can pay whoever they want as the payee. The masternodes can't function as the authority, because they could exploit the code and fork the network.

Evan, please forgive my ignorance, but I don't understand how you can expect a subset of masternodes can function as an authority in the case of InstantX when at the same time you say that they cannot be trusted in case of signing messages for masternode payments. Instead in the latter case we have to trust a few (fixed?? centralized???) uber nodes? Santissima, maybe I'm missing something, but at this stage I have to say, come on and don't rush this. You can do better than that.

 

[eduffield]

eduffield, The problem is that we need a List that is accessible to all Node and that is tampered proof.

We already have that "list" its the blockchain!

What can't we "mine" the data we need from the blockchain?

1) get all MN address, total number "N" and time "active minutes" (not sure where that info is stored ).
2) compare all addresses to the "N" of last paid MN (blockchain info).
3) payment:

MN is not on that list and "active minutes" > 12 hours (or less??).

I'm not a programmer, just brain storming.

The masternode list isn't always in perfect sync across the network. So if a node is on one clients list and not on another, the network would fork.