• Forum has been upgraded, all links, images, etc are as they were. Please see Official Announcements for more information

croutonrant #237 - the happy happy dream of decentralised mining!

thelonecrouton

Well-known member
Foundation Member
Thesis:


The purpose of mining is to secure the blockchain and ensure its integrity via decentralised consensus.

Distributed mining makes attacks upon the blockchain prohibitively expensive.

Centralised mining reduces the cost of an attack and decreases network security in direct proportion to that centralisation.

1000 individual miners are 1000x more expensive to attack than 1000 miners acting as one entity through one server.

Right now mining is almost entirely useless as a security mechanism, as over 50% of the nethash is centralised through only 3 servers - the complete opposite of the intended purpose of PoW mining.

Only 25% of the nethash is required to cause severe blockchain vandalism. > 50% guarantees complete control.

Darkcoin mining is useless unless it provides a provably greater barrier to attack than the Masternode network, or a secure and cost effective backup.

p2pool mining has obvious security advantages over mining via centralised pools, but is not without problems, most notably blockchain dustbloat.

Mandatory solo mining through users wallets gets rid of the dust problem and effectively guarantees distributed consensus but makes mining basically a lottery.

Which is more important to existing and future Darkcoin users, investors, and related businesses - network security or guaranteeing any given miners income?

Mining costs have been claimed to add value to the resultant mined coins, but this is demonstrably not the case. Darkcoin has never been profitable to mine (apart from the first few weeks) and most DRK are sold for less than the mining expense in electricity, never mind the hardware costs of mining rigs.

If PoW mining is to be kept as a security mechanism - and what other useful purpose does it serve? - then IMO enforced solo mining is the simplest way of achieving the desired decentralisation.

Solo mining removes all the middleware from the equation, whose sole purpose in the case of mpos/nomp is to increase income for miners at the expense of network security for all users, and reduces the developer workload as they would be dealing with a simpler and more manageable system, which they are already familiar with.




Proposed implementaion:


Work submitted by solo miners will only be considered as valid if the wallet submitting the work has a positive balance (I suggest minimum 1DRK) and is signed with a valid privkey tied to that wallets's main address.

Wallet balance does not matter beyond that, this remains Proof of Work, not Proof of Stake. The 1DRK requirement serves as an additional minimum deterrent to an attacker running thousands of wallets at purely electrical cost.

Wallets are user-configurable with regard to resources used in mining - number of cores, or maximum CPU load.

The Masternode network will form consensus on the validity of submitted work.




Expected objections:


Why don't you bugger off and do it yourself then, smartarse?

- I would if I could, and I'd be rich as a result, because if implementing this or some alternative scheme for ensuring sensibly distributed consensus is achieved, the blatantly false 'security' provided by all other PoW coins will be exposed as horribly inadequate by comparison.

If this is even possible, why has nobody done it before?

- Masternodes have not existed before now. They offer their own decentralised means of determining block/work validity.


tl;dr: Network security is more important to all Darkcoin users than providing guaranteed income for miners.




Interesting proposal to incentivise solo mining vs pool mining:
https://bitslog.wordpress.com/2014/06/19/theoretical-and-practical-nonoutsourceable-puzzles/
 
Proposed implementaion:


Work submitted by solo miners will only be considered as valid if the wallet submitting the work has a positive balance (I suggest minimum 1DRK) and is signed with a valid privkey tied to that wallets's main address.

Wallet balance does not matter beyond that, this remains Proof of Work, not Proof of Stake. The 1DRK requirement serves as an additional minimum deterrent to an attacker running thousands of wallets at purely electrical cost.

Wallets are user-configurable with regard to resources used in mining - number of cores, or maximum CPU load.

The Masternode network will form consensus on the validity of submitted work.

...and I'll write a little proxy which routes other miners through my wallet. I'll receive their work, sign it with my key, send it to the network and distribute the earnings according to their mining share. Maybe I'll keep 1-2 % for my service.

I think I'll call this proxy "a Pool".
 
What if each privkey had a hash per unit time limit?

A useful limit _now_ means that in a year I can't get my newest mining rig (featuring 8 of the new Nvidia Geforce GTX 1380 (Einstein-Architecture) with about 80 MH/s each) to mine Darkcoin.
 
Up it every 6 months, or as needed. Or don't. The point of PoW is to have as many discrete machines as possible contributing to the total. The goal is not to facilitate miner profits, but to secure the network from attack via distribution of hash.

Making it harder for industrial scale miners makes it more profitable for small miners, which encourages more of them.

edit: situation right now:
2_pools_cropped.png


2 pools = 48.63%
3 pools = 62.34&
5 pools = 74.95%
10 pools = 91.22%

Decentralised cryptocurrency right there. :rolleyes:

PoW is a sitting duck.

How many darknet servers did the FBI take down a few weeks back? What if they realised it would be 100x cheaper to just subvert 3 Darkcoin pools and tank the currency so we couldn't use it at all? $12,000,000 marketcap destroyed overnight at the taxpayers expense.

DEA agent Wendell Campbell for one is on the case - remember this? -

 
Last edited by a moderator:
Hmm. Another option is multi-algo like digibyte have done? They are still fine tuning but in theory it means asics, gpus, cpus everyone can play and because you have blocks being found by different algos a 51% is way more difficult.
 
Hmm. Another option is multi-algo like digibyte have done? They are still fine tuning but in theory it means asics, gpus, cpus everyone can play and because you have blocks being found by different algos a 51% is way more difficult.
Hadn't thought of that, mining as a multi-stage process requiring different hardware for each stage. I could see it being very difficult getting the reward structure right though, so each kind of number crunching yielded close to the same margins, as hardware for each stage evolved and changed.

I think if that kind of architectural change was to be considered, the solution I linked in the OP would be simpler, just make pool mining painfully costly for both miners (who used pools) and pool ops to encourage solo mining instead. From my understanding the algo could be kept the same, it doesn't matter, it would just be the block structure that was altered.

Wish I'd started learning more about this stuff before senility set in...
 
Last edited by a moderator:
Hadn't thought of that, mining as a multi-stage process requiring different hardware for each stage. I could see it being very difficult getting the reward structure right though, so each kind of number crunching yielded close to the same margins, as hardware for each stage evolved and changed.

It's not mining at different stages, each group is essentially competing for the next block with difficulty unique to each class. Block reward would stay the same. Not sure if this could (or would) work in Dark's case. I do like the idea of securing the network with all different kinds of hardware.
 
If you block people or punish people they will find a way to increase their advantage. A warehouse full of asics solomining is no better than a pool. With rewards for mining dropping with mn payments increasing pools like waffle are going to have less sway. I'm not a fan of waffle as it autodumps..and attracts large purely for instant profit hashrates. In the short term there will be coins more profitable and they'll leave us alone for a while. Until the price significantly spikes we'll have some breathing room to look and debate further about stuff like this. The reason I like the digibyte solution is there is no clear advantage for one algo/method. It really encourages mining, all mining as if the diff drops on one algo everyone has a chance of pouncing. Might be very difficult to implement though.
 
Is there a way of making the block reward proportional to the hash speed applied per miner using a bell shaped curve like a gaussian function as the payoff. Starts off promising, reaches a crescendo of flatish niceness and quickly dwindles to despair, forcing an ironic down grade of the computing power to get back payout.Thus creating many small pools, and be good on the electricity bill and environment to boot.
 
Is there a way of making the block reward proportional to the hash speed applied per miner using a bell shaped curve like a gaussian function as the payoff. Starts off promising, reaches a crescendo of flatish niceness and quickly dwindles to despair, forcing an ironic down grade of the computing power to get back payout.Thus creating many small pools, and be good on the electricity bill and environment to boot.
Another brilliant idea. :) Would still need a way of identifying discrete machines (and their hashrates) though. I think. Or engineer an algo with a sweet spot, or maximum hashrate?
 
Quote from the link in the OP, in case anyone skipped it:

A Practical and Simple Proposal: BlockPow

I will propose a kind of PoW puzzle that only practically prevents mining pools for certain cryptocurrency settings. The idea is that if miners try to join a pool, then they incur in overhead and earn less than working solo. Let b (the payload) contain a full block. b must contain all the transactions and the header, and not only the transaction hashes. b should not hide anything. Let h be the block header (which contains the previous block hash and the Merkle tree root of the transactions). Let d be the difficulty. hash-block-length(b) returns the number of blocks processed by the hash function when fed with the block b. The target is divided by hash-block-length(b) so that the difficulty does not depend on the length of the block. Some other function can be used to encourage nodes to add more or less transactions.

Def: BlockPoW

For each r in the nonce-range: if H ( H( r || b ) || h || r) ) < 2^-d/ hash-block-length(b) then return r

return null

The header (h) is appended to the hashed message to allow SPV clients to verify the PoW without requiring the full block to be downloaded. Peers can send only (x,r,h) to SPV nodes, where x = H( r || b ), so they can verify the PoW. The verification procedure is obvious, and is skipped here. r is inserted at the beginning of the message to prevent pool administrators from keeping a secret mid-state of the hash function secret in order to prevent block stealing and also to force the miner to know b in the inner mining loop.

So now mining requires the knowledge of the block b to be mined, and b must be received at each block-chain epoch. This could create an incentive not to include any transaction and use an almost empty b, because that way the bandwidth requirements is decreased. But this incentive should not exists normally, since by including transactions the solo miner gets an additional revenue from fees, which is lost if the block is empty. Anyway, to prevent this possible incentive we can append to b a subset of previous blocks (e.g 100 blocks). The block subset to include could be derived from a peudo-random function seeded by the previous block hash. Then a node would still need to download part or all the block-chain in order to mine.

Now if the miner wants to be a dumb node and take part of a pool, then the current working unsolved block (the template) must be sent each time from the pool admin to each miner. If the pool admin hosts 1000 miners, then to serve them with fresh block templates he needs 1000 times more bandwidth that the miners, making this much more expensive. If miners create another network topology to distribute templates, they are incurring in the same overhead as being actively part of the cryptocurrency network, so they gain nothing.

For example, in a block-chain with a 5 seconds block-rate, such as in NimbleCoin, each block can be as large as 200 Kbytes (100 tps are allowed). A miner will require the block template to be ready as fast as possible, say before 3 seconds, so as not to loose more than 60% of the times the transaction fees present in the block template. This means that a pool admin serving 1000 clients must have a upload bandwidth of at least 60 Mbytes/sec, and load balance servers, because all miners will demand the block template at the same time and will compete for it.

The same miner, working solo, will not loose the 60% of block fees. If block fees are 10% of block reward, then solo miners earn 6% more than pool miners. Also, having a block rate of 5 seconds allows solo miners to receive payments more often and so it reduces the payment variance.

This method to discourage mining pools only work as long as time is takes to transmit a block is comparable to the block interval time, e.g. 20%. This seems not to be a problem since if the cryptocurrency becomes popular, then we can expect blocks to be near full, while if is is not, then nobody will care about mining pools.

For this method to work for Bitcoin, Bitcoin should reduce the block rate to at least 1 minute, and keep blocks of at least 10 Mbytes. Or go the NimbleCoin way, and reduce the block interval to 5 seconds. The sole reduction of the block rate from 10 minutes to 5 seconds would reduce notably the mining reward variance, which is the main reason miners don’t mine solo.

BitcoinBlockPow

The basic BlockPoW is incompatible with Bitcoin ASICs as is but it can be made partially compatible with some tweaks: The value b is replaced by a a a subset or an integrity check of the block.

Using a subset:

First the hashMerkleRoot and hashPrevBlock fields are replaced by the fields: ChildBlock (32 bytes) and ScatteredBlockBytes (32 bytes). ChildBlock is the hash of a message with stores the old hashMerkleRoot and hashPrevBlock. ScatteredBlockBytes is a pseudo-random subset of bytes taken from the block template being mined. The seed for the pseudo-random function that selects the subset is the hashMerkleRoot plus the block time. When a miner scans all the 32bit nonce space, then a new hashMerkleRoot must be created to increase the extra-nonce field or the time must be updated. When this happens, a new subset of pseudo-random 32 block bytes must be collected. If the miner only stores 10% of the block template (e.g. 100 Kbytes instead of 1 Mbyte), then the probability he can build the ScatteredBlockBytes by brute-forcing the seed is 10^-32. If the miner performs 100 GH/sec, then the 32-bit nonce will overflow every 20 msec and the miner could request the ScatteredBlockBytes from the pool admin using a bandwidth of 1 Kbyte/s. A pool hosting 6 PH/sec (such as Eligious, which has 8%) would need to stream more than 60 Mb/s of ScatteredBlockBytes fields. A mining pool having 50% would need to stream 500 Mb/s, which is quite challenging. So miners will download the block normally, and become active part of the network.

Using an integrity check:

ScatteredBlockBytes is replaced by a field BlockHash defined as H( full-block-with-zero-nonce ). Obviously the header must be at the beginning of the block to force the re-hash.
 
TheLoneCrouton, thanks,
Another brilliant idea. :) Would still need a way of identifying discrete machines (and their hashrates) though. I think. Or engineer an algo with a sweet spot, or maximum hashrate?
Thanks, complements for ideas from you means something, I struggle to get the deeper structure but am getting there slowly lol.
I always find it odd, that something cutting edge and hard to understand become common place and kind of simple when enough people seem to have 'got it' . Is almost like there is a connected global consciousness shift of comprehension, a kind of spooky quantum entanglement of minds, I see this now with bitcoin, ahem, anyway ..
 
I don't get:
Nimblecoin (what the ... is this anyway?) 200 Kb per 5 sec (block) -> 4 Mb per minute -> 240 Mb per hour -> 5,76 Gb per day :what:
The idea for Bitcoin to go 1 minute block of 10 Mb each -> 14,4 Gb per day :eek:
Is it one-day-to-live coin or what? Or am I missing something?
 
I don't get:
Nimblecoin (what the ... is this anyway?) 200 Kb per 5 sec (block) -> 4 Mb per minute -> 240 Mb per hour -> 5,76 Gb per day :what:
The idea for Bitcoin to go 1 minute block of 10 Mb each -> 14,4 Gb per day :eek:
Is it one-day-to-live coin or what? Or am I missing something?
As far as I understand it the block structure requires pools to push big enough bandwidth to their miners that it becomes prohibitively expensive, thus encouraging solo mining. It's unclear to me how much of this would need to eventually end up in the actual blockchain though.
 
As far as I understand it the block structure requires pools to push big enough bandwidth to their miners that it becomes prohibitively expensive, thus encouraging solo mining. It's unclear to me how much of this would need to eventually end up in the actual blockchain though.

OK, I read few times...

I think that's the part that is causing confusion:
"For example, in a block-chain with a 5 seconds block-rate, such as in NimbleCoin, each block can be as large as 200 Kbytes", "For this method to work for Bitcoin, Bitcoin should reduce the block rate to at least 1 minute, and keep blocks of at least 10 Mbytes."

But I guess this was only a theoretical approach or he forget the word "template". I highly doubt that any real coin can exist with such parameters. And then suddenly author jumps back to the reality - not by saying it directly but speaking of making things compatible with ASICs. And that's the real part needed to understand:
"First the hashMerkleRoot and hashPrevBlock fields are replaced by the fields: ChildBlock (32 bytes) and ScatteredBlockBytes (32 bytes). ChildBlock is the hash of a message with stores the old hashMerkleRoot and hashPrevBlock. ScatteredBlockBytes is a pseudo-random subset of bytes taken from the block template being mined. The seed for the pseudo-random function that selects the subset is the hashMerkleRoot plus the block time. When a miner scans all the 32bit nonce space, then a new hashMerkleRoot must be created to increase the extra-nonce field or the time must be updated. When this happens, a new subset of pseudo-random 32 block bytes must be collected."

And in simple words it means: block template is flooded with some pseudo-random subset of previous blocks but block header (and we are hashing only header and not the whole block) contains not the flood-data itself but only some pseudo-random 32 bytes (ScatteredBlockBytes) of that flood-data. And pool can't just send [ScatteredBlockBytes which consists of hashMerkleRoot (which depends on generation tx which in its turn contains extraNonce) + block time] because once Nonce is completely scanned (and it happens really fast) it will trigger a change of extraNonce or block time so ScatteredBlockBytes will change too. Uffffff.... So no, no need to store flood-data inside blocks, it's only needed for block template. And we are not wasting disk space in this case.
 
Hadn't thought of that, mining as a multi-stage process requiring different hardware for each stage. I could see it being very difficult getting the reward structure right though, so each kind of number crunching yielded close to the same margins, as hardware for each stage evolved and changed.

I think if that kind of architectural change was to be considered, the solution I linked in the OP would be simpler, just make pool mining painfully costly for both miners (who used pools) and pool ops to encourage solo mining instead. From my understanding the algo could be kept the same, it doesn't matter, it would just be the block structure that was altered.

Wish I'd started learning more about this stuff before senility set in...
Read this about turning mining into a 2 phase process:

http://hackingdistributed.com/2014/06/18/how-to-disincentivize-large-bitcoin-mining-pools/

For the second phase you need to use your private key, so pools can't rely on other people's miners. They need their own hardware for that and that limits their growth. The second phase can require less hash power than the first so they only need a fraction of the total hash power under their direct control. That difference in difficulty could start small and increase with time to give time to people to adapt.
 
Read this about turning mining into a 2 phase process:

http://hackingdistributed.com/2014/06/18/how-to-disincentivize-large-bitcoin-mining-pools/

For the second phase you need to use your private key, so pools can't rely on other people's miners. They need their own hardware for that and that limits their growth. The second phase can require less hash power than the first so they only need a fraction of the total hash power under their direct control. That difference in difficulty could start small and increase with time to give time to people to adapt.
Thanks fernando, great read!

It had occurred to me to use the Masternodes as the 2nd stage, with paramaters adjusted to make it computationally 'easy' so that the MNs wouldn't have to be CPU/GPU/Whatever powerhouses, but then I thought, why require MNs to 'mine' at all - mining is largely make-work, the MNs could just require the actual miners privkey and sign accordingly, thus cutting the pool out of the payment loop and benefiting the miner directly... if that makes sense?

The tuning mechanism they propose sounds straightforward - ideally I'd like to see no one pool exceed 5% max of the total nethash. (I appreciate that their approach does not counter large solo mining farms.) If the inherent advantage of Masternode-subsetting could be integrated as well, we'd have a very expensive system to mount any attack against. In fact the only remaining cheap/easy attack vector would be the lack of diversity in MN VPS providers.

:smile:
 
Thanks fernando, great read!

It had occurred to me to use the Masternodes as the 2nd stage, with paramaters adjusted to make it computationally 'easy' so that the MNs wouldn't have to be CPU/GPU/Whatever powerhouses, but then I thought, why require MNs to 'mine' at all - mining is largely make-work, the MNs could just require the actual miners privkey and sign accordingly, thus cutting the pool out of the payment loop and benefiting the miner directly... if that makes sense?

The tuning mechanism they propose sounds straightforward - ideally I'd like to see no one pool exceed 5% max of the total nethash. (I appreciate that their approach does not counter large solo mining farms.) If the inherent advantage of Masternode-subsetting could be integrated as well, we'd have a very expensive system to mount any attack against. In fact the only remaining cheap/easy attack vector would be the lack of diversity in MN VPS providers.

:smile:
I think that most of the approaches discussed make some sense, but I'm not sure what would be easier to implement. I like the idea of using the masternode network somehow. It would make it more expensive to attack and more difficult to copy.

The lack of diversity in VPS providers also worries me. It is difficult to fight it though, big players have great prices and functionality, so people flock to them. Interestingly, I think that some big VPS providers are not being widely used. I can't be sure now because we don't have chaeplin's site any more, but I think that I didn't see in the VPS split Rackspace, HP or Azurre with an important percentage of nodes. All three of them have generous free trials, so it would make sense to use them more.

Edit: crowning has done now the ISPs page and I can confirm that there are zero Azurre and HP nodes, and two Rackspace ones:
http://178.254.18.153/~pub/Darkcoin/masternode_locations.html
 
Last edited by a moderator:
Back
Top