• Forum has been upgraded, all links, images, etc are as they were. Please see Official Announcements for more information

Official Pre-Proposal: Proposal Evaluation Committee

Will this be beneficial for Dash?

  • Total voters
    39
Proposal Evaluation Committee

Been a hectic week. The PEC got a lot of flack for a number of mistakes I made:

1) When I wrote the Guidelines I decided that when we strike a red flag situation we’ll stop all further effort on the Report: Saving us time and saving the community and MNO’s reading time. Wrong: The 0% reports ignored one of our core directives: to educate the PO in what will make a MNO-friendly proposal and the harshness of the empty report combined with a 0% could scare off potential proposals etc etc – Rectified

2) In the guidelines I placed comments like: “Proposal is childish and immature” and some of the PEC members blasted me. In revenge I gave them the most illogical and messy parts of the Guidelines to rewrite. From what I’ve seen so far, they are doing an excellent job.

3) When writing the guidelines I kept on forgetting the ‘helpful’ aspect of the PEC and slipped into the ‘pitbull – protect’ mode, resulting in guidelines and later directives in our slack forcing evaluators to sound judgmental instead of leaving the judgement to the MNO’s. Rewrites should rectify this.

The PEC is adapting to PO, community and our own input. This is a test run (TG) and we are learning fast.
These guys and lady are working their back sides off for free (for now) and we only have one real purpose:
To try and make Dash stronger
 
MizzyMax said:
Personally the idea of 6 people swaying the way they want to take Dash is not the way to go. Bias is in all of us let's be honest, there is no organization that is not bias towards some group of people, just like certain news stations are republican favored and some are democratic favored. This creates the faulty system we have now with corruption, down the road it could easily play out as someone has friends in the committee or even bribes the committee to give positive feed back. This could also have the ability to ruin the credibility of a perfectly good proposal because of greed. (Ex. Dash force has new competition putting in a proposal. Dash force bribes the committee and the committee spreads rumors and lies about the legitimacy of the new proposal ending up in the failure of a proposal passing)
...
Now if you want to be a group of people who (for free) go around giving feed back on proposals (like the others) Then that would be great, but creating a centralized group of people to help us vote is definitely not in mind.
Click to expand...

Mizzy, it takes guts to stand out and say something against the crowd. But what you said needs to be heard and I'm glad you said it.

My main question is: Why should I pay for someone else to do another's analysis?
It's a similar problem with taxation: Person1 takes money out of person2's pocket for person3's benefit. The further the money is from your control, then the greater the potential for corruption, inefficiency, and bad outcomes. In this case, the further the analysis is from your own, the greater the potential for misinformation, corruption, bad decisions.

So, I'd say: Either you pay someone to do your own analysis, you get together with other MN's to pool your resources and fund your analysis as a group, or the analyzers volunteer for free. Please don't use public funds to further centralize the group to do analysis that I didn't ask for, and on top of it give that group undue influence on the wider group.

On the other hand if this "committee" were simply an optional service for proposers to help them make better proposals with better chance of approval, then OK I would actually vote yes to fund a group to do that. Many proposals are of poor quality, and I would fund a 3rd party to help improve that quality. This is a very different flavor than a committee that gives ratings to influence the greater group - that is ripe for corruption if not now then eventually, as MizzyMax clearly points out. And we'll be stuck with this model that will be difficult to change.

1. I somewhat agree with GrandMasterDash: if we're doing something like this then the name "group" is better than "committee" and having MULTIPLE different groups - this gives more perspectives and competition, but still falls into the problem like the multiple ratings agencies that failed in the last financial crisis.

2. Also sorta agree with jimbursch: A "seal of approval" is a better way to go than "rating". It's a positive stamp that someone can display, like Organic, or GMO-Free, or ADA seal of approval. Better still, are positive stamps given by independent groups, not one funded by the network.

I have to add, and wish I could convey it better - I want to make clear to recognize Bitlong and other evaluator efforts - I do trust that your intentions are for the best of the network in mind. I hope you take my criticism and use it in as good of a way as possible. My advice is to restrict the "PEC" duties to helping proposers come up with better ways to present their proposal, and things along those lines. And if you see that a certain proposal goes forward with clear issues for whatever reason, then you voice those concerns, as an individual, not with the weight of an "official committee rating"
 
Last edited:
Hi Mizzy..sorry Geo - see you just joined us here on the Forum a couple of minutes ago - Welcome ;)
The centralization issues were discussed in detail a couple of months ago. Here's one quote from April:
"all those who worry about this project leading to centralization - A reminder: Committees were catered for right from the start. See Forum: https://www.dash.org/forum/threads/budget-system-funding-voting-dgbb.8762/ (created by the MNO's themselves a year? or two? ago):
"Managers, Committees ….
Some oversight will be needed for the blockchain contractors. Each budgeted item will require either a team manager or a committee. Periodically the managers will write what the person being paid is working on and show the value created from the funds that were allocated.

... Committees will be required to be completely transparent and post updates about the progress they’re making and where the money is going.

In the case where a committee isn’t being transparent or is not posting updates, the community will begin to inquire about the status of the project. If no satisfactory answer is given, support will be lost, masternodes will change their vote and the project will drop off of the budget. In a way this system should be self policing."
 
tungfa said:
did not get anything
is that a new /different slack ?
Click to expand...
Just posting so that it doesn't look like I'm ignoring you. Tungfa has been visiting already
Yes - another bloody slack... way too many as is. Had no choice since the officials slack is full - messages get deleted and to upgrade would cost $10 a head!
Needed something where our discussions and documents would be safe.
 
Biltong said:
Hi Mizzy..sorry Geo - see you just joined us here on the Forum a couple of minutes ago - Welcome ;)
The centralization issues were discussed in detail a couple of months ago. Here's one quote from April:
"all those who worry about this project leading to centralization - A reminder: Committees were catered for right from the start.
... Committees will be required to be completely transparent and post updates about the progress they’re making and where the money is going."
Click to expand...

Yes, I just joined the forums because I felt this was an important enough issue to discuss and I couldn't comment without joining. I'm traderpat on reddit and henrygeorgist on dashcentral. I've objected to the idea of PEC from the beginning. You can see my post on reddit here. (Ok I can't post links because I haven't made 3 posts yet, but see the thread on How will Dash deal with illinformed MN voters?). Well technically I didn't exactly object to PEC, I objected to the idea of the PEC being the ONLY solution. I said a committee could be one (hopefully a minor one) of multiple ways of how to help MNO's make better decisions.

How does "being transparent" solve the issue of centralization? The point still remains that the network is funding a small group of people to give undue influence on everyone's decision-making. These are valid concerns. Please don't brush them aside.

Why not just help proposers make better proposals? That is a valid and needed service that I would support. If a proposal should get a poor rating, then you can state the problems on the forums/dashcentral/wherever as an individual. Having an official-seeming single "rating" where people can mindlessly decide (effectively giving away decision-making power) is the issue that people have who fear this leads to corruption.
 
Please let the "wisdom of crowds" work. I still can't post links since I don't have 3 posts, but look up "The Wisdom of Crowds" on Wikipedia.

Four elements required to form a wise crowd:
1. Diversity of opinion
2. Independence
3. Decentralization
4. Aggregation

Failures of crowd intelligence:
1. Homogeneity
2. Centralization
3. Division
4. Imitation
5. Emotionality

If you want to go forward with the proposal, limit it to helping proposers (who want it) to making the best out of their idea, and presenting it in the best way that your experience allows. That is a very valuable and needed service. Then let the wisdom of crowds happen.
 
Hi Geo
Good points. Oh - and I did find you on reddit and in fact I totally agree with most of what you say there.: Your bold: the problem is we don't have enough good information to make a good decisions - Yes - totally agree.

And it gets even better:
You said:
"Ok... but how do you become more informed? (1) Do you read up on everything and contact the individuals directly yourself? (2) Do you outsource that to an audit committee? (3) Do you want the information to be presented in a way that you can become better informed?

Some people will do (1) but very few... that leads to a small proportion of informed electorate... maybe okay in the beginning. Many here are recommending (2) which is fine... I believe it's necessary at some point to some degree, but it's not the only way to become more informed.

I believe we can make better decisions if (3) certain information was presented better ... if proposals were more standardized so they could be more easily understood. If more information about proposers were readily available. If the info about how proposal money is being spent were better understood and how overall budget is being proportioned. And what are the results of each project? What about success rates for overall projects this month, year, etc? you can probably find out all or most of this information if you work hard enough. But if it's made easier to get this information it would go a long way to make more MNOs better informed, don't you think?"

Yes - I do :)
 
Last edited:
It's getting late, but I googled your: The wisdom of crowds and found a riposte from the BBC right below it with one excellent paragraph which I found quite funny:

http://www.bbc.com/future/story/20140708-when-crowd-wisdom-goes-wrong

No one should need warning about the dangers of herding among poorly informed decision-makers: copycat behaviour has been widely regarded as one of the major contributing factors to the financial crisis, and indeed to all financial crises of the past. The Swiss team commented that this detrimental herding effect is likely to be even greater for deciding problems for which no objectively correct answer exists, which perhaps explains how democratic countries occasionally elect such astonishingly inept leaders.

That in a nutshell is the PEC's job - to provide enough information for the MNO's to make informed decisions
 
Biltong said:
to provide enough information for the MNO's to make informed decisions
Click to expand...

The danger is that the PEC can present wildly inaccurate and misleading information, which was my experience.

That danger can be worked out in the pre-proposal phase, but it can be disastrous if it happens days before the voting deadline, which is what happened in my case. Luckily for me, it appears the MNOs discounted the PEC report and my proposal passed by a wide margin.

In a nutshell, the PEC can make big mistakes like anybody else, and should be held in the same regard as anybody else.
 
jimbursch said:
The danger is that the PEC can present wildly inaccurate and misleading information, which was my experience...
Click to expand...
Hi Jim
That is quite a statement. She was handicapped before, because you provided her with information making her promise that she won't release it, but after a statement like that she is obviously released from that promise. She must be able to defend herself.
She is in hospital at the moment, but she'll be back in a couple of days and I'm sure she would love the chance to explain her decisions.
 
This is exactly why the PEC cannot be trusted.

She can explain her math without violating confidentiality and I can explain why it is wrong without violating confidentiality. That is what I did here:
https://www.dash.org/forum/threads/...-program-by-bugcrowd.15321/page-2#post-131230

For you to arbitrarily declare that someone is no longer bound by their confidentiality agreement is irresponsible. You had the opportunity to address this issue in a responsible manner when I invited anyone to contact me directly and I would share the same information that I shared with TallyHo, subject to the same confidentiality.
 
jimbursch said:
This is exactly why the PEC cannot be trusted.

She can explain her math without violating confidentiality and I can explain why it is wrong without violating confidentiality. That is what I did here:
https://www.dash.org/forum/threads/...-program-by-bugcrowd.15321/page-2#post-131230

For you to arbitrarily declare that someone is no longer bound by their confidentiality agreement is irresponsible. You had the opportunity to address this issue in a responsible manner when I invited anyone to contact me directly and I would share the same information that I shared with TallyHo, subject to the same confidentiality.
Click to expand...
It shows bias. Plain and simple. Obviously tally didn't like your proposal, and probably voted no. So she made an evaluation that showed that bias clearly.

You can fudge these numbers around easily to make it seem ok to give low scores in certain areas when it shouldn't be. For example. Experience, 7/10 that's a 70% yet "you are more then qualified to do the job" as the committee member mentions. Along with technical experience 6/10 60% yet you have created a proposal site on your dime. It shows plenty of technical experience already in that project. And risk vs reward 1/15 6%, let's be honest, it's a lot more risky to have a non functional wallet... Dash could drop in half if it happened. And responsiveness 1/5 20% shouldn't even be a rating. Let's be honest, your judging the proposal and only the proposal. If I choose not to communicate to the pec, I automatically drop my rating by 12%? maybe I don't support them yet it forces you to comply.

That was a large proposal as well, in the top 3 and was such a terrible review. Very poorly managed and bias claims were made I feel. No trust
 
Biltong said:
It's getting late, but I googled your: The wisdom of crowds and found a riposte from the BBC right below it with one excellent paragraph which I found quite funny:

http://www.bbc.com/future/story/20140708-when-crowd-wisdom-goes-wrong

No one should need warning about the dangers of herding among poorly informed decision-makers: copycat behaviour has been widely regarded as one of the major contributing factors to the financial crisis, and indeed to all financial crises of the past. The Swiss team commented that this detrimental herding effect is likely to be even greater for deciding problems for which no objectively correct answer exists, which perhaps explains how democratic countries occasionally elect such astonishingly inept leaders.

That in a nutshell is the PEC's job - to provide enough information for the MNO's to make informed decisions
Click to expand...

Biltong,
Thanks for taking the time to read some of my responses on reddit and looking more into "Wisdom of Crowds" and responding here. I can tell you've put in a lot of time and effort on preparing this "committee" (group, sub-dao, whatever), the write-ups are very detailed, and organizing the evaluators. Again, I also believe you have Dash's best interests at heart. Even though I'm not in favor of the idea in general, I want to make sure you know your efforts are appreciated.

That's a pretty good article from the BBC that you linked. You even quoted a pertinent part: copycat behaviour has been widely regarded as one of the major contributing factors to the financial crisis

I believe one simple rating will further encourage copycat behaviour and biases. Instead of getting "the group's" wisdom, you're getting the evaluator's.

More from the BBC article: He explained that one requirement for a good crowd judgement is that people’s decisions are independent of one another. If everyone let themselves be influenced by each other’s guesses, there’s more chance that the guesses will drift towards a misplaced bias.

Furthermore, if I had my way I would like to even hide "number of yes votes vs no votes" on a proposal. This is a case of more info not being better, since that doesn't contribute to better decision-making. I know that seeing earlier voters giving a yes/no will influence a later voter's decisions to vote yes or no. It shouldn't, but it does. So one way to improve the voting process would be to hide that by default (or we could publicly remind everyone to vote based on the content of a proposal, before looking at the vote tally, but this seems more difficult to address and a bit off-topic).

More from the BBC article: Could there also be ways to make an existing crowd wiser? ... you should add random individuals whose decisions are unrelated to those of existing group members. That would be good, but it’s better still to add individuals who aren’t simply independent thinkers but whose views are ‘negatively correlated’ – as different as possible – from the existing members. In other words, diversity trumps independence.

I look forward to Evolution and the ability for any holder to vote without the 1000-Dash limit - That will give even greater diversity in voting!
 
Biltong said:
Hi Geo
Good points. Oh - and I did find you on reddit and in fact I totally agree with most of what you say there.: Your bold: the problem is we don't have enough good information to make a good decisions - Yes - totally agree.

And it gets even better:
You said:
"Ok... but how do you become more informed? (1) Do you read up on everything and contact the individuals directly yourself? (2) Do you outsource that to an audit committee? (3) Do you want the information to be presented in a way that you can become better informed?

Some people will do (1) but very few... that leads to a small proportion of informed electorate... maybe okay in the beginning. Many here are recommending (2) which is fine... I believe it's necessary at some point to some degree, but it's not the only way to become more informed.

I believe we can make better decisions if (3) certain information was presented better ... if proposals were more standardized so they could be more easily understood. If more information about proposers were readily available. If the info about how proposal money is being spent were better understood and how overall budget is being proportioned. And what are the results of each project? What about success rates for overall projects this month, year, etc? you can probably find out all or most of this information if you work hard enough. But if it's made easier to get this information it would go a long way to make more MNOs better informed, don't you think?"

Yes - I do :)
Click to expand...

Thanks. So... when I wrote those responses, I was thinking about how to make MN Operator's better informed *before* needing to resort to a committee to do it.

If the PEC were to simply help proposal owners present the info better, that's a type of proposal I would vote yes for. You'd be providing a valuable service to help jumpstart and get proposers doing "better" proposals - in effect, we'd be subsidizing the creation of better proposals (if your "committee" makes it more likely a proposer puts up a proposal that gets a yes vote). What I'm more hesitant about is a PEC that gives ratings that easily influences/biases voters.

The ideal solution I was thinking of when I said "presenting better info to MNO's" refers to the type of data that can be presented automatically. That way there's no human bias. It's more objective, since it's data an algorithm can put together - it just needs to be presented well. Reputation (e.g. we can see how many proposals this person has submitted, how much they've asked for, and the types of results, and maybe up/down votes like amazon/yelp/reddit), Transparency (e.g. are they spending the money the way they say they are), Budget proportions (e.g. what category is this? marketing? development? and how many of these proposals do we have already?). Technically all that data (and more) exists and is objective - it just needs to be put together and presented well. That's very different than a single person/group doing analysis and giving a final rating that a voter can be influenced by without thinking.

Sorry that was a bit off topic. Anyway, I hope you see my concerns:
  • Helping proposal owners present their proposal better = A-OK, maybe great!
  • Single rating that biases voters = No-go, please find a different solution.
 
jimbursch said:
She can explain her math without violating confidentiality and I can explain why it is wrong without violating confidentiality. That is what I did here:
https://www.dash.org/forum/threads/...-program-by-bugcrowd.15321/page-2#post-131230
Click to expand...

Thank you, I shall. I worked my way through your cost breakdown and explanation with an Excel spreadsheet, verifying as I went. The numbers added up (regardless of what proportion went where) until the final stage, where in your explanation:

"The total came out to roughly $xxx, which I then converted to a number of Dash, which was then rounded to a number divisible by 3 in order to make 3 monthly payments."

At this point, my calculation agreed with your "roughly $xxx" total in that I had $xxx - $3,500. However, this "rounding to a number divisible by three" resulted in an increase of another 66 Dash.

I raised this concern in my reply to you and you failed to address it. Calculated using the same exchange rate specified in your Pre-Proposal of $180 per Dash, this 66 Dash + $3,500 results in a rounding error of $15,500.

This on top of the 25% already added "to cover exchange-rate risk and fees for DashBudgetWatch" means that over 27% of the total funds requested are for no clear purpose. And in spite of a quarter of the total funds requested already being buffer money, you still raised your proposal by yet another 90 Dash.

Regarding my presenting "wildly inaccurate and misleading information", your email to me included not only the quote from Bugcrowd but your own explanation of the quote. Again, I quote without disclosing the figures:

"Item 1: BugCrowd Reward Pool, $xxxxx -- This is the pool of funds out of which bounties are paid. At this time it is impossible to say how much of this fund will be expended over 12 months. If P1 vulnerabilities (see the VRT attached) are discovered, then this fund will be depleted and it may need to be topped up. "

If this figure was not intended to be the actual amount of the bounty pool and the figure upon which the rest of your proposal was calculated, this was not clear, or even suggested, from either the Bugcrowd quote or your own explanation. It was only after I expressed concerned that this was too small a portion of the total funds requested that you introduced the different options for the size of the bounty pool, interestingly none of which included this one.


Furthermore I take great objection to your remark that "the PEC cannot be trusted". I have gone to great lengths to respect my promise of confidentiality, even though it prevented me from making a true objective evaluation of your proposal and it cripples me from adequately defending myself now. It was a valuable learning experience though since I will never accept an offer of private information again.
 
@Tallyho -- you are absolutely to be congratulated on your integrity and commitment to your promise of confidentiality. The information I shared with you is covered by a non-disclosure agreement that BugCrowd required, and I am sorry it has put you in this awkward position. I have learned that I should not have shared that information with you. The leader of the PEC would have you violate your promise, which is why I say the PEC cannot be trusted.

I believe it was wrong for you to use the information provided to conjure a scenario in which, you claimed, only 13% of the proposal would be used to fund the reward pool, and then proceed to insult my integrity by calling the proposal disingenuous and deceitful, after I went above and beyond what I should have done to be the opposite of deceitful.

Tallyho said:
over 27% of the total funds requested are for no clear purpose.
Click to expand...

100% of the proposal is for one clear purpose: to implement a bug bounty program for Dash. The mistake I believe you made was to interpret the BugCrowd numbers as fixed costs that can be plugged into a spreadsheet for the budget proposal. Those numbers inform my negotiation with BugCrowd, but they do not represent how the funds will be finally allocated. This is a proposal for a 12-month program funded over the first 3 months in a highly dynamic financial and technical environment. I would be a fool not to write this proposal in a way that maintains maximum flexibility to get the best deal for Dash.
 
I like the idea but am concerned there's no marketing people to evaluate marketing proposals.
Ask for one person to drop out or share a bigger pie with the MBA?
Or if you're confident in their abilities, share the evaluations they did for you.

Perhaps marketing guy @craigums could volunteer for a month evaluating proposals?

If this is done well, there will be lot of value added to MNOs and DASH.
 
You must log in or register to reply here.
Back
Top