Welcome to the Dash Forum!

Please sign up to discuss the most innovative cryptocurrency!

Proposal: Infrastructure - Basic Needs

Discussion in 'Pre + Budget Proposal Discussions' started by Ryan Taylor, Jun 19, 2016.

  1. Ryan Taylor

    Ryan Taylor Well-known Member
    Dash Core Team Foundation Member

    Joined:
    Jul 3, 2014
    Messages:
    509
    Likes Received:
    1,597
    Trophy Points:
    263
    This is a cross post from www.dashwhale.org/p/infra-basic-needs

    This proposal is meant to finally address the minimum ongoing infrastructure requirements of the Dash project. Coming out of our recent review of our needs, infrastructure was one area that clearly required some investment. Critical Dash assets are being hosted on private servers, donated servers, etc. In addition, the provider of our Bamboo Cloud – which we use to compile all versions of Dash – announced on May 24th that the product will be discontinued. Needless to say, as the project grows and matures, our infrastructure needs are changing.

    In an ideal scenario – and hopefully in the long-run – we would like to host Dash infrastructure on dedicated equipment, but that would be cost-prohibitive at our current scale. However, there is much that can be done to improve our infrastructure without a great deal of investment.

    First, we need to replace the functionality of the discontinued cloud Bamboo service. After evaluating our options, we propose purchasing our own hardware. By owning our own server, we can compile new versions much more efficiently, which accelerates our automated testing (continuous integration). This will make all of our development teams more efficient, especially during the testing phases of each release. As the development efforts ramp up the second half of this year, having a dedicated Bamboo server is becoming only more critical, and will address a pain-point for our development efforts.

    For other infrastructure needs, we feel that we can continue to operate using cloud-based infrastructure, which has lower upfront costs, more flexibility to add or remove capacity, and yet is still hosted in a professionally managed datacenter. It would also ensure that these assets are under the control of the project, as opposed to individuals who are presently donating capacity.

    A special thanks to @moocowmoo for leading this effort and taking on our infrastructure planning and management.

    Here are the details on the hardware and services we propose as part of our minimum ongoing infrastructure needs. Please note that when 12.1 is deployed a separate proposal will be needed to fund the ongoing costs. This proposal covers only one-time costs and first month’s expenses.

    Compile/Deploy/Backup server
    Budget:
    $3000 for hardware (one-time cost)
    $150 per month for co-location and replacement parts
    Server’s roles:
    - Continuous Integration (Bamboo slave)
    - Private Docker Registry (generate/deploy production images)
    - Online Incremental Backup (first tier hourly backup)
    Specifications:
    (subject to change according to available deals at the time of purchase and USD available based on the Dash exchange rate when paid)
    1U enclosure, redundant power supply
    1x Motherboard (TBD), 2x 1Gb NIC
    2x 8-core AMD Opteron
    128G RAM
    2x 1TB SSD
    2x 6TB HDD

    Cloud infrastructure costs
    Budget:
    $250 per month for AWS and Google services and administration
    Roles:
    - AWS : Route53 DNS zone management
    - AWS : EC2 web server instances
    - AWS : S3 for web server instances
    - AWS : Backups and auditing
    - Google: Gmail for domains

    Requested funding is as follows for the July 6th budget cycle:

    • 362.67 Dash for Bamboo server ($3,000.00 USD @ $8.272 per Dash based on June 18th average rate at https://bitinfocharts.com/comparison/price-dash.html)
    • 18.13 Dash for first month’s Bamboo server colocation hosting and parts ($150 USD)
    • 30.22 Dash for first month’s cloud infrastructure ($250 USD)
    • 5.00 Dash reimbursement for the proposal cost

    Total: 416.02 Dash

    Note: Should any funding remain after the project is complete, we will apply those funds toward future infrastructure funding needs

    Manually vote YES on this proposal:
    dash-cli mnbudget vote-many 6a67175f7a47b09ebd1852e1b6a84ade77e475a818a90e577a074c694a4efaa1 yes
    OR from the qt console:
    mnbudget vote-many 6a67175f7a47b09ebd1852e1b6a84ade77e475a818a90e577a074c694a4efaa1 yes


    Manually vote NO on this proposal:
    dash-cli mnbudget vote-many 6a67175f7a47b09ebd1852e1b6a84ade77e475a818a90e577a074c694a4efaa1 no
    OR from the qt console:
    mnbudget vote-many 6a67175f7a47b09ebd1852e1b6a84ade77e475a818a90e577a074c694a4efaa1 no
     
    • Like Like x 3
    • Agree Agree x 3
    • Informative Informative x 1
  2. fible1

    fible1 Well-known Member
    Dash Core Team Masternode Owner/Operator

    Joined:
    May 11, 2014
    Messages:
    711
    Likes Received:
    722
    Trophy Points:
    163
    Where will the server be hosted? Who will be carrying out the build?

    Thanks,
    Pablo.
     
    • Agree Agree x 1
  3. Ryan Taylor

    Ryan Taylor Well-known Member
    Dash Core Team Foundation Member

    Joined:
    Jul 3, 2014
    Messages:
    509
    Likes Received:
    1,597
    Trophy Points:
    263
  4. fible1

    fible1 Well-known Member
    Dash Core Team Masternode Owner/Operator

    Joined:
    May 11, 2014
    Messages:
    711
    Likes Received:
    722
    Trophy Points:
    163
    I voted yes, I'm happy knowing that @moocowmoo is doing it.

    My main concern is that we should keep our startup spirit and be as frugal as possible, at least until we breach the billion market cap, and perhaps beyond, regardless of available funding. There have been a couple of proposals come forward recently. requesting funding for small stuff the community used to do for free, such as community management, and now server co-allocation. We are only talking a few hundred bucks, but once we stop being frugal and stop thinking like a young startup, we will start making unnecessary expenses in larger ticket items. I would rather we have funding left over every month than spending a chunk of it on stuff we have always donated to the community, be it our time or hosting space.

    Hope that makes sense,
    Pablo.
     
    • Agree Agree x 5
    • Winner Winner x 1
  5. Balych

    Balych Active Member

    Joined:
    Sep 12, 2015
    Messages:
    365
    Likes Received:
    211
    Trophy Points:
    113
    Dash Address:
    Xba1ychX7CjgbRrCKE1LjHjT3jLUhcexs5
    [offtopic]
    Why not Tutanota/Protonmail/etc?
    [/offtopic]
     
  6. balu

    balu Well-known Member
    Dash Core Team Foundation Member Dash Support Group

    Joined:
    Oct 9, 2014
    Messages:
    117
    Likes Received:
    239
    Trophy Points:
    213
    Hey,

    While I agree about the topic of the proposal, I'd like to recommend an alternative solution to the colocated server.
    Since we're trying to act like a startup, we should totally go with renting a server instead of buying one.This would cost less, is better cash-flow wise (no initial investment), would require no hw maintenance (less burden on our core people), and will enable us to upgrade later way easier.

    The things we listed above do not justify a 2-core colocated server with 3k initial investment. I don't want to link exact offers, but compared to a very reputable host like LeaseWeb, what advantage would we get with the solution above? Please compare and evaluate. I think the requirements above could be handled by a 70-80 USD/month server without any upfront cost, and later if required we could upgrade to a dual-cpu one still within 150 usd/month (and without the initial 3k investment)

    Thanks for the consideration and answer.
     
    • Agree Agree x 4
    • Informative Informative x 2
    • Like Like x 1
  7. crowning

    crowning Well-known Member

    Joined:
    May 29, 2014
    Messages:
    1,428
    Likes Received:
    2,005
    Trophy Points:
    183
    I remember the discussion over at Slack, but it's not there any more: what was the reason why we don't run our own mail server?
     
    • Informative Informative x 2
  8. TroyDASH

    TroyDASH Well-known Member
    Moderator

    Joined:
    Jul 31, 2015
    Messages:
    1,251
    Likes Received:
    793
    Trophy Points:
    183
    You make a good point. It can be difficult to judge whether the time and resources someone is putting into the project should be considered a hobby/volunteer work, vs. when it starts meriting a paid job. In the end I think it should still come down to a cost benefit analysis. What tangible things would happen or not happen depending on whether the proposal gets funded? Will the project receive a marginal benefit that is greater than the marginal cost, and greater than the net marginal benefit of other competing proposals? If the answer is yes, then that's how I vote. Proposal owners would do well to attempt to help answer those questions as well without MNs having to guess or pry it out.
     
    • Agree Agree x 1
  9. n00bkid

    n00bkid New Member

    Joined:
    Oct 25, 2015
    Messages:
    26
    Likes Received:
    38
    Trophy Points:
    13
    If a new development team ever gets voted in, what happens with the $3k hardware?

    I agree with some others that it would be preferable to keep these operations on a host. That way these operations can be kept flexible in terms of who runs them.
     
    • Like Like x 1
    • Agree Agree x 1
    • Winner Winner x 1
  10. aka5

    aka5 Active Member
    Foundation Member

    Joined:
    Mar 22, 2015
    Messages:
    17
    Likes Received:
    5
    Trophy Points:
    103
    I totally agree with that, in my eyes renting a server is much better than buying one - for cheap high end ones look hetzner.de for example ... the problem i see with buying one is, who is the owner? it's hard today to let the blockchain own the server, so someone has to be the strawman, but that's not a good solution - renting would be much better !
     
    • Agree Agree x 1
    • Winner Winner x 1
  11. GrandMasterDash

    GrandMasterDash Well-known Member
    Masternode Owner/Operator

    Joined:
    Jul 12, 2015
    Messages:
    2,666
    Likes Received:
    953
    Trophy Points:
    183
    I totally don't get this proposal, imo it's ridiculous. We have 3800 servers and you want us to rent out someone else's resources... eeewwww. I think we should be eating our own dog food.
     
  12. moocowmoo

    moocowmoo Bovine Bit-flipper
    Foundation Member Masternode Owner/Operator

    Joined:
    Jun 15, 2014
    Messages:
    483
    Likes Received:
    603
    Trophy Points:
    263
    Dash Address:
    XmoocowYfrPKUR6p6M5aJZdVntQe71irCX
    We haven't decided yet, but either I or @flare will be performing the build and co-locating the server near whomever builds it.

    Thank you for saying that, I appreciate it!

    That is the goal in mind. I've scratched out plans for our billion-cap dream infrastructure, but would like a single, powerful box to handle most of the backend administrative processing in the mean time.

    I considered dedicated packages, but prefer the isolation that purchasing our own hardware affords.
    One of my goals in building the new infrastructure is high security with as little exposure to logical and social attack vectors as possible.
    I lean a bit extreme on the paranoid side, but it's served me well over the years.

    My intent with this first system is to build a secure base that compiles and deploys all artifacts and services to cloud instances using ipfs, docker, and rancher.
    With so much control in one system, it will only be accessible over a vpn, probably cjdns and only by a few community-trusted people. @eduffield, @flare, @moocowmoo, and maybe a few others that need it. (For Evolution for example)

    As far as upgrades go, these specifications will suffice as a build system for years and years to come. I expect builds to take under five minutes per platform serially, with the capability to run several gitian builds in parallel, probably 3 at a time before any resource contention. This will allow us to go from commit to test suite completion to development version installation (automated mass testnet deploy) within a few minutes.
    I've spec'd this box fast and powerful to supercharge development cycles.
    I've also spec'd it huge (drive space) to be the first level of proper incremental backups. Not to mention docker images aren't tiny at the repository-mirror level.

    I like protonmail, we may end up there.
    But for now we've chosen google to unify administration of all the google accounts for google docs (which we use extensively).

    Because it's easier to farm it out for now.
    Running postfix/dovecot/spamassasin is easy enough, but providing a web interface isn't a priority at the moment.
    As far as third-party exposure goes, any communications that are sensitive are protected by gpg.
    Additionally we'd need several ip's for PTR records to guarantee 100% delivery for our multiple domains. Forwarding it solves this issue for the time being.

    I consider this server to be a step toward unifying the administration of the entire dash ecosystem.
    If a new development team were to be voted in, I'd hope they'd want to write code and leave the administration to someone already doing it. (Assuming I'm doing a good job, hehe.)


    ---

    This path forward is how I've chosen to evolve our administration from a single openvz vps to something dynamic and manageable.
    Owning our own hardware guarantees that we always have a stable, secure base to manage all of dashs services and needs.
    Everything else will be reproducible, deterministic cloud deploys.

    I look forward to building us all a spectacular, reliable system! :)
     
    • Informative Informative x 3
  13. fible1

    fible1 Well-known Member
    Dash Core Team Masternode Owner/Operator

    Joined:
    May 11, 2014
    Messages:
    711
    Likes Received:
    722
    Trophy Points:
    163
    Awesome, I knew there was some good reasoning behind this choice :).

    Pablo.
     
  14. rustycase

    rustycase Active Member

    Joined:
    Apr 19, 2016
    Messages:
    497
    Likes Received:
    117
    Trophy Points:
    113
    Small disagreement...
    The server does not need to be near it's builder, it needs to be on the absolutely best, top of the heap, fiber optic trunk line.

    And, even though I favor the red team, (for my level), Intel makes superior processors, yet unfortunately commands the highest prices for them.
    (Perhaps someone has done specific detailed comparisons ?) { Xeon E3-1230 v5 }

    ...Conducting all business from one box does not seem to be optimal security, imo...

    just a few thoughts I had... Newbie kinda thoughts...
    rc
     
    #14 rustycase, Jun 20, 2016
    Last edited: Jun 20, 2016
  15. moocowmoo

    moocowmoo Bovine Bit-flipper
    Foundation Member Masternode Owner/Operator

    Joined:
    Jun 15, 2014
    Messages:
    483
    Likes Received:
    603
    Trophy Points:
    263
    Dash Address:
    XmoocowYfrPKUR6p6M5aJZdVntQe71irCX
    It is important the server is accessible when physical maintenance is needed.
    The bandwidth is not as critical as you think, as distributing binaries to the public is not its role. That's what cloud instances are for.

    AMD was chosen just because of the Intel premium you mention.

    Optimal security is having the smallest possible attack footprint. This server will never serve content to the general public.
    Optimal redundancy is another matter, and I'm making plans to have a few personal servers as hot-spare temporary failovers should the need arise while we drive to the colo.
    When the infrastructure and budget is more established, these will be dedicated co-located failovers.
    But, keep in mind, this system does not have to be online for our services to be. It is simply an administrative and content generation backend.
     
    • Like Like x 1
  16. rustycase

    rustycase Active Member

    Joined:
    Apr 19, 2016
    Messages:
    497
    Likes Received:
    117
    Trophy Points:
    113
    Well... OKay !
    Seems you are right on top of things and my thoughts had already been considered !

    Best
    rc
     
  17. nmarley

    nmarley Administrator
    Dash Core Team Moderator

    Joined:
    Jun 28, 2014
    Messages:
    360
    Likes Received:
    423
    Trophy Points:
    133
    Dash Address:
    XsUFct27gKMRZ7JFxoxcMgjaBDXndk5kXT
    Having just a single server just screams SPOF to me. Other than that, I think @moocowmoo 's plan is solid.
     
  18. moocowmoo

    moocowmoo Bovine Bit-flipper
    Foundation Member Masternode Owner/Operator

    Joined:
    Jun 15, 2014
    Messages:
    483
    Likes Received:
    603
    Trophy Points:
    263
    Dash Address:
    XmoocowYfrPKUR6p6M5aJZdVntQe71irCX
    Agreed, but I want to start small. All of the server roles have second-tier personal backups.

    Server downtime will only prevent docker instance upgrades and the to-be-built testnet deployment harnessing.
    Annoying in the middle of a testnet deploy and test iteration, but not critical.

    Compiles will happen here on my 4GHz 8 core at home as a bamboo secondary slave. Maybe other volunteers' as well.
    Backups will either be an emergency cloud instance image ready to be spun up. (It won't need to be ginormous, just fill in the gap while the server is repaired) or fail over to the second tier volunteer backup server(s). I've not settled on a backup plan yet.
     
  19. balu

    balu Well-known Member
    Dash Core Team Foundation Member Dash Support Group

    Joined:
    Oct 9, 2014
    Messages:
    117
    Likes Received:
    239
    Trophy Points:
    213
    @moocowmoo I agree with you that we need such a system, and the use cases above I can totally understand.

    My only concern is the costs. Calculating with a 3 year TCO this server would cost 8400 USD, while renting an arguably lower spec, but still good enough server from e.g. Leaseweb is ~3000 USD. That is almost 3* the costs, and even if we added a separate backup server we'd still be at half TCO.
    And on top of this we'd have lower risk and time investment because of buying a service instead of hardware.

    All the technical details are fine as you wrote them down and since I find such an environment highly beneficial I'm really glad that you took this on you, it's just about the money.
     
  20. moocowmoo

    moocowmoo Bovine Bit-flipper
    Foundation Member Masternode Owner/Operator

    Joined:
    Jun 15, 2014
    Messages:
    483
    Likes Received:
    603
    Trophy Points:
    263
    Dash Address:
    XmoocowYfrPKUR6p6M5aJZdVntQe71irCX
    Well, I'm willing to consider other options, but the "good enough" comparison you're making isn't an equivalent comparison.

    Speed is the goal. There is no way we can rent a server with the specifications I'm looking for.

    I'm still rearranging items to fit under budget (would rather have $6000 for a 4 socket, 48 core machine), but the current specs look like this:

    16 cores at 2.8 GHz -- we need massive threading to compile all the PC/Linux/Mac flavors quickly.
    64G Ram -- gitian deterministic builds need lots of room. I want more, but am having to compromise.
    2TB SSD -- Lots of very fast I/O for above. 1TB OS/Userland (low wear), 1TB compile (high wear) - the Samsung 850 Pro's I'm looking at are rated very well and are a fantastic value for the money. -- I'll probably shrink these in lieu of ram after gitian profiling.
    12TB Disk -- Lots of space for backing up every service we have and storing every compiled artifact for ipfs (internal) distribution. Am looking at WD Gold 6TB's.

    I do agree we could do it cheaper, but only by compromising attributes I wish to build upon.
     
    • Informative Informative x 1
  21. fible1

    fible1 Well-known Member
    Dash Core Team Masternode Owner/Operator

    Joined:
    May 11, 2014
    Messages:
    711
    Likes Received:
    722
    Trophy Points:
    163
    You've made a good case, and answered the most relevant questions, no need to nickel and dime it. I'm sure your proposal will pass.

    Pablo.
     
    • Like Like x 1
  22. moocowmoo

    moocowmoo Bovine Bit-flipper
    Foundation Member Masternode Owner/Operator

    Joined:
    Jun 15, 2014
    Messages:
    483
    Likes Received:
    603
    Trophy Points:
    263
    Dash Address:
    XmoocowYfrPKUR6p6M5aJZdVntQe71irCX
    Oh, I don't feel I need to defend. I just want to keep things transparent. Somebody might have good input I haven't considered.
     
    • Friendly Friendly x 1
  23. balu

    balu Well-known Member
    Dash Core Team Foundation Member Dash Support Group

    Joined:
    Oct 9, 2014
    Messages:
    117
    Likes Received:
    239
    Trophy Points:
    213
    @moocowmoo You're right, the specs are not the same. If you really feel like those specs are necessary, go for it, as such high-end spec computers are probably cheaper if colocated then rented. I'd argue that the specs you mentioned is a huge overkill, and waiting a bit more for a build to be ready does not justify the investment, but I won't.

    I know working on overspecced computers is a lot of fun, and probably the productivity gain because of this is worth the excess expenses, because you'll put in extra effort to make it work flawlessly. For a good-enough computer this extra would be missing. Now go and make it happen. :)
     
  24. camosoul

    camosoul Grizzled Member

    Joined:
    Sep 19, 2014
    Messages:
    2,187
    Likes Received:
    1,115
    Trophy Points:
    1,183
    I've got a SuperMicro 2U with 32GB ECC RAM and 8 Xeon cores. Free to a good home. If anyone cares... I don't use it anymore. It doesn't replace what you're building, but it can't hurt as a supplement. Plus, FREE! Everything is high-performance when it's free!

    One of the first platforms with real bare metal IPMI, woot! No more crying to the datacenter for a reboot, or what the error might be...
     
    #24 camosoul, Jun 22, 2016
    Last edited: Jun 22, 2016
    • Like Like x 8
    • Winner Winner x 1
  25. kot

    kot Well-known Member
    Dash Core Team Foundation Member

    Joined:
    Mar 17, 2015
    Messages:
    683
    Likes Received:
    1,834
    Trophy Points:
    263
    Thanks @camosoul ! This is very generous.
    I guess we would make a good use of this hardware - what do you think @moocowmoo ?
     
    • Optimistic Optimistic x 1
  26. moocowmoo

    moocowmoo Bovine Bit-flipper
    Foundation Member Masternode Owner/Operator

    Joined:
    Jun 15, 2014
    Messages:
    483
    Likes Received:
    603
    Trophy Points:
    263
    Dash Address:
    XmoocowYfrPKUR6p6M5aJZdVntQe71irCX
    Wow! That's generous of you @camosoul! I'll give it a home in my rack here and configure it as one of the backups until we have additional budget for more co-location. I'll pm you the details.

    Absolutely! Backup hardware is always a good thing to have!
     
    • Like Like x 5
  27. GermanRed+

    GermanRed+ Active Member

    Joined:
    Aug 28, 2014
    Messages:
    299
    Likes Received:
    109
    Trophy Points:
    113
    I did not take part in the discussion. Perhaps, preventing the DDoS attack will be an administrative burden. Protonmail got DDoS attack back in Nov 2015. I think they paid 15 BTC ransom for the first attack. I think we should have something like FlowingMail on our MN network because Gmail is going to be a hidden issue unless DASH will be eventually owned by Google.
     
  28. GermanRed+

    GermanRed+ Active Member

    Joined:
    Aug 28, 2014
    Messages:
    299
    Likes Received:
    109
    Trophy Points:
    113
    Hey, guys. I want to have a discussion about email account for DASH evolution. I remember seeing it requires a ProtonMail account which is fine for the general user. However, I have real concern about using Gmail or ProtonMail. I have been searching for a solution. Personal email server will not work for people have no resource to handle the DDoS attacks. I came across the idea of FlowingMail and DaveMail. But, they are still in their infancy stage and nobody knows they will work well until we have something to test it. Then, I came across Cassiopeia's BitMail today. Could we have some discussion over this email topic? Should I start a new thread?
     
    • Like Like x 2
  29. kot

    kot Well-known Member
    Dash Core Team Foundation Member

    Joined:
    Mar 17, 2015
    Messages:
    683
    Likes Received:
    1,834
    Trophy Points:
    263
    Hi @GermanRed+
    There will be no requirement to have a ProtonMail or Gmail account (or even any email account) to use Evolution. It will be one of the available options.
     
    • Like Like x 1
    • Informative Informative x 1
  30. GermanRed+

    GermanRed+ Active Member

    Joined:
    Aug 28, 2014
    Messages:
    299
    Likes Received:
    109
    Trophy Points:
    113
    That's good to know. When will we see Evolution coming out? I have to admit that I have been very lazy not reading/following the development here recently. After all, my MNs are running happily and stably so why bother to read the forum. But, then I wonder what other altcoins are doing so I did some readings on their DApps. It seems there are many altcoins with DApps now. I don't know how many are their own ideas but it seems to me that many are what we had discussed in the DASHTalk forum here. The main advantage of DASH I observed (correct me if I am wrong here) is that the DASH network is much bigger than other altcoins focusing on DApps similar to those we have discussed at DASHTalk. Do we have some real statistics to show potential new DASH investors that before we launch Evolution? I forgot to bookmark that site that shows how many real full nodes or 2-nd tier nodes are there for altcoins.
     

Share This Page