• Forum has been upgraded, all links, images, etc are as they were. Please see Official Announcements for more information

How to gitian build dash under Debian Jessie using lxc

elbereth

Well-known member
Dash Support Group
In this tutorial we will learn how to have a working gitian-builder for dash under Debian Jessie.

Remark: Having grsec patch will surely mess with LXC and give you headaches with gitian-builder. Avoid...

Prerequisites
You need a working Debian Jessie installation. This is out of scope of this tutorial.
Create user gitian and add it to the sudo group (or at least give it sudo power).

Setting up Debian Jessie for gitian building
In this section we will be setting up the Debian installation for Gitian building.

First we need to log in as root to set up dependencies and make sure that our user can use the sudo command. Type/paste the following in the terminal:

Code:
apt-get install git ruby sudo apt-cacher-ng qemu-utils debootstrap lxc python-cheetah parted kpartx bridge-utils
adduser gitian sudo

When you get a colorful screen with a question about the 'LXC directory', just go with the default (/var/lib/lxc).

Then set up LXC and the rest with the following, which is a complex jumble of settings and workarounds:

Code:
# add cgroup for LXC
echo "cgroup /sys/fs/cgroup cgroup defaults 0 0" >> /etc/fstab
# make /etc/rc.local script that sets up bridge between guest and host
echo '#!/bin/sh -e' > /etc/rc.local
echo 'brctl addbr br0' >> /etc/rc.local
echo 'ifconfig br0 10.0.3.2/24 up' >> /etc/rc.local
echo 'exit 0' >> /etc/rc.local
# make sure that USE_LXC is always set when logging in as gitian,
# and configure LXC IP addresses
echo 'export USE_LXC=1' >> /home/gitian/.profile
echo 'export GITIAN_HOST_IP=10.0.3.2' >> /home/gitian/.profile
echo 'export LXC_GUEST_IP=10.0.3.5' >> /home/gitian/.profile
reboot

The reboot at the end is just so that cgroup and br0 are mounted/created. Just do it by hand if you wish to avoid a reboot.
Of course all those steps only need to be performed once.

Installing gitian
Re-login as the user gitian that was created during installation. The rest of the steps in this guide will be performed as that user.

There is no python-vm-builder package in Debian, so we need to install it from source ourselves,

Code:
wget http://archive.ubuntu.com/ubuntu/pool/universe/v/vm-builder/vm-builder_0.12.4+bzr489.orig.tar.gz
echo "ec12e0070a007989561bfee5862c89a32c301992dd2771c4d5078ef1b3014f03 vm-builder_0.12.4+bzr489.orig.tar.gz" | sha256sum -c
# (verification -- must return OK)
tar -zxvf vm-builder_0.12.4+bzr489.orig.tar.gz
cd vm-builder-0.12.4+bzr489
sudo python setup.py install
cd ..

Note: When sudo asks for a password, enter the password for the user gitian not for root.

Clone the git repositories for bitcoin and gitian.

Code:
git clone https://github.com/devrandom/gitian-builder.git
git clone https://github.com/dashpay/dash

Setting up the gitian image
Gitian needs a virtual image of the operating system to build in. Currently this is Ubuntu Precise x86_64. This image will be copied and used every time that a build is started to make sure that the build is deterministic. Creating the image will take a while, but only has to be done once.

IMPORTANT: The default bin/make-base-vm will fail with Debian Jessie.
Fixing it is easy, just add sudo before the mkfs.ext4 at line 114:
Code:
$ git diff
diff --git a/bin/make-base-vm b/bin/make-base-vm
index 7fc7818..e51b1a2 100755
--- a/bin/make-base-vm
+++ b/bin/make-base-vm
@@ -114,7 +114,7 @@ if [ $LXC = "1" ]; then
   # Need universe for lxc in lucid
   env -i LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 sudo debootstrap --arch=$ARCH --include=$addpkg --exclude=$removepkg --components=main,universe $SUITE $OUT-bootstrap $MIRROR
   dd if=/dev/zero of=$OUT-lxc bs=1M count=1 seek=10240
-  mkfs.ext4 -F $OUT-lxc
+  sudo mkfs.ext4 -F $OUT-lxc
   t=`mktemp -d gitian.XXXXXXXX`
   sudo mount $OUT-lxc $t
   sudo cp -a $OUT-bootstrap/* $t

Execute the following as user debian:

Code:
cd gitian-builder
bin/make-base-vm --lxc --arch amd64 --suite precise

There will be a lot of warnings printed during build of the image. These can be ignored.

Note: When sudo asks for a password, enter the password for the user gitian not for root.

Getting and building the inputs
First we need to retrieve the dependencies in order to be able to build the dash binaries in gitian-builder.

As gitian user:
Code:
cd ~gitian/dash
git pull
cd ~gitian/gitian-builder
make -C ../dash/depends download SOURCES_PATH=`pwd`/cache/common

You can run this before every build to be sure you are not missing dependencies. It will only download new files.

IMPORTANT: In order to compile the Mac OSX binaries you will need to place the
MacOSX10.7.sdk.tar.gz file in the gitian-builder/inputs folder.
Find out the steps to create this file from the bitcoin documentation.

Building dash binaries
Now you only need to start compiling dash.

REMARK: The first time you run the following commands it will update the base image and compile the full dependencies and it will take A LOT of time. You can monitor the install and build by using:
Code:
tail -f var/install.log
Or:
Code:
tail -f var/build.log

If no errors during the build process the output files will be in build/out directory.
Don't forget to move them before building something else or they will be deleted.

Linux:
Go to gitian-builder directory and to build latest version from master:
Code:
./bin/gbuild --commit dash ../dash/contrib/gitian-descriptors/gitian-linux.yml

To build a specific commit or branch (here v0.12.0.x branch):
Code:
./bin/gbuild --commit dash=v0.12.0.x ../dash/contrib/gitian-descriptors/gitian-linux.yml

Windows:
Go to gitian-builder directory and to build latest version from master:
Code:
./bin/gbuild --commit dash ../dash/contrib/gitian-descriptors/gitian-win.yml

To build a specific commit or branch (here v0.12.0.x branch):
Code:
./bin/gbuild --commit dash=v0.12.0.x ../dash/contrib/gitian-descriptors/gitian-win.yml

Mac OSX:
Go to gitian-builder directory and to build latest version from master:
Code:
./bin/gbuild --commit dash ../dash/contrib/gitian-descriptors/gitian-osx.yml

To build a specific commit or branch (here v0.12.0.x branch):
Code:
./bin/gbuild --commit dash=v0.12.0.x ../dash/contrib/gitian-descriptors/gitian-osx.yml
 
Last edited by a moderator:
I want to try building a windows version from source, can someone point me towards a compilar that I download to start me off, thanks.
 
I want to try building a windows version from source, can someone point me towards a compilar that I download to start me off, thanks.
Use Virtual Box to run a copy of Debian/Ubuntu on your Windows Box and use the instructions above. Getting Dash compiled with Visual Studio is a mess and a tedious tasks
 
Use Virtual Box to run a copy of Debian/Ubuntu on your Windows Box and use the instructions above. Getting Dash compiled with Visual Studio is a mess and a tedious tasks

Ok, will look into that. I always wanted to try compiling sgminer as well, I suspected it would run a tiny bit quicker but have never been proved on my hunch as yet.
 
Ok, will look into that. I always wanted to try compiling sgminer as well, I suspected it would run a tiny bit quicker but have never been proved on my hunch as yet.

Some remarks for future readers who want to build for Windows (on Windows):
  • Building AMD-based mining software on Windows: use MinGW
  • Building Nvidia-based mining software on Windows: use Visual Studio
  • Building Dash wallet on Windows: don't do that, use Gitian. There's a reason why Bitcoin completely removed build instructions for Windows from its repository. Same applies for every Qt-based wallet BTW. And with Gitian you'll get Linux and OSX binaries for free as well.
 
Last edited by a moderator:
  • Building Dash wallet on Windows: don't do that, use Gitian. There's a reason why Bitcoin completely removed build instructions for Windows from its repository. Same applies for every Qt-based wallet BTW. And with Gitian you'll get Linux and OSX binaries for free as well.

So true... I tried last year when there was no Windows binary on coin launch to compile it directly on Windows without gitian.
A true nightmare and I did not succeed.
 
So true... I tried last year when there was no Windows binary on coin launch to compile it directly on Windows without gitian.
A true nightmare and I did not succeed.

Great, and has step by step noob instructions, only thing is I don't have any hard drive left because bitcoin has hoovered up the last 35 gigabytes and Mr Andresen wants even bigger block sizes :rolleyes:
 
Great, and has step by step noob instructions, only thing is I don't have any hard drive left because bitcoin has hoovered up the last 35 gigabytes and Mr Andresen wants even bigger block sizes :rolleyes:
Which btc wallet are you using? I gave up downloading the core wallet and i've been using multibit, works fine for me.
 
Which btc wallet are you using? I gave up downloading the core wallet and i've been using multibit, works fine for me.

So how much space does the multibit use, should I delete this version and put that on, is it worth doing ?
 
Multibit for windows is 8.5MB: https://multibit.org/
Some people prefer Electrum, but I haven't tried Electrum. You might want to compare the two. :)

cheers, will change over, don't take much notice of the other bitcoin wallets, I figured the blockchain info saying 35 gig as the same as me was enough and leave it at that.
 
cheers, will change over, don't take much notice of the other bitcoin wallets, I figured the blockchain info saying 35 gig as the same as me was enough and leave it at that.
Oh.. before you delete that.. If you have a spare usb stick, maybe save the blockchain there just in case you need it later for some testing or something?
 
Multibit for windows is 8.5MB: https://multibit.org/

During my catchup and gitian installation I saw this about multibit (an SPV wallet), its an old comment by Mike Hearn and Peter Todd, but may still stand about the quicker wallet user's lack of security,
https://bitcointalk.org/index.php?topic=252937.0
'remember that if you want to keep the contents of your wallet private, you have to be careful with SPV mode. Because your node doesn't relay transactions, any transaction you do relay is obviously from you. In the future the payment protocol will help, but for now assume that by using an SPV wallet your peers can figure out what coins you own.'

'You should be connecting over Tor, or at least over a proxy, if you want to maintain your privacy. Unfortunately as far as I know right now most SPV clients don't make it easy to connect over Tor, but it is possible.
Full nodes don't have anywhere near as serious a privacy problem even without Tor because they do relay transactions, so unless someone is watching every node out there they can't know if you were the one who broadcast the transaction first.


Good reason to run a full bitcoin wallet ?
 
During my catchup and gitian installation I saw this about multibit (an SPV wallet), its an old comment by Mike Hearn and Peter Todd, but may still stand about the quicker wallet user's lack of security,
https://bitcointalk.org/index.php?topic=252937.0
'remember that if you want to keep the contents of your wallet private, you have to be careful with SPV mode. Because your node doesn't relay transactions, any transaction you do relay is obviously from you. In the future the payment protocol will help, but for now assume that by using an SPV wallet your peers can figure out what coins you own.'

'You should be connecting over Tor, or at least over a proxy, if you want to maintain your privacy. Unfortunately as far as I know right now most SPV clients don't make it easy to connect over Tor, but it is possible.
Full nodes don't have anywhere near as serious a privacy problem even without Tor because they do relay transactions, so unless someone is watching every node out there they can't know if you were the one who broadcast the transaction first.


Good reason to run a full bitcoin wallet ?

I'd say yes just for the sake of the network but if disk space is an issue then the light wallets are good. Even if its not a full node relaying transactions helps. ISPs seen to have a certain fixation with blocking 8333 lately and there was a bit drop in full nodes, still about 10k though. If its only showing 8 connections port 8333 (or 9999 for Dash) is likely blocked and its not running as a full node.

EDIT: Just realised the off topic, sry.
 
Last edited by a moderator:
How fix this ?


gitian@gitian:~/gitian-builder$ ./bin/gbuild --commit dash ../dash/contrib/gitian-descriptors/gitian-linux.yml
sha256sum: bitcoin-deps-linux32-gitian-r9.zip: Нет такого файла или каталога
sha256sum: bitcoin-deps-linux64-gitian-r9.zip: Нет такого файла или каталога
sha256sum: boost-linux32-1.55.0-gitian-r1.zip: Нет такого файла или каталога
sha256sum: boost-linux64-1.55.0-gitian-r1.zip: Нет такого файла или каталога
sha256sum: qt-linux32-4.6.4-gitian-r1.tar.gz: Нет такого файла или каталога
sha256sum: qt-linux64-4.6.4-gitian-r1.tar.gz: Нет такого файла или каталога
./bin/gbuild:238:in `block in <main>': must specify a commit for directory dash (RuntimeError)
from ./bin/gbuild:235:in `each'
from ./bin/gbuild:235:in `<main>'
gitian@gitian:~/gitian-builder$ ./bin/gbuild --commit dash=v0.12.0.x ../dash/contrib/gitian-descriptors/gitian-linux.yml
sha256sum: bitcoin-deps-linux32-gitian-r9.zip: Нет такого файла или каталога
sha256sum: bitcoin-deps-linux64-gitian-r9.zip: Нет такого файла или каталога
sha256sum: boost-linux32-1.55.0-gitian-r1.zip: Нет такого файла или каталога
sha256sum: boost-linux64-1.55.0-gitian-r1.zip: Нет такого файла или каталога
sha256sum: qt-linux32-4.6.4-gitian-r1.tar.gz: Нет такого файла или каталога
sha256sum: qt-linux64-4.6.4-gitian-r1.tar.gz: Нет такого файла или каталога
--- Building for precise i386 ---
Stopping target if it is up
Making a new image copy
cp: не удалось выполнить stat для «base-precise-i386»: Нет такого файла или каталога
./bin/gbuild:21:in `system!': failed to run make-clean-vm --suite precise --arch i386 (RuntimeError)
from ./bin/gbuild:57:in `build_one_configuration'
from ./bin/gbuild:264:in `block (2 levels) in <main>'
from ./bin/gbuild:259:in `each'
from ./bin/gbuild:259:in `block in <main>'
from ./bin/gbuild:257:in `each'
from ./bin/gbuild:257:in `<main>'
gitian@gitian:~/gitian-builder$ ./bin/gbuild --commit dash ../dash/contrib/gitian-descriptors/gitian-win.yml
sha256sum: qt-win32-5.2.0-gitian-r3.zip: Нет такого файла или каталога
sha256sum: qt-win64-5.2.0-gitian-r3.zip: Нет такого файла или каталога
sha256sum: boost-win32-1.55.0-gitian-r6.zip: Нет такого файла или каталога
sha256sum: boost-win64-1.55.0-gitian-r6.zip: Нет такого файла или каталога
sha256sum: bitcoin-deps-win32-gitian-r16.zip: Нет такого файла или каталога
sha256sum: bitcoin-deps-win64-gitian-r16.zip: Нет такого файла или каталога
sha256sum: protobuf-win32-2.5.0-gitian-r4.zip: Нет такого файла или каталога
sha256sum: protobuf-win64-2.5.0-gitian-r4.zip: Нет такого файла или каталога
./bin/gbuild:238:in `block in <main>': must specify a commit for directory dash (RuntimeError)
from ./bin/gbuild:235:in `each'
from ./bin/gbuild:235:in `<main>'
gitian@gitian:~/gitian-builder$ ./bin/gbuild --commit dash=v0.12.0.x ../dash/contrib/gitian-descriptors/gitian-win.yml
sha256sum: qt-win32-5.2.0-gitian-r3.zip: Нет такого файла или каталога
sha256sum: qt-win64-5.2.0-gitian-r3.zip: Нет такого файла или каталога
sha256sum: boost-win32-1.55.0-gitian-r6.zip: Нет такого файла или каталога
sha256sum: boost-win64-1.55.0-gitian-r6.zip: Нет такого файла или каталога
sha256sum: bitcoin-deps-win32-gitian-r16.zip: Нет такого файла или каталога
sha256sum: bitcoin-deps-win64-gitian-r16.zip: Нет такого файла или каталога
sha256sum: protobuf-win32-2.5.0-gitian-r4.zip: Нет такого файла или каталога
sha256sum: protobuf-win64-2.5.0-gitian-r4.zip: Нет такого файла или каталога
--- Building for precise amd64 ---
Stopping target if it is up
Making a new image copy
lxc-execute: failed to mount rootfs
lxc-execute: failed to setup rootfs for 'gitian'
lxc-execute: Error setting up rootfs mount after spawn
lxc-execute: failed to setup the container
lxc-execute: invalid sequence number 1. expected 2
lxc-execute: failed to spawn 'gitian'
./bin/gbuild:21:in `system!': failed to run make-clean-vm --suite precise --arch amd64 (RuntimeError)
from ./bin/gbuild:57:in `build_one_configuration'
from ./bin/gbuild:264:in `block (2 levels) in <main>'
from ./bin/gbuild:259:in `each'
from ./bin/gbuild:259:in `block in <main>'
from ./bin/gbuild:257:in `each'
from ./bin/gbuild:257:in `<main>'
 
i do all step be step , but hv this error on debian

Having the error messages in English would certainly help....

Anyway, let's do it step by step:

- did you succesfully run "bin/make-base-vm --lxc --arch amd64 --suite precise" ?
 
Having the error messages in English would certainly help....

Anyway, let's do it step by step:

- did you succesfully run "bin/make-base-vm --lxc --arch amd64 --suite precise" ?
I think he missed to download/build the dependencies - the above instructions are for the 0.12 branch. v0.11 has a different dependency system.

Code:
sha256sum: bitcoin-deps-linux32-gitian-r9.zip: No such file or directory
 
Last edited by a moderator:
I think he missed to download/build the dependencies - the above instructions are for the 0.12 branch. v0.11 has a different dependency system.

Code:
sha256sum: bitcoin-deps-linux32-gitian-r9.zip: No such file or directory

Nicely spotted!

So, mraksel , you have to follow the instructions here or wait for v0.12.0.x.
 
Back
Top