error gitian build on Debian

naruby

New Member
Feb 17, 2016
29
33
13
I'm trying , my first, OSX build v0.12.1.x with gitian on a remote Debian.
My build fails, this it the tail of build.log
Code:
make: *** [/home/ubuntu/build/dash/depends/work/build/x86_64-apple-darwin11/boost/1_59_0-967ac548613/./.stamp_built] Error 1

make: Leaving directory `/home/ubuntu/build/dash/depends'
The directory /home/ubuntu seams wrong as I've created a user 'debian'
.profile
Code:
declare -x GITIAN_HOST_IP="10.0.3.2"
declare -x HOME="/home/debian"
declare -x LANG="en_US.UTF-8"
declare -x LOGNAME="debian"
declare -x LXC="1"
declare -x LXC_GUEST_IP="10.0.3.5"
declare -x PWD="/home/debian"
declare -x SHELL="/bin/bash"
declare -x USER="debian"
declare -x USE_LXC="1"
declare -ax VERSION='([0]="0.12.1.x")
But why is the outdir in the build.log /home/ubuntu ?
Code:
 set -e
+ export LANG=en_US.UTF-8
+ LANG=en_US.UTF-8
+ export LC_ALL=en_US.UTF-8
+ LC_ALL=en_US.UTF-8
+ umask 002
+ export OUTDIR=/home/ubuntu/out
+ OUTDIR=/home/ubuntu/out
+ GBUILD_BITS=64
+ GBUILD_CACHE_ENABLED=1
+ GBUILD_PACKAGE_CACHE=/home/ubuntu/cache/dash-osx-0.12
+ GBUILD_COMMON_CACHE=/home/ubuntu/cache/common
 
  • Like
Reactions: flare

naruby

New Member
Feb 17, 2016
29
33
13
After a couple of hours sleep (wonderful invention) I came also to this conclusion :)
No worries I keep at it.

btw I guess all build command need to be ./bin/gbuild --commit dash ?
The windows and OSX is probably not changed after copy&paste from the bitcoin gitian manual and still saids ./bin/gbuild --commit bitcoin

(can't post links yet but it's the one you quoted v0.12.1.x/doc/gitian-building.md)
 
  • Like
Reactions: flare

UdjinM6

Official Dash Dev
Core Developer
Dash Core Group
May 20, 2014
3,639
3,537
1,183
After a couple of hours sleep (wonderful invention) I came also to this conclusion :)
No worries I keep at it.

btw I guess all build command need to be ./bin/gbuild --commit dash ?
The windows and OSX is probably not changed after copy&paste from the bitcoin gitian manual and still saids ./bin/gbuild --commit bitcoin

(can't post links yet but it's the one you quoted v0.12.1.x/doc/gitian-building.md)
Good catch! Thanks!
https://github.com/dashpay/dash/pull/704 :)
 

naruby

New Member
Feb 17, 2016
29
33
13
Restarted from scratch on a new VM but no joy.

Code:
[email protected]:~/gitian-builder$ ./bin/gbuild --commit dash=v${VERSION} ../dash/contrib/gitian-descriptors/gitian-linux.yml
Creating build script (var/build-script)
Running build script (log in var/build.log)
./bin/gbuild:21:in `system!': failed to run on-target setarch x86_64 bash -x < var/build-script > var/build.log 2>&1 (RuntimeError)
    from ./bin/gbuild:145:in `build_one_configuration'
    from ./bin/gbuild:278:in `block (2 levels) in <main>'
    from ./bin/gbuild:273:in `each'
    from ./bin/gbuild:273:in `block in <main>'
    from ./bin/gbuild:271:in `each'
    from ./bin/gbuild:271:in `<main>'
tail build.log
Code:
    cp "bin.v2/libs/thread/build/gcc-4.8/release/link-static/threading-multi/libboost_thread-mt.a"  "stage/lib/libboost_thread-mt.a"
...failed updating 1 target...
...skipped 6 targets...
...updated 106 targets...
make: *** [/home/ubuntu/build/dash/depends/work/build/i686-pc-linux-gnu/boost/1_59_0-b211d0cb294/./.stamp_built] Error 1
make: Leaving directory `/home/ubuntu/build/dash/depends'
 

flare

Grizzled Member
May 18, 2014
2,286
2,404
1,183
Germany
Restarted from scratch on a new VM but no joy.

Code:
[email protected]:~/gitian-builder$ ./bin/gbuild --commit dash=v${VERSION} ../dash/contrib/gitian-descriptors/gitian-linux.yml
Creating build script (var/build-script)
Running build script (log in var/build.log)
./bin/gbuild:21:in `system!': failed to run on-target setarch x86_64 bash -x < var/build-script > var/build.log 2>&1 (RuntimeError)
    from ./bin/gbuild:145:in `build_one_configuration'
    from ./bin/gbuild:278:in `block (2 levels) in <main>'
    from ./bin/gbuild:273:in `each'
    from ./bin/gbuild:273:in `block in <main>'
    from ./bin/gbuild:271:in `each'
    from ./bin/gbuild:271:in `<main>'
tail build.log
Code:
    cp "bin.v2/libs/thread/build/gcc-4.8/release/link-static/threading-multi/libboost_thread-mt.a"  "stage/lib/libboost_thread-mt.a"
...failed updating 1 target...
...skipped 6 targets...
...updated 106 targets...
make: *** [/home/ubuntu/build/dash/depends/work/build/i686-pc-linux-gnu/boost/1_59_0-b211d0cb294/./.stamp_built] Error 1
make: Leaving directory `/home/ubuntu/build/dash/depends'
A few more lines of build.log would be useful :)
 

UdjinM6

Official Dash Dev
Core Developer
Dash Core Group
May 20, 2014
3,639
3,537
1,183
Hmm.. Just out of curiosity - have you checked out in ~/dash folder to use proper gitian-build descriptors?
Code:
git checkout v0.12.1.x
 

flare

Grizzled Member
May 18, 2014
2,286
2,404
1,183
Germany
Hmm,

in both cases the underlying compilers died as root cause:

linux
Code:
g++: internal compiler error: Killed (program cc1plus)
osx
Code:
configure: error: C compiler cannot create executables
May i ask what spec (RAM/SWAP/HDD) the host machine has? Could you monitor (command 'top') the RAM/SWAP during gitian-build? It seems the LXC containers are dying due to lack of resources...
 

naruby

New Member
Feb 17, 2016
29
33
13
It's a 512 Mb VPS
I'm a Linux novice so I hope this is the info you need (taken during a build) :

Code:
[email protected]:~/gitian-builder/var$ df
Filesystem     1K-blocks     Used Available Use% Mounted on
/dev/vda1       20504916 12359928   7089292  64% /
udev               10240        0     10240   0% /dev
tmpfs             101268     8528     92740   9% /run
tmpfs             253160        0    253160   0% /dev/shm
tmpfs               5120        0      5120   0% /run/lock
tmpfs             253160        0    253160   0% /sys/fs/cgroup

[email protected]:~/gitian-builder/var$ cat /proc/swaps
Filename                Type        Size    Used    Priority


[email protected]itian:~/gitian-builder/var$ free -g
             total       used       free     shared    buffers     cached
Mem:             0          0          0          0          0          0
-/+ buffers/cache:          0          0
Swap:            0          0          0

[email protected]:~/gitian-builder/var$ free -k
             total       used       free     shared    buffers     cached
Mem:        506324     496740       9584       8536      24884     218476
-/+ buffers/cache:     253380     252944
Swap:            0          0          0

[email protected]:~/gitian-builder/var$ free -m
             total       used       free     shared    buffers     cached
Mem:           494        386        107          8         23        161
-/+ buffers/cache:        200        293
Swap:            0          0          0

Top is of course only 1 monent in time, cpu seams indeed heavy loaded.
To upgrade to a hight tier on Digital Ocean?

Code:
top - 12:49:55 up  7:02,  2 users,  load average: 1.99, 1.84, 1.79
Tasks:  88 total,   3 running,  85 sleeping,   0 stopped,   0 zombie
%Cpu(s): 93.0 us,  6.6 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.3 st
KiB Mem:    506324 total,   499612 used,     6712 free,     6020 buffers
KiB Swap:        0 total,        0 used,        0 free.   181352 cached Mem
CPU
Code:
rocessor    : 0
vendor_id    : GenuineIntel
cpu family    : 6
model        : 62
model name    : Intel(R) Xeon(R) CPU E5-2630L v2 @ 2.40GHz
stepping    : 4
microcode    : 0x1
cpu MHz        : 2399.998
cache size    : 15360 KB
physical id    : 0
siblings    : 1
core id        : 0
cpu cores    : 1
apicid        : 0
initial apicid    : 0
fpu        : yes
fpu_exception    : yes
cpuid level    : 13
wp        : yes
flags        : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl eagerfpu pni pclmulqdq vmx ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm arat xsaveopt vnmi ept fsgsbase tsc_adjust smep erms
bogomips    : 4799.99
clflush size    : 64
cache_alignment    : 64
address sizes    : 40 bits physical, 48 bits virtual
power management:
 
  • Like
Reactions: flare

flare

Grizzled Member
May 18, 2014
2,286
2,404
1,183
Germany
It's a 512 Mb VPS

Code:
[email protected]:~/gitian-builder/var$ df
Filesystem     1K-blocks     Used Available Use% Mounted on
/dev/vda1       20504916 12359928   7089292  64% /
udev               10240        0     10240   0% /dev
tmpfs             101268     8528     92740   9% /run
tmpfs             253160        0    253160   0% /dev/shm
tmpfs               5120        0      5120   0% /run/lock
tmpfs             253160        0    253160   0% /sys/fs/cgroup

[email protected]:~/gitian-builder/var$ cat /proc/swaps
Filename                Type        Size    Used    Priority

Code:
top - 12:49:55 up  7:02,  2 users,  load average: 1.99, 1.84, 1.79
Tasks:  88 total,   3 running,  85 sleeping,   0 stopped,   0 zombie
%Cpu(s): 93.0 us,  6.6 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.3 st
KiB Mem:    506324 total,   499612 used,     6712 free,     6020 buffers
KiB Swap:        0 total,        0 used,        0 free.   181352 cached Mem
I hardly doubt that a 512MB machine without any(!) swap will work out :) For comparism:

The CI-Server which we are using to produce the Dash releases is a Amazon m2.4xlarge instance: 68GB RAM, 8 cpu cores :)

--> https://dashpay.atlassian.net/builds

So you should really consider to upgrade your Digital Ocean instance, I'd say you should be safe using a 2GB Ram instance, but it is just a educated guess :)
 
Last edited by a moderator:

naruby

New Member
Feb 17, 2016
29
33
13
White smoke = Success !!
I've installed a Debian VM on my pc and was able to build the Linux 0.12.0.x at my first attempt. I will build the OSX & Windows also tomorrow.
At least I have no a (basic) understanding of the gitian workflow.

I guess I need to make a pull request to the gitian.sigs repository to get my PGP key registered ?

Thanks for all the advise and patience.
 
  • Like
Reactions: flare

flare

Grizzled Member
May 18, 2014
2,286
2,404
1,183
Germany
White smoke = Success !!
I've installed a Debian VM on my pc and was able to build the Linux 0.12.0.x at my first attempt. I will build the OSX & Windows also tomorrow.
At least I have no a (basic) understanding of the gitian workflow.

I guess I need to make a pull request to the gitian.sigs repository to get my PGP key registered ?

Thanks for all the advise and patience.
Good stuff :)

Re. the sigs: we did not use this repo yet, but as more and more users successfully built with gitian it makes sense to start using it. I'll have a look at the gitian.sigs workflow and come back to you.

Holger
 
  • Like
Reactions: UdjinM6

flare

Grizzled Member
May 18, 2014
2,286
2,404
1,183
Germany
is there a flag that can be used to make gitian use all available threads/cores??
gbuild accepts the '-j <num_procs>' flag to set the number of processes used during compilation (actually this is passed to 'make' command)