• Forum has been upgraded, all links, images, etc are as they were. Please see Official Announcements for more information

[GUIDE] How to run a masternode in a chrooted environment

Figlmüller

Member
Hi,

I'm currently running a few masternodes within chroot environments to add an additional layer of security in order to protect from taking over the whole system if one gets shell or file-system access through the dash daemon.

I assume you already know how to compile and run a master node - this is how you run it in a chrooted environment.

I highly recommend to run your own kernel patched with gr security together with chroot security features enabled. Else chrooting is more or less useless.

I am not going to cover how to patch, configure and compile linux kernels, check out guides on the internet in order to accomplish this task.

To understand this guide you will need to have knowledge about linux operating systems including basic linux shell operations and linux filesystems. So I am not going into linux basics in this guide.

The below mentioned procedure has been tested on the latest stable version of Debian, which is currently Debian 7.7 Wheezy. It should work on most linux distributions including Ubuntu - no guarantee for BSD-like operating systems.

Before doing anything:
Please read the guide carefully and stick to the directions. Replace specific data (such as usernames, directory names,...) where needed. The provided commands are examples only.
To ensure safety, make a backup of your stuff before following this guide! Do not blame me if anything goes wrong on your side.

Index:

[ 1 ] Preparing the environment
[ 1.1 ] Creating the directory structure
[ 1.2 ] Applying ownership to directories
[ 1.3 ] Copy dash (former darkcoin) files
[ 1.4 ] Populate dev directory
[ 1.5 ] Setting up needed libraries
[ 1.6 ] Copying additional files​
[ 2 ] Launching the masternode
[ 2.1 ] Starting
[ 2.2 ] Stopping
[ 2.3 ] Check if the process is really jailed​
[ 3 ] Considerations / Appendix / Things to be aware of
[ 3.1 ] Stopping and starting using the PID way
[ 3.2 ] Running a cron script
[ 3.3 ] Updating your libraries inside of the jail
[ 3.4 ] Log rotation
[ 3.5 ] Master node related scripting​
[ 4 ] End

[ 1 ] Preparing the environment


[ 1.1 ] Creating the directory structure
As a super user, create a new folder for the environment together with the following directories:

Code:
etc/
dev/
usr/
usr/lib/
usr/local/share
usr/local/bin
usr/local/lib
lib/
home/
home/mnuser

Notice: /home/mnuser is the home of the user(name) you are already using for your masternode.



[ 1.2 ] Applying ownership to directories
Now change in the home directory inside of the environment (for example: cd /opt/mnenv/home)
Change the ownership of the user directory to the according user:

Code:
chown mnuser:mnuser mnuser


[ 1.3 ] Copy dash (former darkcoin) files
Copy all the dash (darkcoin) files to a location inside of the previously created environment.

For example the binary file to: /opt/mnenv/home/mnuser/bin/dash/
... where /opt/mnenv is the environment and /home/mnuser the home directory of the master node user inside of it.
Make sure to set the user rights on the copied files too. For example:

Code:
chown mnuser:mnuser /opt/mnenv/home/mnuser/bin/dash/dashd
chmod 700 /opt/mnenv/home/mnuser/bin/dash/dashd

You will also need to copy the dash settings directory into the environment directory. For example:

Code:
cp -R /home/mnuser/.dashcore/ /opt/mnenv/home/mnuser/

Again, make sure everything is readable by the masternode user.


[ 1.4 ] Populate dev directory
In order to run the daemon we need to populate the dev directory in our environment. So as a super user, switch to the dev directory of the environment and execute the following:

Code:
mknod -m 644 random c 1 8
mknod -m 644 urandom c 1 9
mknod -m 666 null c 1 3


[ 1.5 ] Setting up needed libraries
We will need all the libraries linked from the dash daemon. To find out which libraries we need, run:

Code:
ldd /opt/mnenv/home/mnuser/bin/dash/dashd

Which results in something like this:
Code:
        linux-vdso.so.1 (0x00007fff241dc000)
        libboost_system.so.1.55.0 => /usr/lib/x86_64-linux-gnu/libboost_system.so.1.55.0 (0x00007fb1ee7e4000)
        libboost_filesystem.so.1.55.0 => /usr/lib/x86_64-linux-gnu/libboost_filesystem.so.1.55.0 (0x00007fb1ee5c4000)
        libboost_program_options.so.1.55.0 => /usr/lib/x86_64-linux-gnu/libboost_program_options.so.1.55.0 (0x00007fb1ee344000)
        libboost_thread.so.1.55.0 => /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.55.0 (0x00007fb1ee124000)
        libboost_chrono.so.1.55.0 => /usr/lib/x86_64-linux-gnu/libboost_chrono.so.1.55.0 (0x00007fb1edf14000)
        libdb_cxx-5.3.so => /usr/lib/x86_64-linux-gnu/libdb_cxx-5.3.so (0x00007fb1edb24000)
        libssl.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007fb1ed8bc000)
        libcrypto.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007fb1ed4bc000)
        libevent_pthreads-2.0.so.5 => /usr/lib/x86_64-linux-gnu/libevent_pthreads-2.0.so.5 (0x00007fb1ed2b4000)
        libevent-2.0.so.5 => /usr/lib/x86_64-linux-gnu/libevent-2.0.so.5 (0x00007fb1ed064000)
        libanl.so.1 => /lib/x86_64-linux-gnu/libanl.so.1 (0x00007fb1ece5c000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fb1ecb4c000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fb1ec844000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fb1ec624000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fb1ec404000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fb1ec054000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fb1ebe44000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fb1ebc3c000)
        libevent_core-2.0.so.5 => /usr/lib/x86_64-linux-gnu/libevent_core-2.0.so.5 (0x00007fb1eba0c000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fb1ef32c000)

As a super user:
Now you will need to copy every library into its appropriate directory inside of the environment. The current path is listed in the output of ldd.
You may need to create additional directories inside of your environment before doing copy operations. You can omit linux-vdso.so.1.

For example:
Code:
cp /lib/libname /opt/mnenv/lib/

Often libraries are referenced by symlinks. Be sure to copy both, the symlink and the actual library then.


[ 1.6 ] Copying additional files
You will need to copy additional files in order to make name resolution work. Even if the daemon does not use name or reverse ip resolution directly, it is a good idea to copy the files anway as some referenced library or similar may be pissed not finding its beloved name resolution libraries :)

Like with the libraries before, copy the following files:

Code:
/etc/hosts
/etc/resolv.conf
/lib/libnss_dns.so.2
/lib/libresolv.so.2

Notice: Depending on your linux operating system, the last two libraries may be at a different location such as /lib64. You can always find them by performing a search like: find / -name "*libresolv*"



[ 2 ] Launching the masternode


[ 2.1 ] Starting
To start the master node you can use start-stop-daemon which is available on many linux distributions:

Run as root:
Code:
/sbin/start-stop-daemon --chroot /opt/mnenv --chuid mnuser:mnuser --start --exec /home/mnuser/bin/dash/dashd

Tip: To pass parameters to dashd using start-stop-daemon, amend "--" followed by the parameter.
For example:


Code:
/sbin/start-stop-daemon --chroot /opt/mnenv --chuid mnuser:mnuser --start --exec /home/mnuser/bin/dash/dashd -- --reindex


If the masternode crashed with the following error:

Code:
runtime error: locale::facet::_S_create_c_locale name not valid

make sure to set LC_ALL to C before starting the dash daemon:

Code:
LC_ALL="C" && export LC_ALL && /sbin/start-stop-daemon --chroot /opt/mnenv --chuid mnuser:mnuser --start --exec /home/mnuser/bin/dash/dashd


[ 2.2 ] Stopping

To stop, simply run as root:
Code:
/sbin/start-stop-daemon --chroot /opt/mnenv --chuid mnuser:mnuser --stop --exec /home/mnuser/bin/dash/dashd


[ 2.3 ] Check if the process is really jailed
Never trust anyone. We need to check if the root directory of the running daemon is really the environment we created before.

Find out the pid of the process:
Code:
pidof dashd

This gives you the process ID. For example: 8800.

Now:
Code:
ls -ld /proc/8800/root

If we made everything right, the output should look like:

Code:
lrwxrwxrwx 1 mnuser mnuser 0 Oct 31 21:20 /proc/8440/root -> /opt/mnenv


[ 3 ] Considerations / Appendix / Things to be aware of


[ 3.1 ] Stopping and starting using the PID way
Yes, dashd creates a PID file. start-stop-daemon supports starting and killing using created PID files, but I did not manage to get it work. The default approach (kill by process name) works best for me.
Running multiple master nodes on the same machine will not work without using the PID parameters of start-stop-daemon. I am not bothered by it as I generally run every master node within a dedicated VM.


[ 3.2 ] Running a cron script
You can run a cron script in order to start the daemon on reboot and if it crashed. You may want to append --oknodo to the command so start-stop-daemon does not return a non zero exit code if the daemon is already running.


[ 3.3 ] Updating your libraries inside of the jail
You must be aware that the libraries in your jail are not affected by system updates. Keep a list of libraries placed in your jail. If you ever update the libraries on your system, make sure to update them in your jail too, as they may include important security fixes.
 
Last edited:
[ 3.4 ] Log rotation
If you have any active log rotation on the debug.log file, do not forget to update the paths in logrotate.


[ 3.5 ] Master node related scripting
Be aware that you will need to update scripts related to the master node in order to work again.
Not all scripts will work with the new chrooted master node due to their nature.
Anyways, chrooting does not isolate the network, thus RPC is still working.
Another thing that will not work is the dash daemon shell command which should be executed once the daemon encounters an error (in the dash.conf).
Please don't try to add any shell executables such as /bin/bash to your jail in order to run scripts from the inside. This "contraband" in your jail will just poke holes in it.


[ 4 ] End
Thanks for reading, I hope my guide is useful for some of you :)
 
Last edited by a moderator:
This is what I done:

-block everything incoming except 9999/tcp and random ssh port
-key pair auth
-google auth
-fail2ban

Is it enough? Do you suggest to add this layer of security?
Great guide tho.
 
This is what I done:

-block everything incoming except 9999/tcp and random ssh port
-key pair auth
-google auth
-fail2ban

Is it enough? Do you suggest to add this layer of security?
Great guide tho.

Only if you're paranoid enough. Jailing the process reduces possible damage once somone breaks in through the dash daemon.
The points you listed and of course common sense should be enough though.

You could also use virtualizing and create virtual machine templates for your masternodes. Once something is wrong, or even feels wrong you just wipe the old virtual machine and deploy a new template. But as I said, that depends on your level of paranoia.
 
Back
Top