• Forum has been upgraded, all links, images, etc are as they were. Please see Official Announcements for more information

"watchdog-expired" on all masternodes after 12.1?

edificio

New member
Hello,

I just upgraded to 12.1, followed Tao's guide for upgrading and have succesfully started my masternodes.
But they all say "Watchdog_expired", both via VPS command and in the new Dash Core.

What is the error exactly and how do i fix it?

Any help would be much appriciated! :)
 
I got to step 5, then i got this error:

~/.dash/sentinel$ venv/bin/python bin/sentinel.py
-342: non-JSON HTTP response with '401 Unauthorized' from server
Cannot connect to dashd. Please ensure dashd is running and the JSONRPC port is open to Sentinel.

I really don't know what the JSONRPC port is :) But dashd is running and synced
 
I got to step 5, then i got this error:

~/.dash/sentinel$ venv/bin/python bin/sentinel.py
-342: non-JSON HTTP response with '401 Unauthorized' from server
Cannot connect to dashd. Please ensure dashd is running and the JSONRPC port is open to Sentinel.

I really don't know what the JSONRPC port is :) But dashd is running and synced
Do you have rpcuser=<replace> and rpcpassword=<replace> in dash.conf? "<replace>" can be changed to anything but as far as I know they're needed to use the RPC interface. rpcallowip=127.0.0.1 should also be in there to only allow connections form the same machine.

EDIT: A sample, might not need all the options but it runs ok:
Code:
rpcuser=user2468
rpcpassword=87967849bvuiu9dfs6vg65db
rpcallowip=127.0.0.1
listen=1
server=1
daemon=1
txindex=1
logtimestamps=1
maxconnections=256
masternode=1
masternodeprivkey=*********************

EDIT2: If dash.conf edited then client needs to be restarted for changes to take effect.
 
Last edited:
My .dash.conf looks like this:

rpcuser=wdwqdqwd
rpcpassword=wd3f23f3d33
rpcallowip=127.0.0.1
listen=1
server=1
daemon=1
logtimestamps=1
maxconnections=256
masternode=1
masternodeprivkey=***********************
addnode=23.23.186.131

(I changed rpcuser, rpcpassword and masternodeprivkey)

I seems like i have the things you mentioned?
 
My .dash.conf looks like this:

rpcuser=wdwqdqwd
rpcpassword=wd3f23f3d33
rpcallowip=127.0.0.1
listen=1
server=1
daemon=1
logtimestamps=1
maxconnections=256
masternode=1
masternodeprivkey=***********************
addnode=23.23.186.131

(I changed rpcuser, rpcpassword and masternodeprivkey)

I seems like i have the things you mentioned?

I'm not sure, asked around and someone should be back before too long though. Try this command from inside the sentinel folder, it might give some more info:
Code:
./venv/bin/py.test ./test
It runs some sentinel self-tests and should come up ok almost immediately.
 
Here you go:


============================= test session starts ==============================
platform linux2 -- Python 2.7.6, pytest-3.0.1, py-1.4.31, pluggy-0.3.1
rootdir: /home/tieb/.dashcore/sentinel, inifile:
collected 20 items

test/integration/test_jsonrpc.py F
test/unit/test_dash_config.py .
test/unit/test_dashd_data_shims.py ..
test/unit/test_dashy_things.py .....
test/unit/test_models.py ..
test/unit/test_submit_command.py .
test/unit/models/test_proposals.py ...
test/unit/models/test_superblocks.py .....

=================================== FAILURES ===================================
__________________________________ test_dashd __________________________________

def test_dashd():
config_text = DashConfig.slurp_config_file(config.dash_conf)
network = 'mainnet'
is_testnet = False
genesis_hash = u'00000ffd590b1485b3caadc19b22e6379c733355108f107a430458cdf3407ab6'
for line in config_text.split("\n"):
if line.startswith('testnet=1'):
network = 'testnet'
is_testnet = True
genesis_hash = u'00000bafbc94add76cb75e2ec92894837288a481e5c005f6563d91623bf8bc2c'

creds = DashConfig.get_rpc_creds(config_text, network)
dashd = DashDaemon(**creds)
assert dashd.rpc_command is not None

assert hasattr(dashd, 'rpc_connection')

# Dash testnet block 0 hash == 00000bafbc94add76cb75e2ec92894837288a481e5c005f6563d91623bf8bc2c
# test commands without arguments
> info = dashd.rpc_command('getinfo')

test/integration/test_jsonrpc.py:34:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
lib/dashd.py:42: in rpc_command
return self.rpc_connection.__getattr__(params[0])(*params[1:])
venv/local/lib/python2.7/site-packages/bitcoinrpc/authproxy.py:139: in __call__
response = self._get_response()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <bitcoinrpc.authproxy.AuthServiceProxy object at 0x7eff02bd7490>

def _get_response(self):
http_response = self.__conn.getresponse()
if http_response is None:
raise JSONRPCException({
'code': -342, 'message': 'missing HTTP response from server'})

content_type = http_response.getheader('Content-Type')
if content_type != 'application/json':
raise JSONRPCException({
> 'code': -342, 'message': 'non-JSON HTTP response with \'%i %s\' from server' % (http_response.status, http_response.reason)})
E JSONRPCException: -342: non-JSON HTTP response with '401 Unauthorized' from server

venv/local/lib/python2.7/site-packages/bitcoinrpc/authproxy.py:187: JSONRPCException
===================== 1 failed, 19 passed in 0.73 seconds ======================
 
You've tried restarting dashd? Something else maybe worth trying, stop the client (dash-cli stop) and delete all the files with a *.dat extension from the .dashcore folder except wallet.dat, restart the client and test sentinel again. The client will take some time to rebuild those files and sentinel won't run until that's done, it can be checked with "dash-cli mnsync status" but the self-tests should run ok (iirc).

EDIT: And please remove the "addnode=23.23.186.131" line from dash.conf, apparently it's very old :)
 
I don't have a dashcore folder. I just replaced the old dashd with the new one as Tao's guide said. I also haven't removed the old blockchain.. That's probably it?
 
I don't have a dashcore folder. I just replaced the old dashd with the new one as Tao's guide said. I also haven't removed the old blockchain.. That's probably it?

No, the blockchain needs to be reindexed for 12.1 but it's fine otherwise. The all the *.dat files are safe to remove and mncache.dat may be causing your problem, peers.dat can cause some issues, etc. Simpler to remove them all and start fresh if there are any doubts but certainly try deleting mncache.dat.

Can you post a link to Tao's guide please? I've not read it and I might be getting confused on some points. You've got sentinel in your .dashcore folder and the rest of your files should be in there too but he may be organising folders differently.
 

Ah, that one. Ok, it's a little out of date for 12.1 and if you didn't copy or renamed any directories for the update the you've probably got a completely new installation, the old client installs in /home/USER/.dash and the new one installs in /home/USER/.dashcore

Go to the home folder ("cd") and check the output of "cat .dash/dash.conf" and "cat .dashcore/dash.conf", they'll probably be different and the second might come up with nothing if the file exists but it's empty, it will say the file doesn't exist otherwise.
 
I think i'm just gonna try to reinstall the nodes completely, i think I've really made a mess of it, and i have no idea which files are where :)
 
I think i'm just gonna try to reinstall the nodes completely, i think I've really made a mess of it, and i have no idea which files are where :)

That could be the simplest thing to do, it will take a few hours to get the blockchain up to date but at least you'll know everything is as it should be. This script is very useful for managing masternodes btw but it's not up to date with 12.1 yet, it should be up to date in the next few days though so worth checking:
https://github.com/moocowmoo/dashman
When it's up to date it will handle installing both the client and Sentinal but the guy that wrote it is one of the core team and they've all been working flat out for about 26 hours, they'll need some sleep before getting to those things :)
 
They're the ones that deserve it really but no problem at all :) I almost forgot to mention, server requirements have been raised for 12.1 so a server upgrade at the same time might make sense. 1GB RAM minimum and 40GB hard drive space, for now you might get away with less hard drive space (it was 20GB initially) but Sentinals requirements mean that space will be needed at some stage and the RAM is definitely needed.

Feel free to PM if you've any issues, some parts of that update guide can get technical but it's fairly straight forward once you know what each part needs to do:
https://dashpay.atlassian.net/wiki/display/DOC/Updating+to+12.1+-+Masternodes
 
Watchdog_expired problem also here, 50% of my masternodes are in Watchdog_expired status,
first 1.5 hours was ok, enabled, then they changed to Watchdog_expired.

All nodes and settings are same, and sentinel_debug.log is empty, no errors.
All nodes are sync and 'dash-cli masternode status' gives "status": "Masternode successfully started"

Any idea?
 
Watchdog_expired problem also here, 50% of my masternodes are in Watchdog_expired status,
first 1.5 hours was ok, enabled, then they changed to Watchdog_expired.

All nodes and settings are same, and sentinel_debug.log is empty, no errors.
All nodes are sync and 'dash-cli masternode status' gives "status": "Masternode successfully started"

Any idea?

Not sure but they might be following different sides of a fork, they're at the same block height? "dash-cli getchaintips" might also give some indication if that's happening, if it is they should sort themselves out and all get back on the longest chain but starting with "- reindex" might sort things out quicker if they're being stubborn (can take up to a few hours).
 
Not sure but they might be following different sides of a fork, they're at the same block height? "dash-cli getchaintips" might also give some indication if that's happening, if it is they should sort themselves out and all get back on the longest chain but starting with "- reindex" might sort things out quicker if they're being stubborn (can take up to a few hours).
All nodes are in the same block height.
All nodes are installed from the scratch, new os and new dash install and new dowloaded blockchain.
Current status: 50% Watchdog_expired and other 50% enabled.

:mad:
 
Back
Top