What's the deal with these NOT confirmed deadlines?



  • Here's the case: @luxe @haitch @blago @Propagandalf

    Every few blocks or so, jminer will report 'strange dl source' and the corresponding deadline is rejected by the wallet. I get very similar errors when running blago's miner if I run it alongside jminer and both miners find and submit the same faulty deadline, which suggests it's not an issue with jminer. Mining capability doesn't seem to be hindered at all and I am forging one or two blocks a day just as the calculator predicts.

    Here's one of the best examples I've seen. Not one, but two instances of this particular error, followed by a rejected deadline. What makes this once special is that in both instances an identical deadline was found with a DIFFERENT nonce, submitted, and confirmed all in the same round!

    0_1496023785514_deadlines.jpg

    Kudos to @luxe for the nice new jminer debug and resubmit features. 🙂

    So to recap, the sequence of events was as follows:

    101828 send (solo) nonce 100456385
    strange dl source ...
    101828 NOT confirmed!

    101828 send (solo) nonce 69945281
    101828 confirmed!

    55249 send (solo) nonce 356987563
    strange dl source...
    55249 NOT confirmed!

    55249 send (solo) nonce 326476459
    55249 confirmed!

    The same deadline being confirmed from a different nonce doesn't always happen alongside the strange dl error, but it seems to happen too frequently to be pure cooincidence. My setup:

    I have 96TB of optimized plots on twelve 8TB drives.
    All plots were created in direct mode with GPU Plot Generator.
    I mine solo on a dedicated machine with a local wallet on the latest version: 1.2.8.

    One theory is that GPU plot generator is somehow creating semi-corrupted plot files where some nonces are faulty. However, it seems unlikely that a plotting algorithm would create an incorrect nonce only about 1% of the time. On top of that, many of the exact same plot files are currently being mined and they report valid deadlines and some have even forged blocks.

    Another theory is that the algorithm for scanning plot files is shared by both (or all?) miners and that this algorithm is finding and reporting deadlines which are false positives... but only sometimes... on certain plot files... for some reason. If this is the case, then there's not really anything to worry about.

    Any suggestions or comments on what might be causing this or what I might have done TO cause it? Are you guys seeing these errors too?


  • admin

    @sevencardz
    I have seen this once on solo mining myself .. but as it did not happen again i saw no big need fixing/researching it ... the only explanation i can think of, is a bug in jminer ... maybe some method that is not syncronized yet and two threads working on same attributes could cause this in some cases ... however if this only happens once a day or even less ... i would not worry about that very much ... it is very hard debugging into multi-thread parts or even reproduce this while debugging. I will try to harden some more sensitive parts of the round code, to prevent such exceptional cases.
    Thank you for always pointing on such things, i'm personally not so critical ... for me the current version works quite well ... i monitored a little issue on startup, that somehow could cause same round start twice leading to spammed exception logs ... found that more important to fix, like some other things like the 'restart round' on same block and changed generations signature issue etc. always a work in progress 🙂


  • admin

    @sevencardz said in What's the deal with these NOT confirmed deadlines?:

    strange dl source ...

    I have never seen that error, and have absolutely no clue on debugging it.


  • admin

    @haitch not sure if i understand you right, if you want to debug you can do it by setting 'debug=true'


  • admin

    @luxe No, I meant I had no idea where it could be coming from - I've never seen a "strange DL source" error.


  • admin

    @haitch Did you ever see a 'NOT confirmed' ? If you turn 'debug=true' the current version will log infos about the 'stragne DL source' that leads to 'NOT confirmed'.

    In case of a really corrupted plotfile ... the plot will show up over and over again, and you can consider replotting that starnge one. Thats the whole intention.


  • admin

    @luxe Thanks for the info. I( have some GPU plotted files that I consider suspect, they get not confirmed or "really" bad deadline message. Will enable the debug and see if I can track them down.



  • @luxe I see it fairly regularly and I have log files full of them, but I really don't think this specific error is a problem with jminer, or if it is then maybe it's a problem with the mining algorithm that both miners are using. I've gotten Blago's miner to report similar errors in debug mode. jminer and blago's miner both work quite well for me too and like I said, my earnings don't seem to be suffering.

    @haitch Interested to hear what results you get on your setup as well.



  • @cryo Since you're active in the forums now, could you take a look at the issue above?

    @haitch Have you had a chance to test your GPU plots vs your CPU plots? Interested to hear if one or both are throwing this error for you or not.

    I'm getting ready to start plotting on a new machine with Xplotter, so I will be able to compare results myself soon. Will report back here when I do.



  • @sevencardz It may be related to this issue.

    The GPU plot generator comes with a verify command to verify plots files based on freshly computed nonces.
    As you have a large mining power, the fact you experience it from time to time could help to debug this.
    Can you try to verify one file reported by the miner? If it's the same problem as the linked issue, the missing/faultly scoop should be located at the end of the file. So a call like the following should suffice:

    # We generate a test file located at the end of the faultly file
    ./gpuPlotGenerator generate buffer Z:/13668371040637458609_100699660_800_800
    # We verify it against the actual file
    ./gpuPlotGenerator verify O:/13668371040637458609_99174400_1525760_1525760 Z:/13668371040637458609_100699660_800_800
    

    If it's confirmed, I'll track down this bug and help you restore the missing scoops (maybe by adding a repair command).



  • @cryo All this time and I never knew there was a verify command. Here's an offending plot, but the verify command doesn't seem to report any issues:

    2017-06-19 17:49:52.972  INFO 2740 --- [roundPool-1] burstcoin.jminer.JMinerCommandLine       : dl '268656' send (solo) [nonce '320348789']
    2017-06-19 17:49:53.160 DEBUG 2740 --- [SimpleAsyncTaskExecutor-137481] burstcoin.jminer.JMinerCommandLine       : strange dl source 'A:\13668371040637458609_318883840_1525760_1525760'
    2017-06-19 17:49:53.160 DEBUG 2740 --- [SimpleAsyncTaskExecutor-137481] burstcoin.jminer.JMinerCommandLine       : strange dl file chunks '1', parts per chunk '2', block '372896'
    2017-06-19 17:49:53.160  INFO 2740 --- [SimpleAsyncTaskExecutor-137481] burstcoin.jminer.JMinerCommandLine       : dl '268656' NOT confirmed!  [ 3d 2h 37m 36s ]
    2017-06-19 17:49:53.160 DEBUG 2740 --- [SimpleAsyncTaskExecutor-137481] burstcoin.jminer.JMinerCommandLine       : strange dl result '50384046118002', calculated '268656' block '372896' nonce '320348789'
    

    0_1497913660995_Untitled.jpg



  • @sevencardz Can you test around the nonce reported in the miner output (320348789 from your example output):

    ./gpuPlotGenerator generate buffer C:/13668371040637458609_320348689_200_200
    ./gpuPlotGenerator verify A:/13668371040637458609_318883840_1525760_1525760 C:/13668371040637458609_320348689_200_200
    


  • @cryo Oh yes thank you, seems I can't maths. I will try that once I'm home in a few hours. 🙂



  • @cryo OK, I switched to another drive, but I think I've got the hang of it now.

    First, the offending plot and nonce:

    2017-06-16 18:28:48.657  INFO 8580 --- [roundPool-1] burstcoin.jminer.JMinerCommandLine       : dl '285118' send (solo) [nonce '457319171']
    2017-06-16 18:28:48.891 DEBUG 8580 --- [SimpleAsyncTaskExecutor-19339] burstcoin.jminer.JMinerCommandLine       : strange dl source 'E:\13668371040637458609_456202240_1525760_1525760'
    2017-06-16 18:28:48.907 DEBUG 8580 --- [SimpleAsyncTaskExecutor-19339] burstcoin.jminer.JMinerCommandLine       : strange dl file chunks '1', parts per chunk '2', block '371829'
    2017-06-16 18:28:48.907  INFO 8580 --- [SimpleAsyncTaskExecutor-19339] burstcoin.jminer.JMinerCommandLine       : dl '285118' NOT confirmed!  [ 3d 7h 11m 58s ]
    2017-06-16 18:28:48.907 DEBUG 8580 --- [SimpleAsyncTaskExecutor-19339] burstcoin.jminer.JMinerCommandLine       : strange dl result '6440749087779', calculated '285118' block '371829' nonce '457319171'
    

    0_1497999974475_Untitled.jpg



  • @sevencardz Sounds like it's not only affecting the last scoop. I'll run some tests on my side to try to reproduce this.