Slow read speed when mining?



  • @xugaxe

    I wouldn't say there is a "bottleneck", as a 70s scan time is reasonable for Burst's 4 minute block time. I guess you have a 55W/Socket-F Opteron-2423HE system from ~8 years ago. You can not expect the same throughput per GHz when comparing across 4 (!) generations.

    There might be enough resources left for another 4 drives (adding up to 12*8TB=96 TB) and you're still at or around 70 seconds. Expansion either by USB3-Hub or "real" 4-Port USB3-Controller (i.e. Sonnet Allegro Pro, Startech PEXUSB3S44V).

    Anyway, you want to reduce scan time (for what reason ? Look at the block time distribution and start to calculate..).

    With your existing setup, you may utilize more cores by having two threads per disk. Assuming you have two 4 TB plotfiles per disk, put them in different directories and configure your miner accordingly:

    j:\plots9
    j:\plots9b

    The disks should easily deliver 50 MB/s each for two data streams.
    This should bring your scan time down by a third to ~50s.

    OpenCL (GPU) assisted mining is a no brainer when you have only 2 PCI-express slots. There's not enough info on your config so we need to keep guessing - are all onboard SATAs in use ?

    Switching from Socket-F to a modern CPU (AVX2/AVX512) will

    • about quadruple your throughput per GHz
    • consume less energy, if that is a concern,
    • if chosen carefully, be a lot more scalable
      but that all depends on your choice of target-volume, -scan-time, budget for investment and operating costs.


  • @vaxman said in Slow read speed when mining?:

    @xugaxe

    I wouldn't say there is a "bottleneck", as a 70s scan time is reasonable for Burst's 4 minute block time. I guess you have a 55W/Socket-F Opteron-2423HE system from ~8 years ago. You can not expect the same throughput per GHz when comparing across 4 (!) generations.

    There might be enough resources left for another 4 drives (adding up to 12*8TB=96 TB) and you're still at or around 70 seconds. Expansion either by USB3-Hub or "real" 4-Port USB3-Controller (i.e. Sonnet Allegro Pro, Startech PEXUSB3S44V).

    Anyway, you want to reduce scan time (for what reason ? Look at the block time distribution and start to calculate..).

    With your existing setup, you may utilize more cores by having two threads per disk. Assuming you have two 4 TB plotfiles per disk, put them in different directories and configure your miner accordingly:

    j:\plots9
    j:\plots9b

    The disks should easily deliver 50 MB/s each for two data streams.
    This should bring your scan time down by a third to ~50s.

    OpenCL (GPU) assisted mining is a no brainer when you have only 2 PCI-express slots. There's not enough info on your config so we need to keep guessing - are all onboard SATAs in use ?

    Switching from Socket-F to a modern CPU (AVX2/AVX512) will

    • about quadruple your throughput per GHz
    • consume less energy, if that is a concern,
    • if chosen carefully, be a lot more scalable
      but that all depends on your choice of target-volume, -scan-time, budget for investment and operating costs.

    What do you mean by "real" usb 3.0 controller? The LogiLink I have is not good or what? Also I cant put any more on this motherboard as it only has 2 pcie slots so 8 drives max I guess?

    I also cannot use GPU miner as I dont even have a GPU in the system and not planing to get 1 as Im already out of pcie slots.



  • @xugaxe said in Slow read speed when mining?:

    What do you mean by "real" usb 3.0 controller? The LogiLink I have is not good or what?

    "real" in the sense of four USB controllers (chips) behind a pcie-bridge, as opposed to a single controller chip with a Hub fanning out to 4+ Ports.



  • @vaxman said in Slow read speed when mining?:

    @xugaxe said in Slow read speed when mining?:

    What do you mean by "real" usb 3.0 controller? The LogiLink I have is not good or what?

    "real" in the sense of four USB controllers (chips) behind a pcie-bridge, as opposed to a single controller chip with a Hub fanning out to 4+ Ports.

    Aha I understand, but they are so much more expensive (x10), would they make noticeable difference?



  • more cores with cpu miner the more plots can read at once. Thing about jminer it only reads for the current round so it doesn't read everything at once deliberately as it knows what won't generate nonces but still capable of more threads than any cpu. Some say sse4+ and avx2 theres no difference I say it is only marginal and better to get as many cores budget can supply or if cannot use gpu assistance. a intel i7 i9 amd threadripper or ryzon are great multicores



  • @xugaxe said in Slow read speed when mining?:

    Aha I understand, but they are so much more expensive (x10), would they make noticeable difference?

    You are reading ~200 MB/s, and most likely the SSE codepath in your "small" 2 GHz CPUs is the limiting factor here. But there is also the possibility that not-so-efficient USB-controllers and sharing each controller for 4 Disks is slowing you down. These "Pro" Quad-Port cards have fairly good driver support, just one thing you might try.

    I see your indidual core load at 75-80%, that is irritating. Hence I asked for optimized plots - if plots are not optimized (stagger much lower than length in the filename; id_start_length_stagger) it (may) induce a lot of disk seeking and idle compute cores.

    And, as you did not comment on any of my comments about scaling up:
    You have 4 of your 12 cores more or less idle.
    That just means you can either

    • read the existing disks with more than 8 threads, shortening wall-clock-scantime
      OR
    • plug another 4 disks into the system, and most likely your 70s scan time remains constant, but with a higher capacity per system


  • @vaxman said in Slow read speed when mining?:

    s you did not comment o

    Hello, thank you very much again.

    I think I already have the latest drives for the card installed, downloaded them from the net.

    I have plotted my disks with CPU plotter so afaik there is no need for optimizing right?

    And for the last thing you mentioned I cant add any more HDDs to the current system as I have all usb 3.0 ports filled and both pcie slots filled too so the best I can do is use full CPU power on existing HDDs right, but could you please explain me how to do that?



  • @xugaxe said in Slow read speed when mining?:

    And for the last thing you mentioned I cant add any more HDDs to the current system as I have all usb 3.0 ports filled and both pcie slots filled too so the best I can do is use full CPU power on existing HDDs right, but could you please explain me how to do that?

    I explained it above:

    With your existing setup, you may utilize more cores by having two threads per disk. >Assuming you have two 4 TB plotfiles per disk, put them in different directories and >configure your miner accordingly:

    j:\plots9
    j:\plots9b

    @xugaxe said in Slow read speed when mining?:

    I have plotted my disks with CPU plotter so afaik there is no need for optimizing right?

    There are a dozen different "plotters" and most of them can be configured to run either way (small stagger, high sequential output speed - or - high stagger, many disk seeks, slower i/o).

    Please look at your plotfiles filenames. The structure is
    id_start_length_stagger
    if stagger is not equal to length, your file is not "optimized".
    But that does not mean you need to optimize in all cases.
    I don't have time to explain that (again) and it can be found in the Burst wiki. If unsure just post a filename and I can tell you if you should do something about it.



  • @xugaxe said in Slow read speed when mining?:

    @vaxman said in Slow read speed when mining?:

    @xugaxe said in Slow read speed when mining?:

    What do you mean by "real" usb 3.0 controller? The LogiLink I have is not good or what?

    "real" in the sense of four USB controllers (chips) behind a pcie-bridge, as opposed to a single controller chip with a Hub fanning out to 4+ Ports.

    Aha I understand, but they are so much more expensive (x10), would they make noticeable difference?

    Try this:
    Copy a file from an external USB disk to the internal System disk or another fast target. You may cancel the process after 30 seconds, just look at the source disk read speed in Resource Monitor or the expanded Windows Explorer Copy Window.
    If you get less than 100 MB/s read speed this particular USB controller is shitty.
    There is a reason for the price difference - the "real" quad controllers have 4 individual controllers instead of one AND they are fast without straining the CPU. USB is quite complex and implementations differ a lot across chip vendors. The cheap 15$ cards use the cheapest (shittiest) controller chips on the market.



  • @vaxman said in Slow read speed when mining?:

    @xugaxe said in Slow read speed when mining?:

    And for the last thing you mentioned I cant add any more HDDs to the current system as I have all usb 3.0 ports filled and both pcie slots filled too so the best I can do is use full CPU power on existing HDDs right, but could you please explain me how to do that?

    I explained it above:

    With your existing setup, you may utilize more cores by having two threads per disk. >Assuming you have two 4 TB plotfiles per disk, put them in different directories and >configure your miner accordingly:

    j:\plots9
    j:\plots9b

    So that would mean I have to format a disk once again to make this kind of setup right? As now I have 1 plot file that is 8TB and cant make 2 from this?

    @xugaxe said in Slow read speed when mining?:

    I have plotted my disks with CPU plotter so afaik there is no need for optimizing right?

    There are a dozen different "plotters" and most of them can be configured to run either way (small stagger, high sequential output speed - or - high stagger, many disk seeks, slower i/o).

    Please look at your plotfiles filenames. The structure is
    id_start_length_stagger
    if stagger is not equal to length, your file is not "optimized".
    But that does not mean you need to optimize in all cases.
    I don't have time to explain that (again) and it can be found in the Burst wiki. If unsure just post a filename and I can tell you if you should do something about it.

    At least this makes sense to me, after checking it I have length same as stagger so it means at least this part I got good right? :D



  • @vaxman said in Slow read speed when mining?:

    @xugaxe said in Slow read speed when mining?:

    @vaxman said in Slow read speed when mining?:

    @xugaxe said in Slow read speed when mining?:

    What do you mean by "real" usb 3.0 controller? The LogiLink I have is not good or what?

    "real" in the sense of four USB controllers (chips) behind a pcie-bridge, as opposed to a single controller chip with a Hub fanning out to 4+ Ports.

    Aha I understand, but they are so much more expensive (x10), would they make noticeable difference?

    Try this:
    Copy a file from an external USB disk to the internal System disk or another fast target. You may cancel the process after 30 seconds, just look at the source disk read speed in Resource Monitor or the expanded Windows Explorer Copy Window.
    If you get less than 100 MB/s read speed this particular USB controller is shitty.
    There is a reason for the price difference - the "real" quad controllers have 4 individual controllers instead of one AND they are fast without straining the CPU. USB is quite complex and implementations differ a lot across chip vendors. The cheap 15$ cards use the cheapest (shittiest) controller chips on the market.

    Cant even do that right now as I have only about few GB of disk space on my internal HDD :D But checking the specs of the card it shouldnt be the problem?



  • @xugaxe

    Your plot files are optimized - perfect.

    You can't test the USB controllers for speed - no problem. The system works. You can test that in the next build, just buy different parts and cross test.

    You have plotted 8 TB files and can not have multiple threads per disk - also no problem.

    Depending on the case, you may add disks to the mainboards SATA ports to get those idle 4 cores working. SATA-2 is no problem.

    And overall, I say 70s scantime is no problem and I would not bother to replot or redesign this machine.

    Good luck mining !



  • @vaxman said in Slow read speed when mining?:

    @xugaxe

    Your plot files are optimized - perfect.

    You can't test the USB controllers for speed - no problem. The system works. You can test that in the next build, just buy different parts and cross test.

    You have plotted 8 TB files and can not have multiple threads per disk - also no problem.

    Depending on the case, you may add disks to the mainboards SATA ports to get those idle 4 cores working. SATA-2 is no problem.

    And overall, I say 70s scantime is no problem and I would not bother to replot or redesign this machine.

    Good luck mining !

    Hey, thanks :)

    I was just asking so many questions because for the next rig I want to get out as much as I can so my ROI is faster if you understand my concern :)

    Also checking the specs of my mobo it is SATA II, it wont affect the speed if I connect SATA III disks?



  • @xugaxe said in Slow read speed when mining?:

    Also checking the specs of my mobo it is SATA II, it wont affect the speed if I connect SATA III disks?

    SATA-2 has a (theoretical) throughput of 300 MB/s, 11 times what your system does per thread/disk; 27 MB/s.
    You may chuck the bare HDDs from their USB enclosures (voiding warranty).



  • @vaxman said in Slow read speed when mining?:

    @xugaxe said in Slow read speed when mining?:

    Also checking the specs of my mobo it is SATA II, it wont affect the speed if I connect SATA III disks?

    SATA-2 has a (theoretical) throughput of 300 MB/s, 11 times what your system does per thread/disk; 27 MB/s.
    You may chuck the bare HDDs from their USB enclosures (voiding warranty).

    Awesome, thank you very much :)

    Ill buy 4 more SATA HDDs and use full potential of current setup then.


 

Looks like your connection to BurstCoin - Efficient HDD Mining was lost, please wait while we try to reconnect.