File Copy Pause Issues

DominionZA

DominionZA

Executive Member
Joined
May 5, 2005
Messages
8,306
Reaction score
73
Location
Edenvale, Gauteng
I have setup OpenMediaVault on my Mecer Dual Core Xeon server with 5 * 2TB drives in a Raid 5 array.
My network is gigabit all the way through.

Batch file copy operations from my Windows 8 machine sustain at around 135MB/s which is great.

The problem is I often get pauses at the end of a file copy for sometimes up to a minute before the next file copies. Happens mostly on big files (> 1GB).

I am using Teracopy on my Windows 8 machine. I have tried with the standard Explorer file copy but I still get the pauses and the transfer speed is not consistent so stick to Teracopy.

Any ideas what could be causing these pauses? They are so irritating.

I am no linux guru, so if I need to do something on the OpenMediaVault box please be specific :)
OpenMediaVault is based on Debian.
 
During that 'pause period' check if there is still HDD activity on the destination device. The progress indicator might show the copy is finished while in fact it's still writing data, dunno if it caches stuff in ram or what it does.

I've notice this writing iso images to flash drives. dd will tell me the copy is complete and it took x minute at y speed when in fact the light on the flash drive is still going crazy for some time after this.
 
It is probably buffered in more than one place (network buffer, filesystem buffer, etc). So when it shows that it is done, it is on the other side, but not yet on disk.
 
Nod said:
It is probably buffered in more than one place (network buffer, filesystem buffer, etc). So when it shows that it is done, it is on the other side, but not yet on disk.
Click to expand...

I figured it may have been something to do with that - but how to stop it?
I have disabled the write cache and nothing changed. I would rather see a slower transfer speed and no pauses between files.

Thinking maybe I should use hardware raid instead of software raid. It could be that causing the issue?
 
Software raid would have a definite influence. Caching also happens with hardware raid, but it would be less visible, if at all.
 
It may depend on which of OMVs file system options you selected for the drives. File systems like ext4 use delayed allocation (basically they buffer the data). No idea how to overcome this while using Win8 as the source OS. Maybe it is one of those things you have to live with.

Another factor may be that since you are copying from Windows there is a good chance that this is using CIFS (please correct me if I am wrong). I have experienced a lot of intermittent pauses with the Windows implementation of CIFS.
 
garyc said:
It may depend on which of OMVs file system options you selected for the drives. File systems like ext4 use delayed allocation (basically they buffer the data). No idea how to overcome this while using Win8 as the source OS. Maybe it is one of those things you have to live with.

Another factor may be that since you are copying from Windows there is a good chance that this is using CIFS (please correct me if I am wrong). I have experienced a lot of intermittent pauses with the Windows implementation of CIFS.
Click to expand...

Seems to be a combo then. I am using both CIFS and Ext4. Sigh...

Any way around it?
 
Tried to recreate this by copying a 1GB file from a Windows 7 machine to a Linux one. The Linux machine was set up as a samba server with Win7 connecting as a client.

The network has a wireless link at one point so it will be slower than yours. Once the copy had completed the HDD ran on for about three seconds and the operation was finished.

The most likely explanation for the lack of buffering was that the slower network speed allowed the HDD to keep up. Another potential factor is the Windows set-up. If you are using the Linux samba server then have a look at <http://wiki.samba.org/index.php/Windows7> for the Windows side settings. This improved things on my machine.
 
I have given up on OpenMediaServer as well as FreeNAS.

I am moving all my data off the raid array tonight. It will be done in the AM when I get up.

Then going to install Windows Server 2012 and use Windows Storage Spaces in Parity mode (RAID 5).
It offers the same extend pool options that OpenMediaVault gives so quite happy with that.
The whole house is all windows based machines and Xboxes so seems like the most logical route.

Prior to Win Server 2012 I did not know of any Software RAID 5 solutions which is why I went FreeNAS then OpenMediaVault. Tonight I discovered the Storage Spaces feature of Windows Server 2012 and Windows 8 so decided to make the jump.

I know windows well and almost clueless with anything Linux. Just taking up too much of my time trying to figure stuff out.
I will setup OpenMediaVault on one of my HP N40L Microserver machines and tinker around there.

Thanks for all the tips though guys. Appreciate the input.
 
You must log in or register to reply here.
Back
Top
Sign up to the MyBroadband newsletter
X