Azure VPN connection

[My_King]

Hi All,

Hoping for some insight as to what may be wrong.

Current infrastructure:
100Mbps Up and down from VOX
FortiGate 60E as the firewall

Now we have an IPsec VPN to Azure and all is up and running. Servers are North Europe based.

My issue:

As soon as I start a file copy from on prem to Azure I get horrible speeds.
The copy is also not consistent.
Copy goes to 1.2MBs for 2 seconds and then pauses for 10 seconds, goes to 1.0MBs for 2 seconds then 0Kbps again for 10 seconds.
This makes copying any large file to Azure a big issue.

Logged the call with Vox and they said I need to multi thread my copy.

How do I tell Windows to Multi thread my copies when the program I use have no arguments to use.

Also anyone else having this issue?

 

[Traq]

Which program are you using to copy at the moment?

Are you copying directly into a VM with a mapped drive? If not, and it is just going to Azure Storage, use azcopy.

**Edit:

The South America outage caused quite a bit of issues between yesterday and today for us, not sure if it will be related or not.

 

[My_King]

Dude, first off, thanks for responding.

Azure storage is my only alternative I have now.

So basically to answer you question....Windows Explorer, on any on prem pc(COPY), navigate to \\AzurePC (Paste).
Build in Windows Explorer copy.

Logged a call with VOX and was advised the Windows copy does not do multi threading of uploads.

Your thoughts?

 

[My_King]

Also been having this for a while. So this is no outages.

I get the whole multi thread thing.

I was expecting this to work exactly as LAN.

Problem is when you have a program that access files through the VPN, how to do multi threading

 

[pancakes]

I don't think it has anything to do with multi threading. Your CPU most likely would handle a 2MB/s transfer on a single core without any problem. I think the issue may lie with the connection to Azure. Have you tried testing a file transfer without the VPN?

I know that Azure does have certain IOPS limits depending on the storage type you're using. If I recall (I may be mistaken) that there is a 1000 IOPS limit by default, which may only allow for a max throughput of around 4MB/s. What type of storage are you using?

 

[EasyUp Web Hosting]

I don't think it has anything to do with multi threading. Your CPU most likely would handle a 2MB/s transfer on a single core without any problem. I think the issue may lie with the connection to Azure. Have you tried testing a file transfer without the VPN?

I know that Azure does have certain IOPS limits depending on the storage type you're using. If I recall (I may be mistaken) that there is a 1000 IOPS limit by default, which may only allow for a max throughput of around 4MB/s. What type of storage are you using?

Not multithreaded CPU, what they mean is like using a download manager. It "divides" the file into pieces and start "multi" downloads at the same time. When you download a 100mb file, you don't get 1x 10mb/s when using a 100mbps fibre connection, you will get 10x1mb/s "connections" to that one file. Sorry, I am trying to simplify it for you, so I hope you get the idea.

@OP, not sure if you have googled at all, but have a look at: https://blogs.msdn.microsoft.com/granth/2009/12/07/multi-threaded-robocopy-for-faster-copies/

 

[TheGuy]

I don't it's specific to Azure. Smb protocol is just horrible over high latency links. It's way too chatty to for Wan usage, your only option is storage explorer or AZ copy.

When SA gets a local azure with lower latency it will improve the copy speeds a little.

 

[pancakes]

Not multithreaded CPU, what they mean is like using a download manager. It "divides" the file into pieces and start "multi" downloads at the same time. When you download a 100mb file, you don't get 1x 10mb/s when using a 100mbps fibre connection, you will get 10x1mb/s "connections" to that one file. Sorry, I am trying to simplify it for you, so I hope you get the idea.

@OP, not sure if you have googled at all, but have a look at: https://blogs.msdn.microsoft.com/granth/2009/12/07/multi-threaded-robocopy-for-faster-copies/

Thanks. The term multithreading is normally associated with parallel computing. I think Robocopy uses a form of interleaved multithreading that allows it to spawn connections for each chunk of data that is prepared and moved. SMB 3 and BITS should have at least addressed the instability with file transfers. Not 100% sure though.

 

[pancakes]

I don't it's specific to Azure. Smb protocol is just horrible over high latency links. It's way too chatty to for Wan usage, your only option is storage explorer or AZ copy.

When SA gets a local azure with lower latency it will improve the copy speeds a little.

Do you think local Azure will happen any time soon? I haven't heard much about them in SA, other than opening express routes.

Not sure if it will be a big success here considering the market size.