I wanted to throw this out there and see if anyone else has had a similar experience or know of anything to look into.
I have a client that makes an “app” whch alot of our other clients use. That app uses old FoxPro as the database. In short, there are literally THOUSANDS of little files that get accessed and transfered over the network if setup in a traditional client/server environment.
What I have seen for years now, is that putting the database files on a network share from server to client really slows down the “performance” of the app. If we attach the disk directly to a session host it runs very well, but over a network share it becomes almost unusable sometimes.
I have gone through the rounds with proximity groups, larger sized vm's and disks, teamed nics, i have looked at network traffic and lots of other things. I have never seen actual numbers or evidence of it but I am sure Azure throttles the traffic based on some kind of number of files transferred in a given period of time. I can copy 100GB worth of a few files over a share in a decent amount of time but copying these database files (literally thousands) which only end up being a fraction of the size (10GB for example) takes a very very long time.
Anyway, any thoughts would be appreciated. I just saw some new network optimized v6 VM sizes that looked interesting, but they are in preview and only in EastUS from what i saw AND only avalable in preview after getting approved to use them, but not sure even those would make much of a difference.
Azure Network "throtling" transfers of large number of files across vm's
I haven't had an opportunity/situation to look at a condition like this but this ‘sounds’ like the file share performance differences between native disk access, and the SMB protocol. While I agree, I wouldn't have expected performance to degrade significantly between disk and an SMB share that is less than 1ms of latency away, that is what it feels like to me, from experience.
To confirm, there is no vpn connection between the source and destination right? That woulud definitely slow things down, but it doesn't sound like that is the case here, just checking.
Honestly, I'm not sure what could be causing this, but I think using robocopy to test some raw file copies would be a good idea. Take a sufficiently large batch of these files and copy to a source location folder (so you aren't messing with anything live). Then from the desired destination for testing, use robocopy to test and mess with some of the options. First thing I would look at is the /MT= setting. If you set it to 1, it uses one vcpu thread (which I am pretty 99% sure means you'll be limited to a single thread) - and then start bringing it up. Compare the time it takes to copy the sample set at 1 thread, 4 threads, 8 threads, 16 threads…. if the performance dramatically goes up, I'd suspect its a function of the app itself copying with a single thread. If it does not go up with more threads, then it busts my first theory.
Fair warning, fi the client machine only has 2 cores, multithreading might improve for a bit but then drop down dramatically as it overwhelms the cores. I'm just trying to think of a way to break the problem down into smaller pieces. As the ever so wize Nerdio wizard Chuck Mikuzis has said many a time, troubleshoot one problem at a time. In this case, my gut says its something with -how- the app is transfering the high volume of files.
Passing that, I've used NTTTP Test virtual machine network throughput by using NTTTCP | Microsoft Learn - to validate network throughput (not completely conclusive) - when I've app team partners question whether the network is causing the particular client/server app to be ‘slow’. And to be honest, with v5 vm's, its typically been incredibly fast throughput.
If you make progress or resolve, do feel free to post back here - odds are good someone will find the results useful!
Please sign in to leave a comment.
Comments (1 comment)