GetLabel via Parallel.ForEach Loop Error

Post your questions regarding using the Vault and Fortress API in your programs.

Moderator: SourceGear

Post Reply
ejhansen71
Posts: 60
Joined: Thu Nov 13, 2014 10:12 am
Location: Lake in the Hills, IL

GetLabel via Parallel.ForEach Loop Error

Post by ejhansen71 » Thu May 26, 2016 8:42 pm

I have a process that does GetLabel to 5 different servers. To speed up the process, I used Async/Await for each of the server calls. I was told it would be better to use Parallel.ForEach. The problem is that I get the following error(see below). Is there a way to fix this problem? I use the code below the error to run the GetLabel.

vault>
<error>
<exception>System.IO.IOException: The process cannot access the file '\\ghprodetl4\src_trs\script\test.test' because it is being used by another process.
at VaultClientIntegrationLib.GetOperations.performLabelGet(String objectPath, String label, String labelSubItem, String labelWorkingFolder, String destPath, GetOptions go)
at VaultClientIntegrationLib.GetOperations.ProcessCommandGetLabelToLocationOutsideWorkingFolder(String objectPath, String label, String labelSubItem, GetOptions getOptions, String destPath)
at VaultCmdLineClient.VaultCmdLineClient.ProcessCommand(Args curArg)
at VaultCmdLineClient.VaultCmdLineClient.Main(String[] args)</exception>
</error>

var psi = new ProcessStartInfo
{
FileName = vaultRequest.PropertiesFile.Plink,
Arguments = string.Format("-P 44022 {0}@ghelsiapp \"GETLABEL -repository \"{3}\" \"{4}\" \"{5}\" -backup no -setfiletime modification -{1} \"{2}\" -verbose \""
, vaultRequest.Username, workingFolder, writeLocation, vaultRequest.Repository, ObjectPath, vaultRequest.ActualLabelName),
RedirectStandardError = true,
RedirectStandardOutput = true,
RedirectStandardInput = true,
UseShellExecute = false,
CreateNoWindow = true
};
var getLatestTask = new Task(() => ProcessDos(psi, vaultRequest));
getLatestTask.Start();
getLatestTask.Wait();
Thanks!

Eric

jclausius
Posts: 3702
Joined: Tue Dec 16, 2003 1:17 pm
Location: SourceGear
Contact:

Re: GetLabel via Parallel.ForEach Loop Error

Post by jclausius » Fri May 27, 2016 7:50 am

Some questions:

a) Is the process that shells out to the command line physically running on the Vault Server, or is it on a different machine?

b) Is the destination location for each iteration of your loop located on the same machine as the process running the command (depends on A), on the Vault Server itself, or another different machine altogether?
Jeff Clausius
SourceGear

ejhansen71
Posts: 60
Joined: Thu Nov 13, 2014 10:12 am
Location: Lake in the Hills, IL

Re: GetLabel via Parallel.ForEach Loop Error

Post by ejhansen71 » Tue May 31, 2016 8:11 am

Machine 1: Personal PC
Machine 2: Application Server
Machine 3: Vault Server
Machine 4: Linux server via SAMBA

Answers:
A) Different Machine (#2)

B) Command is started on 1 then executed on 2. Machine 2 then executes the command and sends it to 3 which sends it back to 2 and finally to 4 to write.
Thanks!

Eric

jclausius
Posts: 3702
Joined: Tue Dec 16, 2003 1:17 pm
Location: SourceGear
Contact:

Re: GetLabel via Parallel.ForEach Loop Error

Post by jclausius » Tue May 31, 2016 3:28 pm

Off Topic - Too bad you're not just using a part of the repository tree. You could do the same thing with Vault's Shadow Folders.

OK. Back to Business. The file lock is a bit of a nuisance, but perhaps this can be negated by looking at the process a bit closer. If I understood your description, here's what is happening:

For Loop {
a) Machine 2 issues a GETLABEL command line request to Machine 3 for label X.
a.1) Machine 3 authenticates, check permissions, etc. It then let's Machine 2 know the request has been granted.
a.2) Machine 2 requests files from Machine 3.
a.3) Machine 3 copies files from database to local temp space for retrieval
a.4) Machine 2 starts copying files from Machine 3 to Machine 2's temp space
a.5) Machine 2 expands files from Machine 2's temp space to Machine 4's location which is a network location
}

Since steps a thru e are static for each iteration, what if you modified this a bit:

a) Machine 2 issues a GETLABEL command line request to Machine 3 for label X.
a.1) Machine 3 authenticates, check permissions, etc. It then let's Machine 2 know the request has been granted.
a.2) Machine 2 requests files from Machine 3.
a.3) Machine 3 copies files from database to local temp space for retrieval
a.4) Machine 2 starts copying files from Machine 3 to Machine 2's temp space
a.5) Machine 2 expands files into a non-working folder which is a local location to Machine 2

For Loop {
b) Machine 2 next copies files from Machine 2's non-working folder from step f to Machine 4's location which is a network location.
}

This would eliminate some of the redundancy for static (non-changing) operations and may help with some parallelism as well if you place some of the copy within the loop into a thread. But be careful not to saturate your network.
Jeff Clausius
SourceGear

Post Reply