Major issues with Branching

[Sourav]

We are using the following version of Vault:

[6/2/2006 2:59:03 PM] Version Check: This Vault client is version 3.1.6.3658
[6/2/2006 2:59:03 PM] Version Check: Your Vault server is version 3.1.6.3658


We are one week from release and just branched our code to enable parallel development from the mainline trunk and branch and have encountered several high level issues with Vault.

1. Branched files are stamped with the branch date rather than the actual last modified date of the file. See Figure 1 (mainline trunk) vs Figure 2 (branch).

2. Branched files are not branched with their original labels from the mainline trunk. See Figure 2 vs Figure 1.

3. Show History of branched files displays incorrect labeled dates and sort order by version (See Figure 3).


First, others have already posted about issue #1 above (http://support.sourcegear.com/viewtopic.php?t=5383). We need to know when this fix will be available because it affects our ability to support our customers and troubleshoot problems quickly.

Second, as part of our build process, we label the repository, then using the GETLABELDIFFS command, we get only files that have changed between a previous label and the new label. Without labels in the branch, we cannot GET the correct set of files to build.

Third, show history on a branched file displays all label actions, however, their dates are wrong and it is sorted incorrectly. This is inconsistent with show labels on the same file because show labels only displays new labels applied after branching. Is this a bug in the branch function or in the GUI client display?

We migrated from VSS to Vault in order to use its advanced branching and labeling features. However, to meet the release date we must find a quick fix or workaround to our current situation.

Please advise us on how to resolve these issues.

Thank you

[lbauer]

1. Branched files are stamped with the branch date rather than the actual last modified date of the file. See Figure 1 (mainline trunk) vs Figure 2 (branch).

The branch modifies the file by creating a new version, so currently, the branch time (current time) is used for the modification time of the file. We do have a feature request logged to use the source object's modification time as the modification time for the newly branched item. I've added your "vote." This feature is being considered for Vault 4.0, due out later this year.

2. Branched files are not branched with their original labels from the mainline trunk. See Figure 2 vs Figure 1.

The labels do not show up in Show Labels, but they do appear in the file history, and you can get the label from Show History on the file. We have a feature request logged for Vault 4.0 for the labels to appear in Show Labels on branched items. I'll add your "vote."

3. Show History of branched files displays incorrect labeled dates and sort order by version (See Figure 3).

There was a fix in Vault 3.1.8 that put labels next to the version that was labeled, rather than putting them at the time of the branch in History.

Label dates in History (branched or original) are the date of the version to which the label was applied. So if version 5, dated May 12, 2006 is labeled on June 1, the label date in Show History will be May 12, 2006. I believe we are planning to change this in a future release, so that the label appears next to the version that's labeled, but still carrying the date the label was applied.

as part of our build process, we label the repository, then using the GETLABELDIFFS command, we get only files that have changed between a previous label and the new label. Without labels in the branch, we cannot GET the correct set of files to build.

The labels are still in file history. Have you tried doing a Get Label and then comparing the two labels?

[Sourav]

1. Branched files are stamped with the branch date rather than the actual last modified date of the file. See Figure 1 (mainline trunk) vs Figure 2 (branch).

This feature is being considered for Vault 4.0, due out later this year.

Will this feature definitely make it into 4.0 and what time frame can we expect the release? We are willing to wait before looking for another source control solution, but we must know the time frame.

2. Branched files are not branched with their original labels from the mainline trunk. See Figure 2 vs Figure 1.

The labels do not show up in Show Labels, but they do appear in the file history, and you can get the label from Show History on the file.

This is fine if we are using the GUI client. However, our build process uses the console client and cannot get the label on the file from show history.

3. Show History of branched files displays incorrect labeled dates and sort order by version (See Figure 3).

There was a fix in Vault 3.1.8 that put labels next to the version that was labeled, rather than putting them at the time of the branch in History.

Do we need to re-branch our code after upgrading to 3.1.8 or is this a display only fix?

Label dates in History (branched or original) are the date of the version to which the label was applied. So if version 5, dated May 12, 2006 is labeled on June 1, the label date in Show History will be May 12, 2006. I believe we are planning to change this in a future release, so that the label appears next to the version that's labeled, but still carrying the date the label was applied.

Is this a version 3.1.8 behavior since we don't see it in our version 3.1.6 installation?

as part of our build process, we label the repository, then using the GETLABELDIFFS command, we get only files that have changed between a previous label and the new label. Without labels in the branch, we cannot GET the correct set of files to build.

The labels are still in file history. Have you tried doing a Get Label and then comparing the two labels?

Clarification: we label the repository at a specific folder path. The mainline trunk displays these labels on Show History (See Figure 4). However, the branch only shows labels applied after branching (See Figure 5). When we run GETLABEL on the branched folder, it returns the following exception.
Note: Label "1085q11.332" was applied to the mainline trunk one day before we branched.

D:\>vault.exe GETLABEL "$/Branchesxxx/1085 Release/1085q11 (3.8.00)/Data_Files/CD-Db/EDM" "1085q11.332"
<vault>
<error>
Could not find label "1085q11.332" created at item "$/Branchesxxx/1085 Release/1085q11 (3.8.00)/Data_Files/CD-Db/EDM".
</error>
<exception>
System.Exception: Could not find label "1085q11.332" created at item "$/Branchesxxx/1085 Release/1085q11 (3.8.00)/Data_Files/CD-Db/EDM".
at VaultCmdLineClient.VaultCmdLineClient.ProcessCommandGetLabel(String reposItem, String label, String labelSubItem)
at VaultCmdLineClient.VaultCmdLineClient.ProcessCommand(Args curArg)
at VaultCmdLineClient.VaultCmdLineClient.Main(String[] args)
</exception>
<result success="no" />
</vault>

It is vital that we get some idea of the release date for 4.0 and if it will address these issues. Our VP of Engineering has expressed his willingness to wait for a fix, however without knowing when it will be available, he is considering other source control solutions.

[lbauer]

Will this feature definitely make it into 4.0 and what time frame can we expect the release?

We have not firmed up the feature/bug list yet for Vault 4.0. Vault 4.0 will be released late this year. I have bumped this up in priority, though.

This is fine if we are using the GUI client. However, our build process uses the console client and cannot get the label on the file from show history.

It might help if we understood more about your build process. Since this is a branch, is there a reason you don't have two distinct builds? One for the old branch and one for the new branch?

Do we need to re-branch our code after upgrading to 3.1.8 or is this a display only fix?

This fixes the display. We made several history display changes in Vault 3.1.7 and 3.1.8.

[Sourav]

It might help if we understood more about your build process. Since this is a branch, is there a reason you don't have two distinct builds? One for the old branch and one for the new branch?

Let me summarize our build process. We do have two distinct builds, one from the branch and one from the mainline trunk. Every time we build, whether from the branch or mainline, we label the branch or mainline repository path. When we build, we use GETLABELDIFFS to get a set of data files ONLY if they were changed or updated since a "baseline" label. For example, if there are 100 data files and 20 were updated since the "baseline" label, we only get 20 data files.

Last week we created a branch in preparation for the release and tried to build from the branch. However, because the branch did not carry over the labels from the mainline (using GETLABEL, GETLABELDIFFS, or Show Labels from the GUI) we never get the 20 data files during the build process, in fact we got no data files. We need the labels in the branch in order to incrementally get data files that were changed. The build process from the branch or mainline is identical and is controlled by specifying the repository path and specific labels.

I hope this provides a better understanding of our build process.

If there is another method of achieving the same result, please advise.

[jclausius]

I see part of your problem. It is not a problem all of the time, but rather a problem for the first build after a branch. There are some possible work-arounds, and others may want to chime in:

A)
Note - you will need Vault 3.1.8 or Vault 3.1.9 for this to work, but I believe you could place a dummy file in the main trunk. In essence, the file is now a place holder used to determine the labels. So when you look at labels for the dummy file, the previous label should be the last label applied to the branch or if no labels have been applied to the branch, the last label applied to the trunk (just before the branch).

Once you've discovered the label names, your normal build process resumes.


B)
In our build process, we determine the last version of a folder by the history command. Then an entire get by folder version is done on that folder into an empty directory on disk. The build process uses those files for building. The folder get is for about 4100 files and takes just over 5 minutes.

This ensures a "clean" folder for building. Depending on how many files are in the label, it may take a bit of time, but the tradeoff is no remnants of other files retrieved from somewhere else, or possibly temp files left over from a build, etc. It is just a brand new folder with nothing but the source files for the label.

[ian_sg]

As the resident build-lackey, I have a couple of other thoughts on this:

In our world, creating a new branch is relatively rare. The build scripts require tweaking to build from a new branch. I assume you do the same, as you have to tell your script the new repository path and presumably a new disk path where the code is built.

It seems to me that your build system would need to do a complete get after the branch, retrieving ALL the source, not just what's changed. If the labels were carried over this wouldn't happen unless your build script somehow handles a new branch as a special case.

Are you actually parsing the output of GETLABELDIFFS to determine what source to retrieve, or is that just the mechanism for determining if there's been a change? You can (and probably should) let Vault figure out what files have changed and need to be dowloaded.

Jeff briefly described how our build works. You could very easily modify it to do an incremental get (retrieve only the changes) since that appears to be important in your environment:

1. Use the VERSIONHISTORY command to determine if there's been a source change. You can limit the output to 1 row and see if the version that's returned is greater than the last time you polled.

2. Use the GETVERSION command to retrieve the source into a working folder. This folder would be empty on the first build after a branch, but after that Vault will automatically figure out what's changed and only download what's necessary. It will even download just the changes (as opposed to the full files) for those files that have changed, so if your files are big and your network slow, this is a big win.

3. Build as before

4. Use the LABELVERSION command to label the code that makes up this build.

5. Goto 1

On a new branch, the previous version in step 1 is effectively 0, so you automatically get a build on the first poll.

Finally, you might want to investigate CruiseControl.NET. It is an open source server written in .NET that works with Vault and automatically gives you the behavior described above, provided you use a recent build and Vault 3.1.8.

There's some additional information about CruiseControl.NET and its integration with Vault, including the algorithm above, on my blog.