Shelveset Usage Scenarios, Limitations and Bugs

[Virgil Smith]

We have recently upgraded to Vault Pro 5.1.1 and have had a horrible time trying to adapt shelvesets to any meaningful usage. This is unfortunate as several of my users (effectively) requested shelveset support prior to our upgrade.

I see three main usage scenarios for (or views/facets of) shelveset functionality: Backups, Machine Migration/Hopping, and what I would call "micro-branches". Unfortunately, 2 out of 3 are unsupported.

A. Backups
The repetitive "end of day" or similar backup usage seems fairly obvious. Unfortunately, it is not supported due to 2 factors.

1) When creating a shelveset there is no support to pick an existing shelveset to replace (other than re-typing the name very very carefully).

2) When unshelving Vault complains about every file in the shelve set that MATCHES a file in the working folder. See [url]http://support.sourcegear.com/viewtopic.php?f=5&t=13173[/url]. The solution to this is NOT complicated configuration dialogs or "suffering-through" as this is the EXPECTED/NORMAL condition. Furthermore, other operations (add/delete/rename/move) seem to fail entirely.

B. Machine Migration
This usage actually works. :mrgreen: Of course, this is because undoing the local changes is acceptable in this usage. This has "move semantics" whereas backups tend to have "copy semantics".

C. Micro Branches
I was pleased to discover that users are allowed to store multiple shelvesets and even "share" them. This (should) enable uses for which the only analogy I know of from "traditional" SCM is branches. Having "light-weight" / "short-term" branch support seems very very useful. By this I mean a branch that does not bother with full history tracking, does not pollute the repository folder structure, etc.

However, key support is missing for any such usage:
1) There is no support for merging a shelveset to the "trunk" (or rather to the "micro-branch"/shelveset represented by the pending change set). See item A.2.

1.1) There is no support for merging changesets.

2) There does not seem to be a way to diff a shelveset against the current version of the repository (only the "working folder" or "base-line" or another shelveset).

3) There does not seem to be any way to create a "full branch" from a shelveset. I can't even determine the repository/sub-folder version from which a shelveset was made. Even if I could I would have to: (a) branch from that version (b) change the working folder for the "trunk" to the working folder of the branch (c) unshelve (d) restore the working folder for the "trunk".


So... this is an OK feature that is agonizingly close to being a REALLY USEFUL feature.

[Beth]

I will need to tackle each of these items you listed individually and will do that in the next few posts. Before I begin, I want to say that shelving is to be viewed as temporary. It's not really meant to be a place that other parts of the Vault repository can interact with. For example, rather than having it behave "like" a branch, once should just make a branch. It is possible to delete branches once they are no longer needed.

I can put in feature requests to make shelves be able to interact more with different parts of the repository.

Another sceneario I've seen shelve sets used for is code review. A user needs to have code approved by management or QA before it is checked in. The other user can unshelve the code, review it, and let the original coder it met the necessary criteria. Unless the original coder made changes to the code, there is no need to unshelve before checking in. If the user did move onto other coding, that user can shelve the new code, unshelve the reviewed code, check in the reviewed code, then unshelve the code that was being worked on before the check in and continue working.

[Virgil Smith]

OK, let me try to clarify a couple things:
[quote="Beth"]It's not really meant to be a place that other parts of the Vault repository can interact with. For example, rather than having it behave "like" a branch, once should just make a branch. It is possible to delete branches once they are no longer needed. [/quote]
I was afraid my analogy would go wrong, I just don't have a better one to draw upon. Yes, one can make a lot of branches to perform backups, machine migrations, code reviews, etc. In fact, many people do. But like the old SVN "tags are labels" lie, it is incredibly awkward in many circumstances. (BTW: This was a significant part of why we bought Vault.)

A shelveset is a "change-set", and it is cool to be able to hang on to more than one at a time. However, when multiple "change-sets" interact you need merging, which is really what I'm after. Merging makes me think of branches, but what we are dealing with here is more an extension of merging in the CVS copy/merge model.

[quote="Beth"]Another sceneario I've seen shelve sets used for is code review. A user needs to have code approved by management or QA before it is checked in. The other user can unshelve the code, review it, and let the original coder it met the necessary criteria. ... If the user did move onto other coding, that user can shelve the new code, unshelve the reviewed code, check in the reviewed code, then unshelve the code that was being worked on before the check in and continue working.[/quote]
Great Example, but that just points right back to the issues I pointed out:

If the code review is just a visual inspection (rather than compiling/testing), the obvious thing to do is compare the shelveset contents to the current version in the repository, which is not supported (repeat of issue C-2). Instead the reviewer has to shelve his own work, pull down the shelve set, then diff his working copy against the repository. (Yes to literally see what the coder was doing one should diff against the "baseline", but only doing that just increases the chances that the original coder will run into checkin time merge issues; which almost by definition he won't be qualified/approved to handle.)

"Then if the user moved onto other coding" he is going to have to "unshelve the code that was being worked on", which means "merging that shelveset to the trunk", which Vault does not support. (see issues A-2 / C-1). Consider for instance if his new work presumed that some or all of his prior changes would be accepted (which is fairly likely). This means that changes in this new shelveset existed in the original shelveset and were applied to the "trunk" (the current repo version) while the shelveset was "shelved". The result? Vault will complain (prompt) about every matching modified file and any adds/deletes/renames/moves will cause the unshelve to fail and the user will have to restart the unshelve operation after unchecking that file in the set.

Vault could use logic similar to that used to merge branches to help resolve conflicts when unshelving. If Vault knows what version the shelveset was based on it can spot changes in the shelveset that match changes applied to the repository. For instance, it need not abort an unshelve containing a rename operation just because the working folder already has an item with the new name when, that item is there because it was a renamed copy of the same item.


In fact these same problems exist (to a far lesser extent) when doing "normal" copy/merge checkins and if Vault used the sort of logic I just mentioned, 90+% of my user's assistance requests/complaints would probably stop. Heck, the inability to commit a delete of a no longer existing item is the top reason I have to go help a user. However, when working in that direction, most of Vault's GUI elements are designed to help the user manually fix resolve the problems. Item status displays, "resolve merge status", etc.

[Beth]

Backups
The repetitive "end of day" or similar backup usage seems fairly obvious. Unfortunately, it is not supported due to 2 factors.

Using shelve as a way of backing up code what is on a user’s computer is fine. When I mentioned branching, I didn’t intend to suggest a branch be created as a way to backup what is on a user’s computer.

A shelveset is a "change-set", and it is cool to be able to hang on to more than one at a time. However, when multiple "change-sets" interact you need merging, which is really what I'm after. Merging makes me think of branches, but what we are dealing with here is more an extension of merging in the CVS copy/merge model.

I’m assuming that the user is not working from the code that was shelved then, is that correct?

1) When creating a shelveset there is no support to pick an existing shelveset to replace

I have a feature request open for this functionality that I will add your “vote” to.
F: 14923

There is no support for merging changesets.

I have a feature request open for this functionality that I will add your “vote” to. I also have a work-around that can do this. It’s not a very graceful work-around, but once I finish these responses, I’ll post instructions.
F: 15592

2) There does not seem to be a way to diff a shelveset against the current version of the repository (only the "working folder" or "base-line" or another shelveset).

I can put in a feature request for this functionality. Currently, you would have to unshelve the shelve set to diff against the current repository.
F: 15880

3) There does not seem to be any way to create a "full branch" from a shelveset. I can't even determine the repository/sub-folder version from which a shelveset was made. Even if I could I would have to: (a) branch from that version (b) change the working folder for the "trunk" to the working folder of the branch (c) unshelve (d) restore the working folder for the "trunk".

Branches can only be created from items checked into Vault. If you think you may be branching, you may want to either create the branch first, and then code your changes, or Label the parent folder first so that you know where you are starting from. Currently the shelve feature doesn’t allow one to check in to a different location than where the shelve was made from.

In the unshelve dialog, the path the shelve set came from is listed. Also, only shelve sets for the repository you are currently in are shown.

I can put in a feature request for this as well.
F: 15881

Yes, one can make a lot of branches to perform backups, machine migrations, code reviews, etc.

Sorry I gave the wrong impression, but branches wouldn’t be a good choice for those actions. I was thinking more about request #3 when mentioning the branching or #1 for merging.

[Beth]

Here's one way I found of merging changes from 2 shelve sets.

Go to Vault Tools – Options – Diff/Merge. In the arguments for Diff, remove the arguments /ro1 or /ro2 or both. The /ro argument is for setting one of the panes to read-only. Removing that makes the panes writable. Then click OK.

Unshelve one shelve set to the working folder.

Click Unshelve and select the second shelve set. Click Details. Right-click an item in the shelve set details and select Show Differences. Select the option to diff against the working folder copy and click OK.

Inside the diff window, identify which side is the working version. That is what you will merge to. Select any items in the left window and either apply those changes to the working version pane, or append or prepend those changes. Once the working version is finished, save the changes and close the diff window. Close the shelve set windows without unshelving. Now if you look at the working version of the file, you will find the changes you merged in, and you can check that file in.

[Beth]

Finally, I wanted to understand a little better how your users are working. It sounds like they start working on a feature and create a shelve set for the changes for that feature. Then they perform a new get of the original code and code a different feature. Does this sound correct? Then, before checking in, both features need to be merged into one check in?

Is there a lot of delay between when they start coding and when they check in?

For the sake of having something that would work for you while waiting for changes, is it possible for your users to check in more often? Through the use of comments and labels, one can identify what stage their code is at and possibly have a little more control.

Do you only have the one trunk for your code, or are there branches as well?

Do you ever have bug fixes that have to go in after code is released?

[Virgil Smith]

Thanks for the feature requests/votes; and the manual merge tip (removing the /ro Diff options).

I guess I'd expand the manual merge by opening a second client window to queue up any adds/deletes/moves/renames while viewing the shelveset in the first instance... A quick test shows that will work though it can cause "bad" items in the commit set on the first instance (same file both modified and deleted). (However, paper/pencil or screenshots and using only a single instance should lower that to the normal error rate and they are easily corrected.)

[quote="Beth"]Finally, I wanted to understand a little better how your users are working. It sounds like they start working on a feature and create a shelve set for the changes for that feature. Then they perform a new get of the original code and code a different feature. Does this sound correct? Then, before checking in, both features need to be merged into one check in?[/quote]

The typical case is more like: They start working on a feature, then they find that the rest of the team can't wait for the feature as a whole to be completed and tested, because some critical piece(s) are needed to support other development (or testing or training). So, they must set "most of the feature" aside, code/test the smaller change(s), checkin those changes, then resume work on the "complete feature". So the "merge" isn't at the point of check in, it's at the point of unshelving the in progress "complete feature". So... unshelve will always fail in the typically case.

We are also in the process of converting our main product from linking against .NET 2/3 and .NET 4 and trying to use a shelveset to hold onto the changes while we get them all lined up.

[quote="Beth"]Is there a lot of delay between when they start coding and when they check in? [/quote]

Probably less than a week.

[quote="Beth"]For the sake of having something that would work for you while waiting for changes, is it possible for your users to check in more often? Through the use of comments and labels, one can identify what stage their code is at and possibly have a little more control. [/quote]

No. Checking in code that won't build, crashes, or flat out does not function is not acceptable. OK, there are a lot of subdivisions that don't interact with the rest of the system and no-one uses until they are mostly completed. However, once a piece has passed that stage dysfunctional checkins are not permissible.

[quote="Beth"]Do you only have the one trunk for your code, or are there branches as well?[/quote]

One trunk with branches for certain customers/programs (or a shared release version). Also, time consuming changes/reworks sometimes get a branch.

[quote="Beth"]Do you ever have bug fixes that have to go in after code is released?[/quote]

Yep, those generally go into the trunk and then merge into customer/release branches.

[robe070]

I have posted complaints about ShelveSets before and would like my vote added to all the features that Virgil has requested. ShleveSets could be really useful as opposed to barely usable!

[Beth]

So, they must set "most of the feature" aside, code/test the smaller change(s), checkin those changes, then resume work on the "complete feature". So the "merge" isn't at the point of check in, it's at the point of unshelving the in progress "complete feature". So... unshelve will always fail in the typically case.

Another way I think you could deal with this case is to unshelve and allow it to overwrite what's on disk. In some cases, due to the baseline being old, that will force it automatically into a Needs Merge state. At that point, the Show Merge can be performed to merge the last changes and the shelved item together.

If there aren't conflicts, you don't even have to show merge. You can perform a Get Latest with an automatic merge and the shelved version and the repository version are merged.

The one thing my latest description depends on is that the file isn't currently edited with changes that weren't checked in. Since you mentioned the changes from the critical piece being checked in, this could work.

[Beth]

robe070: Sorry, I don't have anything to add to the discussion you are already having with one of our developers. Hopefully one of the suggestions I make here can also help you work closer to the way you wish. If you have a specific use case that causes you grief, just let me know what it is and I can tell you how I would deal with it with our current product. The suggestions I'm making won't replace the feature requests, but they might make it possible for the users to continue working sooner rather than later.

[Beth]

We are also in the process of converting our main product from linking against .NET 2/3 and .NET 4 and trying to use a shelveset to hold onto the changes while we get them all lined up.

I could see this kind of change being huge. For something like this, you might be happier with a branch, and then merging the changes from branch back to trunk when they are completed. I'll check with our developers as to what they did when doing a similar thing.