Shadow folder doesn't reflect changes to repository files

[craigr]

Just did a VSS handoff -- all went well. Set up a shadow folder for one of the repository folders. Didn't encounter any problems. The folder already exists and has files in it from when it was a shadow folder under VSS. Set up the new shadow folder as "optimized for speed" so that it wouldn't try to sync change in the shadow folder back to the repository.

Checked out, edited, and checked in a file. No change to the shadow folder.

Created a new file and checked it in. No change to the shadow folder. I don't see anything in the log that indicates an error, and I do see it downloading the new file. It just doesn't show up in the shadow folder.

If there are potential permission issues, I need to know what user needs access to the shadow folder. AFAIK permissions are fine. They haven't changed from when VSS was maintaining this folder.

Ideas?

[craigr]

Additional info: I picked another project and gave it a shadow folder path to a "temp" folder that didn't exist. I chose the "optimize for speed" option and it created AND populated the folder from the repository. I was able to see new and edited files show up in that folder. So the basic functionality is working; it just isn't working when I give it an existing folder with existing content.

[craigr]

Message deleted

[Beth]

Shadow folders almost always need an empty folder to start with. When it populates the folder it sets the statuses on the files and can continue forward from there.

I can put in a feature request to change this functionality in the future.

Do you have files in your shadow folder that are going to be different than what is in Vault?

Do you have files in your shadow folders that aren't going to be in Vault?

[craigr]

I thought about that, but couldn't figure out why it wouldn't copy a new file into the shadow folder. I'll try deleting the files in the shadow folders.

The main problem with that is I have some folders with hundreds or thousands of files and it will take a lot of time to copy files into there that technically were already there. But as you've said before, "you only have to do it once."

The main use we have for shadow folders is in maintaining our websites. We used VSS to manage the site. There are a few cases where there are some folders that aren't in VSS and a few cases where we started using VSS after there were already several hundred files there so only the new ones are checked in. I'll just have to do this carefully (assuming starting with an empty folder fixes it to begin with).

So yes, it would be nice if shadow folders operated a little more blindly. Just overwrite whatever is there with whatever I check in. That's how VSS worked.

[Beth]

I've entered in a feature request to sync with files that are already present. Thank you for your feedback.

F: 15919

[craigr]

Just to be clear: Once I get shadow folders working, I can add files and folders that do NOT exist in the repository, right? If my shadow folder is /xyz and Vault has subfolders /xyz/1 and /xyz/2, I can go directly to the folder on the file system and create folders /xyz/foo and /xyz/bar and put files in those without messing up shadowing in Vault, right?

[Beth]

I can add files and folders that do NOT exist in the repository, right?

Yes. You will probably want to use the Speed Optimized setting.

[craigr]

Here's a conundrum:

I'm using shadow folders to manage three websites. The home directories for the websites contain a mix of files that exist in the repository and those that don't. For example, there is data uploaded by customers, logs, and catalog pages that are created by our shopping cart. None of these can be checked in to source control.

I've moved the two simple sites and am working on the main site. The problem is that Vault itself is installed on this site. In order to get shadow folders to work, after making a copy of the files that aren't in Vault I have to delete the whole folder (Vault shadow folders don't work unless Vault is allowed to create the shadow folder due to a bug in the way Vault figures out if it needs to update the files in the folder). However once I delete the folder so that Vault can re-create it, the Vault Admin site itself is no longer there, so I can't set up the shadow folders. If I leave the Vault Admin files in the folder so that I can create the shadow folders, then Vault won't populate the shadow folder because it will see the folder already exists. If I remove the admin files, I can't set up the shadow folders.

So: What are my options to continue to work around this big flaw in the way shadow folders are implemented in Vault? Can I simply copy the SgDav, VaultPro, and VaultService folders to another site on the same server, or do I need to run the installation program again? If I re-install, can I connect to the same repository database and legacy VSS data, or do I need to do another 16-hour handoff (resulting in the loss of all my changes since last week)?

Would it be quicker to wait for you guys to remove the one line of code that says "If the folder already exists, panic and don't do anything instead of simply copying the modified file to the shadow folder."?

Craig

[Beth]

I think you would be much happier with Vault and the shadow folders in different websites. A reinstall of Vault doesn't take very long. If you don't recall what settings you chose in the past during the install, if you send me the Vault_install.log from %temp% on your server, I can send you back the options you chose. To send that log, send an email to support at sourcegear.com (attn: Beth) with a link to this forum thread and the log.

I'm running a few tests to check on the VSS connectivity when a reinstall happens. I think it should be fine, but better to check.

Can I simply copy the SgDav, VaultPro, and VaultService folders to another site on the same server

That tends to not work for most users.

[Beth]

I ran a quick test where I uninstalled from one website and installed Vault on the same server but to a different website and a client was still able to view VSS history.

[craigr]

This thread from four years ago mentions an issue I've recently asked again about, since we're now moving Vault to a new server. The issue is as you can see above -- Vault required me to delete the folder and its existing content before it would work as a shadow folder. Beth put in a feature request for this bug to be fixed. Is this fixed now? It will affect how I go about migrating data from the old server.

[jclausius]

I see the request F:15919 is still open, but am a bit confused on the issue. I thought that as long as the Vault Shadow Folder had write access and possibly read-only attribute was unset, that shadow folder would just overwrite any necessary files when the association is made.

Given that Beth is out this week, please allow me a bit of time to investigate this a bit further.

[craigr]

No in this case we already were using the shadow folder under VSS/SOS so the permissions were OK. I tried changing permissions on the files (they may have already been read/write, I don't recall) and that didn't help.

PLUS, even a new, non-existing file would not write to the shadow folder. The only way to get it to work was to delete the folder and allow Vault to create it. Since some folders contained files that were not in Vault, it meant I had to make a copy of each folder, delete it, set the shadow folder in Vault, allow Vault to shadow the files into the folder, then copy the non-Vault files back into the folder and delete my copy. I did this for several dozen folders. It was fun. So much fun that I don't want to do it again.

[jclausius]

I just tried a test...

I have a Vault 8 repository with over 2100 files in a folder ($/Dir/FolderX)

1) Did a GET to a non working folder so I'd have just the files. - no _sgvault, etc.
2) Copied the files to C:\Temp\ShadowTest\FolderX... At this point, it is important to know the files found in Vault's repository MATCH the files on disk.
3) Removed the read-only flag on ALL files/folders of C:\Temp\ShadowTest\FolderX\ .
4) Set permissions on C:\Temp\ShadowText\FolderX to FULL Control for EVERYONE.
5) Went in to the Vault admin tool, I created a Shadow folder in the repository for $/Dir/FolderX to C:\Temp\ShadowTest\FolderX. Used the defaults on options for 'Don't set read-only flag...' and 'Optimize for Speed'.
6) Finally, I made changes to some of the sub-files found in $/Dir/FolderX/Some/Sub/Folder/Changes...
7) Committed the changes.

Afterwards, I looked at the Shadow Folder log file, and the log had entries it updated the files with the changes. I also verified the files on disk also had been updated with those changes.

Does that test match what you would be doing with Shadow Folders and an existing web-site?