"Add Solution to Fortress" Hangs on large solution

If you are having a problem using Fortress, post a message here.

Moderator: SourceGear

Aaron Young
Posts: 33
Joined: Fri Aug 06, 2004 7:58 am
Location: Red Wing, MN
Contact:

"Add Solution to Fortress" Hangs on large solution

Post by Aaron Young » Wed Jun 13, 2007 8:17 am

Using Fortress v1.0.0 (15759) - Demo License:

When I try to use the "Add Solution to Fortress" context menu option within the VS2005 IDE to add a solution containing 146 projects,
the process seems to just hang at the "Adding solution <name> to Fortress..." dialog, (it doesn't fail to respond, it just seems to cycle endlessly.)

I left the process in this state overnight (approx. 16hrs) and it never completed.

I have successfully added a smaller solution (12 projects) in the same fashion without an issue.

I turned on logging at both the server and IDE level, neither seems to shed any light on the problem (the logs are attached.)

Server Log
IDE Log

Note: The IDE log may be missing a couple of "Setting working folder ..." entries, I noticed there were some in the Event Log which must have happened as I was monitoring the log file during the process.
Aaron Young
Red Wing Software

Beth
Posts: 8550
Joined: Wed Jun 21, 2006 8:24 pm
Location: SourceGear
Contact:

Post by Beth » Wed Jun 13, 2007 10:29 am

Do you have a previous Vault Server Log that covers the time that you tried adding the larger project? Vault server logging is always running on some level, but it's either set to quiet or debug. Timeouts should still show up in the quiet mode. Archived Vault logs should just have a date as part of the name and be in the same location as the current one.

I think first we need to check the previous log for a timeout. If that's not the case, but maybe one project having a problem, then that problem project needs to be identified.

Would you be willing to add this in chunks? By that I mean, open the solution, but remove all but 20 of the projects (don't delete them from disk, just remove from that solution). Then add that to source control. Then add the projects back into the solution. Each time it should ask if you want it added to source control since the solution is under source control.

First, check for a timeout though.

Aaron Young
Posts: 33
Joined: Fri Aug 06, 2004 7:58 am
Location: Red Wing, MN
Contact:

Post by Aaron Young » Wed Jun 13, 2007 12:09 pm

The log I attached was from during the attempt to add the large solution to Fortress. I had cleared the log and then turned on Debug logging prior to the attempt. So there's no timeout that I can see.
Would you be willing to add this in chunks? By that I mean, open the solution, but remove all but 20 of the projects (don't delete them from disk, just remove from that solution). Then add that to source control. Then add the projects back into the solution. Each time it should ask if you want it added to source control since the solution is under source control.
In general or to see if it works? I will try it, my guess is it will work, so I wouldn't be opposed to doing it this way. In our everyday use we likely wouldn't be adding solutions with this many projects as we'd be starting new solutions, however for testing purposes I was trying to add existing solutions (currently managed with Vault) to see if I'd run into any issues.
Aaron Young
Red Wing Software

Aaron Young
Posts: 33
Joined: Fri Aug 06, 2004 7:58 am
Location: Red Wing, MN
Contact:

Post by Aaron Young » Wed Jun 13, 2007 1:15 pm

I tried removing all but 20 projects from the solution and then adding the solution to Fortress... I got the same result.

Then I removed all but one project and was able to add the solution to Fortress successfully. I could then add the other 19 without issue (one at a time.) So it seems to be an issue with the number of projects added at once.

When I checked the server log and compared the failed 1st attempt to the successful (single project) attempt, the only difference was that the 2nd attempt had an entry of "BeginTx beginning transaction" followed by all the project files/folders being added. Where as the failed attempt matched exactly, except it stopped at the entry "GetRepositoryOptions returned: Success", no timeouts or failures.

- Aaron.
Aaron Young
Red Wing Software

vanzelone
Posts: 1
Joined: Thu Jul 12, 2007 2:38 pm

Same Problem Here

Post by vanzelone » Thu Jul 12, 2007 2:42 pm

We are looking at Fortress for an internal project here at work. I have the server setup on a remote box and I'm using Vistual Studio 2005 with SP1 installed.

When I try to add a solution with more than one project it just hangs. There is almost NO network activity at all. When I remove all but one project my network lights up like a XMAS tree and it starts uploading fine.

Sorry guys but this is kind of a deal breaker for us. Almost all of our solutions here are multi-layered and we break them out into separate projects...

ian_sg
Posts: 787
Joined: Wed May 04, 2005 10:55 am
Location: SourceGear
Contact:

Post by ian_sg » Thu Jul 12, 2007 3:45 pm

We'll investigate this.

Have you tried the same solution using the 2003-compatible client (within Visual Studio 2005)?
Ian Olsen
SourceGear

ian_sg
Posts: 787
Joined: Wed May 04, 2005 10:55 am
Location: SourceGear
Contact:

Post by ian_sg » Fri Jul 13, 2007 1:17 pm

The current Fortress solution is 59 projects, and we all use the new 2005 client every day, so unfortunately it's not as simple as "a solution with several projects doesn't work." Nonetheless, there's obviously something strange afoot for some of you.

I'd like to gather some more information about this to try to see what's going on here. If anyone experiencing this issue would be willing to "go under the microscope," so to speak, please email me: ian at sourcegear dot com.

Thanks,
Ian Olsen
SourceGear

rgrimard
Posts: 15
Joined: Wed Aug 29, 2007 3:52 pm

Post by rgrimard » Wed Aug 29, 2007 3:56 pm

I'm having the same problem. Brand new Vault Server. My VS2005 solution contains 33 projects. I "add solution to vault" and it hangs forever. CPU is pegged. devenv.exe using 362MB of memory. No network activity. The Vault Server has no cpu activity and nothing new in the database.
Was there any resolution to this issue? We have many solutions and we are evaluating Vault against a number of other SCM products. I'd rather not use the "add one at a time" work-around, as that would not look good.

ian_sg
Posts: 787
Joined: Wed May 04, 2005 10:55 am
Location: SourceGear
Contact:

Post by ian_sg » Wed Aug 29, 2007 4:04 pm

We have found and fixed a number of issues related to this, to be included in the next release (Fortress 1.0.5/Vault 4.0.5). A date for this release has not yet been set.

In the mean time, for folks experiencing this issue, the "2003-compatible" client works just fine in Visual Studio 2005.
Ian Olsen
SourceGear

gmagana
Posts: 145
Joined: Wed Feb 18, 2004 10:51 am
Location: Santa Ana, CA, USA

Post by gmagana » Wed Aug 29, 2007 8:00 pm

Just a note: These outstanding bugs affect some of us greatly... And we are not in a position to give our clients the standard SourceGear "we don't have a date for the next release yet" response. Right now I have gotten bit by this "Add solution to Fortress" bug with an important project. I unfortunately happen to be working in a project in a different country than the rest of my development team, and have to transfer a 30MB compressed file containing all the solution files. We have to transfer this back and forth because we don't want to miss a single file update. We have almost lost this customer directly because we cannot add this project to Fortress. I have not even mentioned all the time we have wasted babysitting this very important file transfer every time we have to do it (at least once a day). This should not be, we have a source control system that supposedly should be enabling us to do this in about 10 seconds.

I'm just mentioning so hopefully you guys at SourceGear do not get complacent about the extreme importance of Vault/Fortress to us, your customers. Hopefully you are not holding this update because bug fix releases are a bothersome thing for you to do. There are some of us checking these forums daily in the hope today was the day some of these bugs get fixed.

SourceGear has screwed up the 4.0 release badly. You have gotten much criticism for it, hopefully you have learned from it, as much of that criticism is valid. Some of us have much patience with you guys because we know you responded quickly in all the previous versions to trouble reports, but patience wears thin as time goes by, and specially when you acknowledge that you have fixes for problems, but for some mystical reason you refuse to release those fixes immediately. It's like there is a huge disconnect between you and your customers. It's almost like our priorities are not your priorities anymore. I wonder if you are trying too hard to become the next Microsoft, and, as Microsoft does, you will take as much time as you like in releasing bug fixes.

I think if whoever is in charge of the version control products at SourceGear had to waste almost an entire day waiting for a solution to add to source control only to have it end in failure, maybe the bug fix would be put out right away. Not only are we spending unbillable time on this, but we are taking up time we need to provide timely service to our own clients. Do you even appreciate this? Can you relate?

Anyway, my personal advice to SourceGear is that even with Fortress, there are free alternatives to your products... Source control is extremely criticial to your customers, so if I were you, I'd pay attention to my customer's priorities (I'll spell it out: Reliability), and adopt those priorities as my own. Your customers are paying you 20% of the full license cost per year in support. It would be a good idea to support them, no? Are we paying for your support so that you can tell us you agree there are problems, but the bug fixes will be ready whenever you guys feel like releasing them? I think something changed fundamentally at SourceGear as of the 4.0 release. You guys must have a new focus or corporate direction leading to something good for SourceGear, but something is taking you guys away from helping your clients.

If there was ever a cheerleader user for SourceGear it's me, but I myself am wondering if you've lost touch with us and it's time to give SVN and other free solutions a fresh new look. If I were you, I'd be worried about it. For the first time ever, SourceGear is becoming part of the problem after a long history of being part of the solution.

EDIT: Regarding this bug - It's kind of useless to suggest using the 2003 client since we are using the 2005 client for everything else.
gabriel magana-gonzalez

rgrimard
Posts: 15
Joined: Wed Aug 29, 2007 3:52 pm

Post by rgrimard » Thu Aug 30, 2007 6:37 am

ian_sg wrote:We have found and fixed a number of issues related to this, to be included in the next release (Fortress 1.0.5/Vault 4.0.5). A date for this release has not yet been set.

In the mean time, for folks experiencing this issue, the "2003-compatible" client works just fine in Visual Studio 2005.
Should I be able to use the 2003 client to get the solution in and then switch to the 2005 client?

Thanks for the response.
Ryan

rgrimard
Posts: 15
Joined: Wed Aug 29, 2007 3:52 pm

Post by rgrimard » Thu Aug 30, 2007 7:02 am

ian_sg wrote:We have found and fixed a number of issues related to this, to be included in the next release (Fortress 1.0.5/Vault 4.0.5). A date for this release has not yet been set.

In the mean time, for folks experiencing this issue, the "2003-compatible" client works just fine in Visual Studio 2005.
I tried the 2003 client and about 1 minute into it I get a dialog that says "Failed to check in one or more files. Some files remain checked out or not added". The end result is my repository has 2 folders in it which correspond to my two asp.net website projects. The other 31 project folders do not exist. No files were checked in at all. Visual Studio 2005 then shows all my files with a plus sign, indicating that my solution is bound to a source control product and the files need to be checked in. I select the solution and "Check in". I get the same dialog again and nothing is checked in.

I'm in a bind here. Again, the issue is just getting the solution into vault. After that, I'm sure it will work just fine. The vault 2003 client workaround for adding a solution doesn't work. Any other suggestions?

Thanks
Ryan

jeremy_sg
Posts: 1821
Joined: Thu Dec 18, 2003 11:39 am
Location: Sourcegear
Contact:

Post by jeremy_sg » Thu Aug 30, 2007 7:54 am

At the moment, the reason that we're not sure when 4.0.5 will be out is due to the fact that we're working on two really invasive features that people have requested and we want to make sure that everything is solid before promising a date. The two large features that we're working on are a fix for the double-get problem (where if files have been added to a project you must first do a get to fetch the project and then another get for fetch the new files) and a new UI to allow people to easily bind projects without binding the solution. Waiting for those two very large changes has caused a delay in getting out the smaller changes. Please understand that we're not holding out bug fixes because we're too lazy to do a release. We're being vague about the release date of 4.0.5 to make sure that it's right.

With that said, I've never been against giving out pre-release builds to specific customers to verify that we've fixed the bug that they're seeing. It anyone watching this thread would like to try a pre-release build of 4.0.5 to verify that it fixed the Add Solution hang that you're seeing, please email me using the button below this post.

Beth
Posts: 8550
Joined: Wed Jun 21, 2006 8:24 pm
Location: SourceGear
Contact:

Post by Beth » Thu Aug 30, 2007 7:55 am

rgrimard: Can you send me an email at support at sourcegear.com (attn: Beth) with a link to this forum thread? I'd like to walk through this with you.

jclausius
Posts: 3702
Joined: Tue Dec 16, 2003 1:17 pm
Location: SourceGear
Contact:

Post by jclausius » Thu Aug 30, 2007 9:48 am

gabriel:

Like you, we hate bugs! We strive to ensure any release (major or maintenance) not only fixes existing bugs, but also does not introduce new bugs. The problem is compounded with new projects, especially on a project like the VSIP client because new test procedures must also be developed. We're working hard to solve these problems and ensure the VSIP client's quality and reliability.

The planning and testing for this will take some amount of time. In fact, you perfectly summarized the delay in between these maintenance releases:
gmagana wrote:Source control is extremely criticial to your customers, so if I were you, I'd pay attention to my customer's priorities (I'll spell it out: Reliability)

In the interim, we need to figure out how to solve your problem NOW. Jeremy has offered a Pre-Release Vault 4.0.5 client in which the problem should be solved. But I'd also like to suggest another option. You can use the MSSCCI IDE client for this one project. With Visual Studio 2005, things are no longer an "all or nothing" when talking version control. You can have a mixture of distinct MSSCCI or VSIP bound solutions. You mention this will not work in your situation, but having one bound MSSCCI based solution and project would be better than how you describe your current situation.

If there's anything else we can do, just let us know.

Thanks.
Jeff Clausius
SourceGear

Post Reply