Error: "An exception was encountered during the transac

levank · Post by **levank** » Sun Jul 30, 2006 10:24 am

Sorry if this has already been mentioned, but i could not find a description of similar problem

We have attempted to import VSS database which was just under 2Gb. The version of VSS was 6.0c with the hotfix as advisied on this website. Vault version was 3.5. We have run the VSS database fox utility that comes with it before importing. And VSS was not used while import was taking place. VSS database and import tool were on the same machine. Vault server was on different machine

This is how it went:

1. The import has started normally and reached about half way through the process.
2. while the import was going on, the log file was registering lots of "**Failed** to Get version..." errors, was this because of problems with the VSS database?
3. Suddenly import has slowed down considerably, with "Reestablishing connection" message on the screen. At some point import stopped completely
4. At the same time Vault server was displaying Windows popup about no virtual memory left. All of the available memory was taken by SQL server. Vault clients could not connect to the Vault server either
5. Log file had lots of "An error occurred while trying to end a transaction." errors
6. After long period of inactivity, it has been decided to restart Vault server (not the machine that was running import tool). This resuted in import process being terminated. The log file had many of the errors shown at the end of this message
7. Vault server completed restart, but Vault clients still cannot connect to the server. Error displayed is: "The Vault Server could not establish a connection to the database. Please check the server's log for detailed information. (FailDBConn)"

Could anybody help? What is causing those errors, memory usage, slowness and errors? Thank you

################### The error ###########
An exception was encountered during the transaction. Exception: 1 : Failure at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
at VaultClientNetLib.ClientService.VaultService.EndTx(String strTxID, Int32 nTxAction, VaultDateTime& dtTxBegin, Int64& nNewRevision, VaultResponseItem[]& responses)
at VaultClientNetLib.VaultConnection.EndTx(String strTxID, Int64& nNewRevision, VaultResponseItem[]& responses, Int32 nAction, VaultDateTime& txBeginDate)
at VaultClientOperationsLib.ClientInstance.Commit(ChangeSetItemColl givenItems, String strChangeSetComment, Boolean keepCheckedOut, Boolean removeLocalCopy, CommitType committype, VaultDateTime dateImport, Int32 nUserIDImport, Int64& nRevID, Int32[]& retBegEndTx)

Beth · Post by **Beth** » Mon Jul 31, 2006 9:24 am

What version of the VSS automation component are you running? Can you go ahead and send me your import log and the server log file?

Have you gone through the KB article we have on importing? This may help: http://support.sourcegear.com/viewtopic.php?t=7.

Also, do you use IDE integration in VSS? Here's a KB article on how to deal with that: http://support.sourcegear.com/viewtopic.php?t=3953

What are the specs on the machines you are using to accomplish this (RAM, hard disk speed, etc.)?

levank · Post by **levank** » Tue Aug 01, 2006 5:15 am

Hi Beth and thank you for reply

1. We have installed 6.0c on Vault server, but thinking about it now, it is quite possible that we were running import tool on the machine with 6.0d. Could it be the cause?

2. Yes, we followed the instructions in article

3. Yes, we do use IDE integration, but we use it with SOS rather then with VSS. Would that make any difference?

4. We used two servers. One was running Vault and the other had VSS database, SOS and Vault import tool. Specifications are as follows:

VSS database, SOS and Vault import tool:
Processor: 864 MHz
Memory: 256 Mb
Hard Drives:1 of 8.4 Gb
Operating System: W2003 Web Server

Vault server:
Processor: 2 of 1400 MHz Intel Pentium III
Memory: 1.5 Gb : (2 off 256 + 2 off 512)
Hard Drives: 2 off 36Gb Mirrored
Operating System: Windows Server 2003 SE

We will make sure we downgrade to 6.0c and try again this weekend. Is there anything else we should do?

Thanks
Levan

Beth · Post by **Beth** » Tue Aug 01, 2006 8:29 am

That looks like a good place to start. This forum post I believe had similar issues: http://support.sourcegear.com/viewtopic.php?t=6281&.

The RAM on the VSS database seems awfully low and can cause the export to be slow and to time out. I believe the Import/Export tool can be run from the other server if it's really necessary, but it's not recommended.

Thanks

levank · Post by **levank** » Fri Sep 08, 2006 4:15 am

Hi

It is about month and a half since we first tried to import data into Vault from VSS. We have been doing it every weekend since.

I think we have experienced all problems imaginable and i must say we managed to resolve most of them.

But today we got new lot. Latest import failed with large number of errors in the log. I will list one of each type below:

1.
Item $/Production caused the transaction to fail: An item with the same name or object id already exists in the selected folder.
Transaction failed

2.
Item $/Production/OLR/ClosedShows/ClosedShows_2006/kbb06ab/VisitorDetails.asp caused the transaction to fail: Invalid object version specified.
Transaction failed

3.
An error occurred while trying to end a transaction.
Transaction failed
(we have great deal of these)

4.
The label $/Production/OLR/ClosedShows/ClosedShows_2006/ats06 (BeforeAmendedtoShortly) was renamed to BeforeAmendedtoShortly vssimport 2.

5.
Label Verification: The label Before"shortly" was not imported because of the VSS error "Invalid OLEVERB structure"

6.
The label $/Production/OLR/ClosedShows/ClosedShows_2006/kbb06ab (BeforeCutOff) could not be imported correctly. The server returned: A database error has occured (FailDBInsert)

7.
Label Import: The label $/Production/OLR/ClosedShows/ClosedShows_2006/kbb06ac (BeforeCutOff) could not be imported because an exception occured.

Has the data been imported? Has it failed? I can see some of the data in Vault, but is it all of it?

Will it be safe to re-run the import on the same data, to try to import missing files?

Please help
Levan

Beth · Post by **Beth** » Fri Sep 08, 2006 12:43 pm

Have you changed what machine you are using for the import, or changed the amount of RAM? There are a number of things that could get in the way of the import, such as memory, network connectivity, database corruptions, etc.

Could you send me your import log found in: C:\Program Files\SourceGear\SourceSafe Import Tool ? You can either use the private feature here or send it in an email to beth at sourcegear.com and just reference this post.

levank · Post by **levank** » Sat Sep 09, 2006 3:28 am

Beth

Thank you for your response

Yes, we have changed the machine. The import is not running on P3 server with 1Ghz processor and 2Gb memory. It is connecting to Vault server via 1Gb network connection

Server has Windows 2000 and VSS version 6.0c installed

Database corruptions (I am assuming you are talking about VSS database corruption) cannot be an issue, because i have archived database from VSS and restored it into different location. Also VSS Analyse utility did not show any problems

I am attaching the log file as requested

Thank you
Levan

levank · Post by **levank** » Sun Sep 10, 2006 1:44 pm

Over this weekend i have attempted to import another chunk of our database (about 1.6Gb in size). As before i archived the data, restored on specially prepared machine. run VSS Analyse utility and then started import.

This time round we are not getting any significant errors in the log file. However the import process slowed down considerably, with each (small) file taking 5-10 minutes to upload. The processor on the Vault server is constantly showing 90%-100% load. The process that is taking up most of the processor is "w3wp.exe"

Restaring IIS on the Vault server helps to speed up the process, but this only lasts 5 minutes and everything slows down again

At current rate it will take several years to complete import - I don't think we can wait that long

Information on our Vault server: Pentium III, 1.4GHz, 1.5Gb RAM, 34Gb HDD, 1Gb Network card

I am coming to the conclusion that successful import of VSS database into Vault is an impossible dream. It is destined to fail if your database is large and contains many files with large history and lots of shares. SourceGear needs to come up with some sort of way of importing such databases in chunks, without links between projects being lost and with the ability to scrap portion of the import that went badly and keep everything else. And the fact that you cannot move files between repositories makes it even worse, because at least you could try to import a chunk into separate repository and then move files to main repository if import was successful. Now i have manage to import three chunks, but chunk number four messed it all up for me, so i have to start all over again

Thanks

Beth · Post by **Beth** » Mon Sep 11, 2006 9:51 am

If you look at this posting, there are additional suggestions farther down:http://support.sourcegear.com/viewtopic.php?t=8

One I think you might try is:

Something else that is worth doing if you have a large database, which I found speeds up the process substantially is to archive off sections of the source VSS db, then restore them to a fresh VSS DB on the import machine, and import that fresh DB.

Also, it looks like from the previous log that labels are coming at the same time? Labels take a long time to import since the label needs to be "reconstructed." The labels might have to be imported separately.

anobleperson · Post by **anobleperson** » Mon Sep 11, 2006 6:55 pm

I might be able to save you some time levank. We spent 8 months trying unsuccessfully to import our complicated VSS database. In the end we decided it was never going to happen and settled for just importing the last six months of our history without labels (took a week for 8Gb). We keep our VSS database (readonly) around for any queries over 6 months old.

Even though we analyzed the VSS database, exported the folders we wanted and imported into a fresh database, it still could not be reliably imported. VSS could mostly fetch the history from 2 years ago, but it wasn't possible to use the APIs to get it out (we tried separately to Vault).

cheers,
ash.

levank · Post by **levank** » Thu Sep 14, 2006 4:29 am

anobleperson

Thank you for your response. Would be nice if this sort of information is given to people within the document describing best practices for the import.

I am currently running one more session of the import and if that fails, i will follow your advise.

I just checked the progress on my import - it tells me that estimated time it will take is 4007 days. This is ELEVEN YEARS

Beth
I have already split my database into smaller chunks. I am currently importing the last chunk - 1.5Gb in size. I cannot split it any further, as all the files in it are extensively linked with each other.

Can you explain - how can i import labels separately?

Meanwhile i had to schedule IISRESET to be run on the machine once every hour. This keeps import process going, otherwise it comes to a halt

Thank you

Beth · Post by **Beth** » Thu Sep 14, 2006 8:27 am

Thank you for your response. Would be nice if this sort of information is given to people within the document describing best practices for the import.

(please see the first response to this thread)
http://support.sourcegear.com/viewtopic.php?t=7 and http://support.sourcegear.com/viewtopic.php?t=8
And here's one if you were using IDE integration with http://support.sourcegear.com/viewtopic.php?t=3953

From the first link it states
"Make sure the IIS doesn't get confused and kill ASP.NET during long operations. In your machine.config file, look in the processModel section and find the setting called responseDeadlockInterval. The default value is three minutes. We recommend changing it to at least an hour."

I'll have to get back to you on the labels part.

levank · Post by **levank** » Thu Sep 14, 2006 8:38 am

Beth

I have seen those threads and followed the recommendations (except one with IDE integration as we don't use it)

responseDeadlockInterval has been changed to a value in excess of one hour

Yet the import is dead slow. Now it is estimating to finish in 3523 days, which is only about 10 years, so we are moving ahead pretty quickly

We have almost never used VSS client - only SOS. May be this makes difference? I do not know.

levank · Post by **levank** » Fri Sep 15, 2006 4:03 am

Good news

We have been testing shadow folders earlier and accidently left one enabled. Once we removed it, import process suddenly became lot faster. Now it uploads file a second rather then file in about a minute.

While this is a good news and we might actually complete the import in few days time, it is very concerning to see what kind of effect shadow folders have on the system. We will be forced to use shadow folders to compensate for the lack of "Web deploy" functionality that we use so extensively in SOS

Source Gear - can we have "Web deploy" please?

Beth · Post by **Beth** » Fri Sep 15, 2006 10:40 am

We currently have a feature request that I will add your vote to. You will probably want to read up on Shadow Folders in the meantime.

SourceGear Support

Error: "An exception was encountered during the transac

Error: "An exception was encountered during the transac

Month and a half - and we are still trying to import

VSS import looks like an impossible task