DiffMerge on 20MB file hangs system

[dewest225]

We have a 20MB text file in our Vault repository. It has two revisions. I showed history for the file, selected the two revisions, then right-clicked and selected Diff. After waiting 16 minutes, and noting the memory usage hovering around 1.6GB (I have 2 GB on my Vista box), and the CPU usage for sgdm.exe staying around 50%, I finally did "End Task" in Task Manager to kill DiffMerge.

I tried this previously with detail level set to "Lines and Character". I then changed it to "Lines Only", and tried it again. The performance was the same both times, and it didn't finish in ~15 minutes either time.

For comparison, I tried using UltraEdit to compare the files, and it took about a minute. We tried VSS, and it took about 10 seconds. When I compared identical 20MB files in UltraEdit, it took about 2 seconds to tell me the files were identical.

Comparing identical files with DiffMerge took about 2 minutes, and still used about 1.5GB of memory. Plus, it took nearly 5 minutes to shut down!

I love DiffMerge for normal size files, but its poor performance and huge memory usage on large files is killing us! Any ideas for improving it? Are there plans to improve its handling of large files?

This horrible performance is a real problem for us. Any ideas to improve it? Do you have an improved version in the

[jeffhostetler]

You're right, it shouldn't take this much time/resources.

And yes, "Lines only" mode should be much faster than the
"Lines and Characters" mode.

I don't have anything currently in the works to address
this problem, but I'll add it to my list.

sorry I can't be of more help right now,
jeff

[CraigNicholson]

I've witnessed the same problem. I've got a SQL script that has a line of 74760 characters long, which causes the sgdm.exe process to quickly start allocating over 1.2GB of memory on my 2GB machine running Windows 7 x64 and SourceGear Vault 5.0.1.

This needs to be fixed.

[jeffhostetler]

Sorry, I must have missed this message when it was first posted.

Yes, DiffMerge is *extremely* slow on long lines and uses way too
much memory. I don't have a fix for this at this time.

75k characters is a bit long. I'm surprised that it's not causing you
problems in other places, like text editors. Perhaps you could break
it up into a bunch of shorter lines.

But, I should also point out that DiffMerge is going to be sluggish
when comparing multiple 20MB files -- even if they have short lines.
There's just a lot of stuff going on under the hood.

Sorry I don't have a better answer for you at this point in time.
jeff

[CraigNicholson]

Thanks for the response Jeff.

Visual Studio, Notepad++, SQL Server Management Studio, Notepad and even WordPad don't have a problem with the long line.

The reason the line is long is how SQL Server serializes SQLCLR assemblies by default. Its not my idea to have such a long line but its a 70KB hex encoded binary block.

I just tested it on 5.0.3 and it seems to be fixed now. It wasn't fixed in 5.0.1.

[jclausius]

That is very odd, as Vault's DiffMerge didn't really change.

Craig, were you by chance on Windows 7? I wonder if the %VAULT_EXE_DIR% fix might have addressed the problem. In any case, you are working now.

[EDIT]
Ahh!! Craig, I just re-read your post. Seems like that might have been the culprit afterall.

[CraigNicholson]

The problem existed on Vault 5.0.1 running on Windows 7 x64. Upgrading from 5.0.1 to 5.0.3 now seems to have resolved the issue.

Remember my file was 75KB big, not 20MB like the initial reporter.