RAM Disc for large TMs? Thread poster: OTMed (X)
|
|---|
OTMed (X)
Poland
Local time: 00:44
English to Polish
+ ...
Hi All,
For a number of reasons I have been using Wordfast more than Trados for some time now. However one of wfs disadvantages is apparently slower TM reorganizing/processing/segmenting, particularly with medium to large sizes TMs and/or segments including mixed text and numeric structure.
For the last couple of days I have been updating a project with a not-so-large (but also not very tiny) TM of approx. 110 k units. Since there is a couple of thousands of segments in... See more Hi All,
For a number of reasons I have been using Wordfast more than Trados for some time now. However one of wfs disadvantages is apparently slower TM reorganizing/processing/segmenting, particularly with medium to large sizes TMs and/or segments including mixed text and numeric structure.
For the last couple of days I have been updating a project with a not-so-large (but also not very tiny) TM of approx. 110 k units. Since there is a couple of thousands of segments in the project and I need to wait 0,5-1 s (well that’s my rough approximation only of course) before each segment is opened, I am waiting for the system at least a couple of hours for this project only. This made me think again about a way to eliminate the TM processing lag.
What is particularly infuriating is that the process (segmenting, reorganizing) is clearly way faster on a 2.4 MHz 512 MB laptop than on a P IV 3.0 MHz HT 1 GB RAM machine. Of course, I have done all kinds of tweaks, including defragmenting the HD, optimizing the TM, and even assigning high priority level to MS Word process. Still, the desktop reads the TM file from the HD before opening the next segment, and my frustration is steadily growing…
Since the need is the mother of all invention I have come to the stage where I am thinking about adding 512-1024 MBs of RAM and mounting a RAM disc.
Theoretically this should eliminate the TM read lag problem, provided that a system could be used to mirror/copy the RAM to the HD every now and then so that in case of problems the volatile RAM content is backed up to the HD.
Do you think such strategy may work? Did any of you tried similar solution? Would you recommend a particular software for creating/managing the RAM disc?
Thanks for your input. ▲ Collapse
| | | |
Jerzy Czopik 
Germany
Local time: 00:44
Member (2003)
Polish to German
+ ...
| This should be faster... | Mar 1, 2006 |
although I never tried to create a RAM disk under Windows XP. I used to have one back in the early ninenties with DOS and than Windows 3.1.
But you might encounter problems with backing up the content of RAM disk when the file stored there is still in use. A backup copy of such file - if even created - will be almost surely useless. So to backup it you would have to close the applications using files in RAM disk. In this way you will almost certainly invest more time than you will be savin... See more although I never tried to create a RAM disk under Windows XP. I used to have one back in the early ninenties with DOS and than Windows 3.1.
But you might encounter problems with backing up the content of RAM disk when the file stored there is still in use. A backup copy of such file - if even created - will be almost surely useless. So to backup it you would have to close the applications using files in RAM disk. In this way you will almost certainly invest more time than you will be saving. But this is just my opinion.
As for the very fast notebook - this would mean, that all the components in the notebook are optimal selected. Your PC might be theretically faster, but when the components do not fit, than you will loose speed. For example a very fast processor used with a no-name motherboard or with a slow chipset and possibly slow RAM too will not bring the power you expect.
TM work is AFAIK uses a lot of HDD power - maybe this is the weak point of your PC? I noticed a significant increase of speed since I started to use two Western Digital Raport HDDs (74 GB each), which build together a stripped RAID-matrix (RAID0). For security purposes I run backup each day (scheduled) using the third HDD in my PC.
I´m afraid I didn´t tell you anything new...
Regards
Jerzy ▲ Collapse
| | | |
OTMed (X)
Poland
Local time: 00:44
English to Polish
+ ...
TOPIC STARTER | RAM disc copying should be possible without closing all apps | Mar 1, 2006 |
Hi Jerzy,
Jerzy Czopik wrote:
But you might encounter problems with backing up the content of RAM disk when the file stored there is still in use. A backup copy of such file - if even created - will be almost surely useless. So to backup it you would have to close the applications using files in RAM disk.
Well, you may *copy* any file to another location without closing it. Say, the last TM write is 12:00:00, and the copy operation will take the TM file and copy its 12:00:00 state to another location, i.e. from RAM to HD. You may do exactly the same thing with any .doc that is currently opened.
Jerzy Czopik wrote:
As for the very fast notebook - this would mean, that all the components in the notebook are optimal selected. Your PC might be theretically faster, but when the components do not fit, than you will loose speed. For example a very fast processor used with a no-name motherboard or with a slow chipset and possibly slow RAM too will not bring the power you expect.
Thats funny, I have exactly the opposite. The primary desktop machine has all kinds of top shelf asus, kingston, gainward etc components whereas the laptop is a 2years old generic googlegear.
Jerzy Czopik wrote:
TM work is AFAIK uses a lot of HDD power - maybe this is the weak point of your PC? I noticed a significant increase of speed since I started to use two Western Digital Raport HDDs (74 GB each),
I was also thinking about upgrading the HD array but, again, the HD is a 16 MB cache 7200/s 200 GB 150 MB/s serial ATA, and the laptop is 4 MB cache 4800(?)/min.
Raptors must be much faster but I do not think the HD is the source of the problem.
What is even more funny, the laptop is all installation with a bunch of tested sharewares and suchlike, and the desktop is a clean&fresh 1 month old XP installation...
Thanks for your input, Jerzy. I think I will give the Ram disc a go and will see what happens.
[Edited at 2006-03-01 09:11]
| | | |
Robert Zawadzki (X)
Local time: 00:44
English to Polish
+ ...
| In general, I do not think RAM disks make sense any more | Mar 1, 2006 |
With all caching and lots of RAM available today I think that RAM disks are not necessary. Are you sure disk activity when you open a TU is caused by accessing TM? I'd rather suspect swapping to load some code. Check with Windows Task Manager. Perhaps you have some processes using lots of memory. With 1 GB of RAM there should be no disk activity when you use a relatively small TM (at least when it's not a beginning of work - it should be cached). There are some programs known to use a lot of res... See more With all caching and lots of RAM available today I think that RAM disks are not necessary. Are you sure disk activity when you open a TU is caused by accessing TM? I'd rather suspect swapping to load some code. Check with Windows Task Manager. Perhaps you have some processes using lots of memory. With 1 GB of RAM there should be no disk activity when you use a relatively small TM (at least when it's not a beginning of work - it should be cached). There are some programs known to use a lot of resources. I can immeddiately think of two: Norton Antivirus and Microsoft's SQL Server (MSDE version cannot be set to use a specified amount of memory - it takes as much as it can). Having one of these on 1GB machine can make it appear to have less than on a 512 KB machine. Yes, a RAM disk can be a way to protect memory from such greedy beasts.
And there can be a virus using memory, but this is another issue. ▲ Collapse
| | |
|
|
|
OTMed (X)
Poland
Local time: 00:44
English to Polish
+ ...
TOPIC STARTER | What is running in the background | Mar 1, 2006 |
Robert Zawadzki wrote:
With all caching and lots of RAM available today I think that RAM disks are not necessary. Are you sure disk activity when you open a TU is caused by accessing TM?
Positive. Disc activity is clearly heard and seen before each segment is opened. I would normally think that wfast should be able to load the entire TM (its 80 MB) to RAM and process it from within the memory without having to read it from the HD with every segment.
Robert Zawadzki wrote:
I'd rather suspect swapping to load some code. Check with Windows Task Manager. Perhaps you have some processes using lots of memory. With 1 GB of RAM there should be no disk activity when you use a relatively small TM (at least when it's not a beginning of work - it should be cached). There are some programs known to use a lot of resources. I can immeddiately think of two: Norton Antivirus and Microsoft's SQL Server (MSDE version cannot be set to use a specified amount of memory - it takes as much as it can). Having one of these on 1GB machine can make it appear to have less than on a 512 KB machine. Yes, a RAM disk can be a way to protect memory from such greedy beasts.
You are very right, there is always a number of processes running in the background, from antivirus+firewall (I use BitDefender), google desktop search, firefox, 2 dictionaries and suchlike.
The thing is eliminating (killing) all non-MS threads, with RAM load around 20% does not seem to help at all.
[Edited at 2006-03-01 09:55]
| | | |
Robert Tucker (X)
United Kingdom
Local time: 23:44
German to English
+ ...
| Memory allocation | Mar 1, 2006 |
Again, are you sure the problem is with the size of the TM?
Using the Open Source CAT tool Transolution (written in Python) on a 2GHz Celeron, 256MB RAM machine with a source file of seven or eight thousands words I have to wait two or three seconds for it to open or close a segment or move between segments. With a source file of a few thousand segments this goes up to a few minutes. But the thing is that it is independent of whether the TM is connected or not.
T... See more Again, are you sure the problem is with the size of the TM?
Using the Open Source CAT tool Transolution (written in Python) on a 2GHz Celeron, 256MB RAM machine with a source file of seven or eight thousands words I have to wait two or three seconds for it to open or close a segment or move between segments. With a source file of a few thousand segments this goes up to a few minutes. But the thing is that it is independent of whether the TM is connected or not.
There is a lot on the web about Python not deleting memory it has used and does not require again and so, of course, starts using HDD when it could still be using RAM. I question, therefore, whether that may not be the case with WF.
That said, I wonder if CAT tool producers might like to consider writing software which will allow translators with more than one machine (i.e. a cluster) to have the chip on one machine just processing the TM.
[Edited at 2006-03-01 13:59] ▲ Collapse
| | | |
Gerard de Noord
France
Local time: 00:44
Member (2003)
English to Dutch
+ ...
| Some stats from my PC | Mar 1, 2006 |
I think there must be something wrong with your setup.
I have:
Total Physical Memory 512,00 MB
Available Physical Memory 189,00 MB
Total Virtual Memory 2,00 GB
Available Virtual Memory 1,96 GB
Page File Space 1,22 GB
Reorganising a 83.485 Kbyte TM containing 489.776 TUs takes 3 minutes and 46 seconds.
Matching takes a split second.
I really think Wordfast has no performance problems on curren... See more I think there must be something wrong with your setup.
I have:
Total Physical Memory 512,00 MB
Available Physical Memory 189,00 MB
Total Virtual Memory 2,00 GB
Available Virtual Memory 1,96 GB
Page File Space 1,22 GB
Reorganising a 83.485 Kbyte TM containing 489.776 TUs takes 3 minutes and 46 seconds.
Matching takes a split second.
I really think Wordfast has no performance problems on current PCs.
In the past, Yves Champollion disapproved of RAM disks but I forgot why.
Regards,
Gerard ▲ Collapse
| | | |
Robert Zawadzki (X)
Local time: 00:44
English to Polish
+ ...
| Going deep into technical details | Mar 1, 2006 |
I lately bought my son a monster game machine (3.6 GHz, HT processor, 2 GB RAM, expensive graphic card) and noticed that it does not perform too well. The problem was that there was only one IDE channel with both HDD and DVD-R drives connected to it. After I bought a marginally more expensive SATA drive (separate controller, so a DVD does not affect its performance), the machine started to fly.
Perhaps you have something like that. If you do not have a SATA controller, you should ha... See more I lately bought my son a monster game machine (3.6 GHz, HT processor, 2 GB RAM, expensive graphic card) and noticed that it does not perform too well. The problem was that there was only one IDE channel with both HDD and DVD-R drives connected to it. After I bought a marginally more expensive SATA drive (separate controller, so a DVD does not affect its performance), the machine started to fly.
Perhaps you have something like that. If you do not have a SATA controller, you should have at least 2 IDE channels. HDD should not use the same channel slower devices (CDs, DVDs) use.
But it does not explain apparent lack of caching.
[Edited at 2006-03-01 13:25] ▲ Collapse
| | |
|
|
|
Luca Tutino
Italy
Member (2002)
English to Italian
+ ...
| Term recognition? | Mar 1, 2006 |
I am not familiar with WF. However I noticed that with Trados large TM get somewhat slower only when I have Term recognition on.
Of course this won't explain why your new desktop does not perform well. Did you try a performance test on it? I would connect to some pc magazine and look for a software to look for week points in your system.
Hope it helps
| | | |
OTMed (X)
Poland
Local time: 00:44
English to Polish
+ ...
TOPIC STARTER | Problem solved (kind of) | Mar 2, 2006 |
Thanks again for all your valuable input.
Having reduced the size of TM to 10 000 units I tried all kinds of tricks I could think of and finally decied to give the Ram disc a go.
In the meantime I have crossposted this msg to the wf yahoo group. Yves, the wf developer, suggested to go for the wf server, but didnt have enough time to test-drive it.
After a couple of hours worth of testing I was able to make the following observations:
1. A Ram d... See more Thanks again for all your valuable input.
Having reduced the size of TM to 10 000 units I tried all kinds of tricks I could think of and finally decied to give the Ram disc a go.
In the meantime I have crossposted this msg to the wf yahoo group. Yves, the wf developer, suggested to go for the wf server, but didnt have enough time to test-drive it.
After a couple of hours worth of testing I was able to make the following observations:
1. A Ram disc is very easy to mount/unmount and backup. All it takes is a freeware/shareware soft any decent backup soft set up to copy/mirror/backup your ramdisc to a hd or network storage of your choice every X of minutes.
2. A small Ram disc of, say, 100 MB should in most cases accomodate both large TM AND large rtf/doc.
3. The speed of wf TM processing (opening segments, scanning, reorganizing TM etc.) clearly increased which made my work considerably faster.
Therefore I think I will keep the Ram disc in my toolbox for large MT/large file projects.
The grim mistery of fast wordfast on a slower, older laptop remains unsolved... ▲ Collapse
| | | |
Jerzy Czopik
Germany
Local time: 00:44
Member (2003)
Polish to German
+ ...
| What about swap file? | Mar 2, 2006 |
How is your Windows set up?
Do you use fixed swap file, and if - how big it is?
My experience is, that using a big swap file, located on a completly isolated partition, twice as big as the swap file itself, helps to keep Windows running smooth. In various computer magazines and optimizing tools I found the information, that a swap file of 2,5 times the RAM memory is optimal. Now I´m using such swap file, located on a different (usually third) partition on the same HDD as the ... See more How is your Windows set up?
Do you use fixed swap file, and if - how big it is?
My experience is, that using a big swap file, located on a completly isolated partition, twice as big as the swap file itself, helps to keep Windows running smooth. In various computer magazines and optimizing tools I found the information, that a swap file of 2,5 times the RAM memory is optimal. Now I´m using such swap file, located on a different (usually third) partition on the same HDD as the system. One could increase the speed of the system, if this swap file would be located on a separate very fast HDD - but this is unpracticable.
Maybe also you have different settings for working directories in Word?
I agree, that a notebook with such parameters must be slower that a desktop with all bells and whistles, but maybe a whistle in your desktop is blocked? Did you check the transfer rate of your HDD? Does the HDD reach the transfer rates it should?
I used a Samsung Spinpoint SP1614C SATA I 160 GB as my main HDD. After havng tested its transfer rates with Sisoft Sandra I was really disapponited, as the real speed of this HDD was behind a fast EIDE HDD... Now I use it for backup purposes and have a RAID0 matrix as my main HDD. This matrix consists of two fast WD Raptor HDDs. But even now I do not get the full performance I expected. The reason is my controller - it is a very easy one, which I bout for some 20 Euros. It uses the SiL 3114 and the mainboard has SiL 3112 chip. Both seem not to be from the fastest kind.
These are simply some more thoughts - maybe one of them might bring you near to your solution.
Regards
Jerzy ▲ Collapse
| | | |
Login to reply/comment 
