If you have 2HDS, low RAM, Verifing Time Solved!

[Regent]

Re: If you have 2HDS, low RAM, Verifing Time Solved!

According to microsoft if you turn off the page file, instead of loading everything into physical memory it just leaves it on your hard drive. This means the indexing system is different and access times are even longer.

That's ridiculous. If Windows doesn't load anything into RAM, how is it that BF2 uses 1.3 Gb of RAM during play? I'm guessing that Windows decided to load files into RAM.......

[imported_The_Revrend]

Re: If you have 2HDS, low RAM, Verifing Time Solved!

Rev: Yeah, that's why I was saying it would make windows faster across the board, not just BF2. I was really suprised though, that running a pagefile on a second physical drive on the same IDE channel actually makes windows run slower than a single pagefile. If you can find the answer to that one please post.

I covered that :) I realize it's a big post. . . but let me explain again.

When you have two drives on the same IDE Chain. Windows Reads from one drive (OS) stops all activity on the chain and then writes to the second drive (scratch/pagefile disc) and the chain and has to do this for every chunk of data it copies. This is what causes the system to slow down, worse than having the scratch disk be the OS Drive on a system utilizing only one IDE Chain. IDE can't coordinate traffic on the same chain efficiently.

With the drives on seperate IDE chains, the controller is able to to the read write read write process continuously. . . the ways SCSI can on the same chain.

Make Sense?

Rev

[imported_The_Revrend]

Re: If you have 2HDS, low RAM, Verifing Time Solved!

That's ridiculous. If Windows doesn't load anything into RAM, how is it that BF2 uses 1.3 Gb of RAM during play? I'm guessing that Windows decided to load files into RAM.......

I don't think you're understanding what he's trying to convey, and what he is trying to convey it does match my experiences. He's not saying it doesnt load it into RAM at all, he's saying it leaves it on the drive instead of loading it into the pagefile which is Virtual Memory (Virtual RAM) When the info needs to be loaded off of the HD and because XP is optimized to work with Virtual Memory and has to use a diffrent disc access scheme to load it into memory, it slows the process down. At least, that's what I get fromt he posted articles. . .

Rev

[Painl3ss.NL]

Re: If you have 2HDS, low RAM, Verifing Time Solved!

The OP is correct. I don't know about the actual performance benefit, but it depends on:
1. how much ram you have in the first place.
2. the seek times of the harddisks.
3. How fragmented your disks are.

As you know, defragmenting your harddisk can have quite a speed improvement. This is because repositioning the harddisk read/write heads is relatively expensive and it is best to access consecutive sectors on the disk.

With this in mind look, look at what happens:
BF2.exe writes to virtual memory (RAM), if there is no more free memory left, the OS will free up virtual memory by removing pages from virtual memory and copy them into the page file (pages it thinks it won't need directly).

If the pagefile is on the same disk as bf2.exe it will read from some part of the harddisk, REPOSITION THE HD-HEAD, and write to another part of the harddisk (the pagefile) and it will do this time after time.
If you put the pagefile on a different disk it doesn't have to reposition the hd-head so often.

If you don't see any difference with a 2 disk setup, it means:
1. You have enough RAM (so the OS does not have to free anything and the pagefile remains unused)
2. Your application and/or swap file are so fragmented that it has to reposition the head a lot of time anyway.

[Regent]

Re: If you have 2HDS, low RAM, Verifing Time Solved!

The OP is correct. I don't know about the actual performance benefit, but it depends on:
1. how much ram you have in the first place.
2. the seek times of the harddisks.
3. How fragmented your disks are.

For the purpose of this thread, the original poster is incorrect. The title of the thread is:
"If you have 2HDS, low RAM, Verifing Time Solved!"

Here is the performance benefit claimed for BF2:

If you have this:

• 2 seperate Hard-Disks; not 2 different partitions.
• Less then, or 1GB RAM.

Ur cured from long verifing times.

While having your pagefile on a separate disk on a separate IDE chain will boost paging performance, and overall computer performance during heavy I/O tasks, it has nothing to do with the time it takes to verify client data in BF2. During the verify process BF2 is performing an MD5 hash check on the files from the hard disk. The disk is performing a read operation. No disk writes are in progress at this time! Therefore, it doesn't make any difference wether the files being read are on the hard drive in their original place, or on a different hard drive, containing the swap file.

[Painl3ss.NL]

Re: If you have 2HDS, low RAM, Verifing Time Solved!

During the verify process BF2 is performing an MD5 hash check on the files from the hard disk. The disk is performing a read operation. No disk writes are in progress at this time! Therefore, it doesn't make any difference wether the files being read are on the hard drive in their original place, or on a different hard drive, containing the swap file.

How do you think a CPU can access anything on a disk?
A CPU sees its registers and the virtual memory. To be able to do the md5 check the data on the disk has to be loaded into the virtual memory and this can trigger the chain of event which lead to swapping/paging out of data.
It depends whether the block of data that are being read in are big or small.
It's a trade-off: lots of little reads are inefficient because of the overhead per read, but a few big reads can trigger swapping/paging. Most programmers would go for farely big chucks I would guess, though.

It's been a while since i worked on assemblers and compilers (and it wasn't an x86-core). But I haven't come across a cpu that has assembly instruction that accesses the harddisk immediately.



P.S.
I think somebody said that removing the pagefile worked fastest (never tried it myself). This must be because of the paging algorithm, it may try to free-up memory even if it does not yet need it. The purpose being there is always free memory available so new applications load faster, the downside being that it actually starts moving virtual memory to the swap file even if it is not yet strictly necessary.

Bottom line, it's not that easy to predict how much benefit a system will have (if any), it's kind of complex and I may have overlooked things, but those who call it bs, should come with some better arguments.

I think you missed to point that any analysis of harddisk data will require virtual memory and that may cause paging/swapping.

[imported_The_Revrend]

Re: If you have 2HDS, low RAM, Verifing Time Solved!

On point that needs to be made. Virtual Memory is HD Space that is being accessed through some type of overlay software in the Operating System to emulate ram. If you write to virtual memory, you write to the HD

Rev

[Regent]

Re: If you have 2HDS, low RAM, Verifing Time Solved!

On point that needs to be made. Virtual Memory is HD Space that is being accessed through some type of overlay software in the Operating System to emulate ram. If you write to virtual memory, you write to the HDRev

My point exactly. And since the verify process is reading the disk, it doesn't matter where the info is being read from, it will still take the same amount of time.

How do you think a CPU can access anything on a disk?
A CPU sees its registers and the virtual memory. To be able to do the md5 check the data on the disk has to be loaded into the virtual memory and this can trigger the chain of event which lead to swapping/paging out of data.

So what's your point? The disk has to be read, regardless. This thread is about verifying Client Data in BF2. The claim is that moving the page file to a separate disk will somehow cause it to be read faster, which is incorrect.

I think somebody said that removing the pagefile worked fastest (never tried it myself). This must be because of the paging algorithm, it may try to free-up memory even if it does not yet need it. The purpose being there is always free memory available so new applications load faster, the downside being that it actually starts moving virtual memory to the swap file even if it is not yet strictly necessary.

That's why I don't run a page file.

I think you missed to point that any analysis of harddisk data will require virtual memory and that may cause paging/swapping.

I'd like to point out that even with no page file present, and 2Gb of RAM, BF2 still reads the data off the disk. If the data read during the MD5 check is paged by the OS, why doesn't my system verify 60 times faster than when it had a page file, since the data would have been loaded into RAM?

I never stated that having the page file on another disk wouldn't improve overall computing performance. Did you bother to read the thread? You're the one that needs a better argument, or at least one that's on topic.

BTW, the page file is virtual memory. RAM is memory.

[Lazy8s]

Re: If you have 2HDS, low RAM, Verifing Time Solved!

My point exactly. And since the verify process is reading the disk, it doesn't matter where the info is being read from, it will still take the same amount of time.

It does matter. The pagefile has a much smaller and more efficient FAT meaning the search time is alot smaller. also, all the information is stored contiguously on the disk meaning seek time is greatly reduced as well.

Quote:
Originally Posted by Painl3ss.NL
I think somebody said that removing the pagefile worked fastest (never tried it myself). This must be because of the paging algorithm, it may try to free-up memory even if it does not yet need it. The purpose being there is always free memory available so new applications load faster, the downside being that it actually starts moving virtual memory to the swap file even if it is not yet strictly necessary.


That's why I don't run a page file.

It slows it down even more, however. The theory is by removing the page file windows will put everything into memory thus speeding up page hits but it doesn't. Instead it leaves it on the hard disk meaning instead of having all the pages in 1 file with a nice FAT stored contiguously in one point, it has to search your hard disk's FAt and then go find the page wherever it may lie.

Also, when you have to page file windows has nowhere to dump memory errors meaning it's much harder to diagnose computer problems.

I'd like to point out that even with no page file present, and 2Gb of RAM, BF2 still reads the data off the disk. If the data read during the MD5 check is paged by the OS, why doesn't my system verify 60 times faster than when it had a page file, since the data would have been loaded into RAM?

BTW, the page file is virtual memory. RAM is memory.

Lol best quote ever. I should make this a sig in tech forums. It's obvious you don't understand and that's fine, I didn't know until I took a whole class on it. Do some more reading on what the page file is used for. I posted a few links here, but google is your friend for sure.

[Regent]

Re: If you have 2HDS, low RAM, Verifing Time Solved!

It slows it down even more, however. The theory is by removing the page file windows will put everything into memory thus speeding up page hits but it doesn't. Instead it leaves it on the hard disk meaning instead of having all the pages in 1 file with a nice FAT stored contiguously in one point, it has to search your hard disk's FAt and then go find the page wherever it may lie.

:laugh:

Virtual Memory is always in use, page file or not, and cannot be ‘turned off’. I merely set the system up to use no page file space on the disk. When BF2 asks for an allocation of Virtual memory space, the addresses are assigned to RAM. Why don't you post a link to a tech document that says "In the absence of a page file, Windows doesn't load files into RAM that would otherwise have been loaded into the page file." Complete bull$hit. I said this before, and I'll say it again. If Windows doesn't put files that would normally be paged into RAM, why does BF2 still use 1.3 Gb of RAM?? Your claiming that a page file is faster than physical RAM, for Christ's sake... You've failed to provide an explanation or post a link to any information that backs up your rather hilarious version of logic.


Also, I never said disk performance was equal between files located on a disk, and files that are paged. I stated that the performance difference between reading files from a page file located on the system drive, and a page file located on a separate drive, would be negligible because the verify process isn't an I/O intensive operation. It's a read operation, plain and simple. That is, after all, the topic of this thread.

Also, when you have to page file windows has nowhere to dump memory errors meaning it's much harder to diagnose computer problems.

Have you ever used a dump file to diagnose a computer problem? I sure haven't... Just in case, I did a search of the forums of pcper.com.

91,000 members, 3,550,554 posts, and only 5 results for a search of "memory.dmp" in the forums. None of them mention using memory.dmp to troubleshoot a problem. Guess there's a good reason Im not worried about it.

[Lazy8s]

Re: If you have 2HDS, low RAM, Verifing Time Solved!

:laugh:

Virtual Memory is always in use, page file or not, and cannot be ‘turned off’. I merely set the system up to use no page file space on the disk. When BF2 asks for an allocation of Virtual memory space, the addresses are assigned to RAM. Why don't you post a link to a tech document that says "In the absence of a page file, Windows doesn't load files into RAM that would otherwise have been loaded into the page file." Complete bull$hit. I said this before, and I'll say it again. If Windows doesn't put files that would normally be paged into RAM, why does BF2 still use 1.3 Gb of RAM?? Your claiming that a page file is faster than physical RAM, for Christ's sake... You've failed to provide an explanation or post a link to any information that backs up your rather hilarious version of logic.

1) I already posted 2 documents that stated this from the microsoft website.
2) If it used 1.3gig when you had a page file, you deleted the page file, and it STILL uses 1.3gig that means it did not put anything else into memory when the page file was removed. Thank you for difinitively proving me correct.
2) I'm claiming a page file is faster than searching for a page on the hard drive, and it is. If you can show me a quote where I said a page file was faster than RAM I will be glad to revoke that statement because it is certainly not true. That's like saying a flash drive is faster than L2 cache, but I assure you I never even implied that.
4) I posted proof the first time and you kept arguing so I tried to clear it up because you either never went to the links or you didn't understand what they said. I'm trying not to point out for everyone how you blatently do not understand but keep arguing anyways. You're welcome for my trying to be nice. Posting laughing at me when you're just proving me right and at the same time showing even more your lack of understanding is just poor form. I'm not here to get into a flame war, I just want to clarify for people who do read this and actually do want to understand why it is a bad idea to remove your paging file.

Also, I never said disk performance was equal between files located on a disk, and files that are paged. I stated that the performance difference between reading files from a page file located on the system drive, and a page file located on a separate drive, would be negligible because the verify process isn't an I/O intensive operation. It's a read operation, plain and simple. That is, after all, the topic of this thread.



Have you ever used a dump file to diagnose a computer problem? I sure haven't... Just in case, I did a search of the forums of pcper.com.

91,000 members, 3,550,554 posts, and only 5 results for a search of "memory.dmp" in the forums. None of them mention using memory.dmp to troubleshoot a problem. Guess there's a good reason Im not worried about it.

Yes I use memory dump to diagnose problems all of the time where I work. you know that blue screen of death? memory dump. You know how a program crashes and windows asks you to submit the error? memory dump. Maybe 91,000 people don't use the memory dump file, or maybe they look at it and it makes it easy enough to diagnose the problem they don't have to post about it. Either way it's none of my concern how many people understand how a computer works.

[Regent]

Re: If you have 2HDS, low RAM, Verifing Time Solved!

Quite frankly, your assertion that "Windows doesn't page to RAM" is absurd enough to invalidate your opinion altogether, and I don't wish to argue the point further here. Feel free to post the argument in a tech forum of your choice, and we'll continue the debate in the presence of members who have a better tech background than "20+ years old, done finished some tech schoolin', and now work at Best Buy."

Post the link to the forum when it's posted.

[Painl3ss.NL]

Re: If you have 2HDS, low RAM, Verifing Time Solved!

On point that needs to be made. Virtual Memory is HD Space that is being accessed through some type of overlay software in the Operating System to emulate ram. If you write to virtual memory, you write to the HD

I must admit I used the term virtual memory in a confusing (wrong ?) way in this context.
Instead of "virtual memory" read RAM, and what you call "virtual memory" read swap/page file.
So I'll use RAM and pagefile instead from now on.

(In my defence the CPU actually sees a virtual "address" space which a memory magement unit translates to ram addresses, and it was bedtime for me ;-))

Quite frankly, your assertion that "Windows doesn't page to RAM" is absurd enough to invalidate your opinion altogether

When Windows runs out of ram memory, it pages a part of the RAM to the harddisk to free up RAM memory (the pagefile).

Feel free to post the argument in a tech forum of your choice, and we'll continue the debate in the presence of members who have a better tech background than "20+ years old, done finished some tech schoolin', and now work at Best Buy."

Not called for. If credentials are required, I'ld probably do ok (definately in age ;-), and related to that a lot of years experience in the embedded soc market). Point is, credentials are not verifiable in a forum, but arguments are.
So lets stick to the arguments.

So what's your point? The disk has to be read, regardless. This thread is about verifying Client Data in BF2. The claim is that moving the page file to a separate disk will somehow cause it to be read faster, which is incorrect.

I agree that the disk is read regardsless, my point is that what is read from the disk ends up in RAM memory (to do the actual verification), thus possibly triggering writing to the pagefile, in which case it's better to have your pagefile on a separate disk.

because the verify process isn't an I/O intensive operation. It's a read operation, plain and simple.

If a disk read is not I/O, then what do you consider I/O?

I'd like to point out that even with no page file present, and 2Gb of RAM, BF2 still reads the data off the disk. If the data read during the MD5 check is paged by the OS, why doesn't my system verify 60 times faster than when it had a page file, since the data would have been loaded into RAM?

You have 2Gb of RAM (and no pagefile). Because of the 2GB the data will NOT be paged to a pagefile, but it WILL be copied into RAM (to do the MD5 check).
You can ask yourself why it is so much slower on a 512M system . That is because it cannot keep all the data in RAM and will start moving parts of the data from RAM to the pagefile. And then we can save on HD seek times if we put the pagefile on another disk.
Note that whatever is written to the pagefile, has to be written back into RAM at some point, so when this process (paging) starts you'll take a huge performance hit.

Off to work, I'm in a different timezone I guess ;-)

[Lazy8s]

Re: If you have 2HDS, low RAM, Verifing Time Solved!

Quite frankly, your assertion that "Windows doesn't page to RAM" is absurd enough to invalidate your opinion altogether, and I don't wish to argue the point further here. Feel free to post the argument in a tech forum of your choice, and we'll continue the debate in the presence of members who have a better tech background than "20+ years old, done finished some tech schoolin', and now work at Best Buy."

Post the link to the forum when it's posted.

Who said windows doesn't page to RAM? I read all the posts and I can't find it anywhere. I said windows won't put the pages from the page file into RAM if you turn it off, which is true. No need to post this in a tec forum. Not only do I not want to bring a debate to their forum which would look terrible but you have all the links to find out for yourself on the first page of this post.

[Princess_Frosty]

Re: If you have 2HDS, low RAM, Verifing Time Solved!

The pagefile is part of your total virtual memory which is a combination of your physical memory and the pagefile, the Physical memory is your system RAM and your pagefile is a file on your hard drive.

Typically when loading files and data, they are taken from the hard drive and put into RAM ready to be used, however if the amount of data you're loading exceeds the amount of RAM you have then your pagefile is used. This means copying files from the disk to itself in the pagefile location. When the files in the pagefile are needed they are swapped with files that are in RAM. This is the common cause of machines with low RAM having reasonable frame rates but the occasional extreeme drop in frame rate for only a fraction of a second (chances are a sound file or something was swapped)

Having your pagefile stored on a 2nd disk (assuming that disk is of equal read/write/seek speed) should speed up loading on machines with little RAM, depending exactly on how much RAM you have and how much data you're loading

Having the pagefile on a different partition which is on the same physical disk wont help however, you're still reading and writing to the same disk.

The problem a lot of people will find with this is that often 2nd hard disks are smaller and slower and tend to be drives you had a long time ago, defragged far less and when you setup a pagefile on the disk is usualy fragmented itself (unless theres a fair bit of contiguous space on the disk)

A lot can also depend on what connection the drives are using, which is primary and slave etc, I'd recommend that you use 2 disks of equal speed both being set to master on 2 different channels.

My setup is 2xRaptor (10,000 RPM) SATA hard drives in RAID 0 (stripped) which means its more or less best I keep my pagefile default since my primary hard drive is well over twice as fast as a standard medium sized IDE/SATA drive.

If you're running 2Gb or more RAM I wouldn't bother moving your pagefile, although disabling the pagefile all togther while appearing clever often results in more problems and slow downs than any gains you get from it.

Remember moving and resizing the pagefile can be potentially problematic for a system, its best to know exactly what you're doing before you change anything, make sure you understand what it is you're doing and the possible side effects of doing it wrong, before you change anything.