Spare some chang- er... time?
I need to run a test on a data structure very similar to the one described here (http://www.cplusplus.com/forum/lounge/47826/#msg259997 ), but I don't have, nor do I know anyone with, the kind of hardware to run it properly.
I would need:
* Windows XP or newer. A UNIX is a possibility if you're willing to compile the sources yourself.
* 10 GiB of free space on solid state storage of any kind connected through any interface. Even a USB drive is good enough, as long as it's big enough and has no moving parts.
* Around 48 hours to transfer the test data (assuming an average rate of 55 KiB/s).
The actual test should take, I think, less than a second. I'm testing how much disk seeking affects the performance of the algorithm (my hypothesis is "a lot" :o)) and exactly how fast it can be made on an ideal setup.
I would need:
* Windows XP or newer. A UNIX is a possibility if you're willing to compile the sources yourself.
* 10 GiB of free space on solid state storage of any kind connected through any interface. Even a USB drive is good enough, as long as it's big enough and has no moving parts.
* Around 48 hours to transfer the test data (assuming an average rate of 55 KiB/s).
The actual test should take, I think, less than a second. I'm testing how much disk seeking affects the performance of the algorithm (my hypothesis is "a lot" :o)) and exactly how fast it can be made on an ideal setup.
Erm... Care to expand on your system requirements? I can't imagine with a process that intensive you're only limiting factor is storage, especially if you're expecting a turn around from a 10GB SQL DB of less then a second.
Seek time is going to be EVERYTHING in this. Is that why you are shying away from moving parts? The only real advantage an SS drive would have over say a RAID 0 would be the write time (though it would be a significant advantage).
Seek time is going to be EVERYTHING in this. Is that why you are shying away from moving parts? The only real advantage an SS drive would have over say a RAID 0 would be the write time (though it would be a significant advantage).
That's why I said the data structure was similar, not identical. The 10 GiB are used by a flat file that keeps a compressed hash-based index. SQLite will participate very little in the test, precisely because it can't keep up with the throughput. O(log n) access is too slow for such a large table.
The test won't include write times, only read times. I haven't yet decided on how I'm going to grow the index.
A typical image query on this file takes between 1000 and 15000 reads of 5100 bytes each. After sorting the reads to minimize the distance the disk's head has to travel (of course, this assumes the file is not fragmented, which is somewhat reasonable since it's written in a quick burst), I get about 5.5 ms per read.
On the other hand, a device with zero seek time and a sustained transfer rate of 10 MiB/s should be capable of 2000 reads/s, placing its worse time just higher than an HDD's best time of 6 s.
The test won't include write times, only read times. I haven't yet decided on how I'm going to grow the index.
A typical image query on this file takes between 1000 and 15000 reads of 5100 bytes each. After sorting the reads to minimize the distance the disk's head has to travel (of course, this assumes the file is not fragmented, which is somewhat reasonable since it's written in a quick burst), I get about 5.5 ms per read.
On the other hand, a device with zero seek time and a sustained transfer rate of 10 MiB/s should be capable of 2000 reads/s, placing its worse time just higher than an HDD's best time of 6 s.
Last edited on
Topic archived. No new replies allowed.