Death by 1000 Programming Sins

[Cheraphy]

138: Not checking your pointers. (see reference 1)
139. Spaghetti Code, Kludges, 'nuff said. (but still see reference 2)

Ref-1) http://xkcd.com/371/
Ref-2) http://xkcd.com/844/

[kfmfe04]

140. Stepping in C++ doo-doo, by not reading the C++ specs...
141. ...then reading the C++ specs and still not understanding what the heck it's saying 8'O

Ref: http://www.cplusplus.com/forum/general/44245/

edit: gotta luv noobs who think C++ is easy - lol

[3hM2Nwbp]

@kfmefe04
By the C++ specs, do you mean the ISO standard specification, or the standard library (or both?).

IMO the standard library documentation still leaves some to be desired...some parts still seem rather ambiguous to me. ( Example: http://cplusplus.com/forum/general/43306/ )

[3hM2Nwbp]

142. Claiming platform independence while your make-file won't work on Windows (without major modifications).

143. Not documenting the EXACT VERSIONS of any and all library dependencies. (Ever get these mysterious linker errors that you can't find support on when trying out a snazzy new API?)

[kfmfe04]

@LL

If some error happens during the operation, the stream's badbit flag is set, and if the appropriate flag has been set with ios::exceptions, an exception is thrown.

too vague - when is badbit set and when is exceptions set? do they ever overlap?

If a friend declaration appears in a local class (9.8) and the name specified is an unqualified name, a prior declaration is looked up without considering scopes that are outside the innermost enclosing non-class scope. For a friend function declaration, if there is no prior declaration, the program is ill-formed. For a friend class declaration, if there is no prior declaration, the class that is specified belongs to the innermost enclosing non-class scope, but if it is subsequently referenced, its name is not found by name lookup until a matching declaration is provided in the innermost enclosing nonclass scope.

too convoluted (feels just like a clever perl script that I wrote a year ago and now, I have no idea what it means...)

144. Changing template code in too many places (often >1) at a time before compiling (risks getting buried by an avalanche of very hard to read template error messages)

145. Intermixing template code and polymorphism in a brittle manner (making code extremely hard to refactor)

146. Relying on the debugger to warn programmers on the required interface for a template class, without the benefit of useful comments

147. Returning a non-const handle to a class member needlessly

edit: 143. is a scary one, very scary...

[hamsterman]

kfmfe should have written instead of what he wrote:

edit: gotta luv noobs who think C++ is good - lol

I should really switch to something more consistent.._{sorry about being off topic}

As for "on topic",

148. Being paranoid about performance

It's the root of many sins.

[kfmfe04]

hamsterman is right, as usual - I stand corrected - it's amazing how enthusiastic beginners are at something they know so little about (not unlike getting married?)

consistently guilty of 148 - leads to unnecessarily complex code

I think it's a C++ disease - we see it all over the place, from STL to boost

[strongdrink]

@#148 Amen! I must admit that I have this disease.. altough i dont use boost.. i just make my own stuff..

[S6k9GNh0]

You know, boost classes might perform better than what you can do yourself in most cases. Boost Spirit, Boost ASIO, etc. are all extremely optimal.

[kfmfe04]

149. do not confuse logical with the physical (memory vs files)

I recently generated a 4.3GB binary file for fast random access (csv is 2GB but much slower) - thinking that files are contiguous like memory (only half right), I naturally used fseek( ..., SEEK_SET ) to offset from the beginning of the file

well, everything worked fine until I tried to access the end of the file - surprisingly, it took forever, so I surrendered and looked up the man pages on fseek and found the SEEK_END option so now, before I do any fseek(), I do a quick calculation to see if I am closer to the beginning or the end of file - took a bit of playing around to calculate the offsets correctly, as I was reading records in by chunks and the chunks may not necessarily line up coming from the front or the back

which brings up the lesson of the day: files are logically contiguous but could be physically fragmented (no duh, when you think about it), which explains why there are two options for fseek() (actually, there is a third I didn't mention, SEEK_CUR), both of which are useful for big files. OTOH, memory appears to be logically and physically contiguous - you can check by dynamically allocating 4 GB on the heap: you may be denied, but allocating 1 GB four times may actually work

but I wonder, is it the nature of blocks on disks that cause the fragmentation? for example, are SSDs more like memory (logically and physically contiguous) or more like a drive (logically contiguous, but physically fragmented)? I'm guessing it's the former...

[helios]

That's weird. Seeking should take constant time regardless of the whence parameter unless you're using a very crappy file system or an odd device. In particular, FAT, NTFS, ext*, and ReiserFS should not take longer to seek to the end from SEEK_SET than from SEEK_END. Their files are not linked lists of blocks.

but I wonder, is it the nature of blocks on disks that cause the fragmentation? for example, are SSDs more like memory (logically and physically contiguous) or more like a drive (logically contiguous, but physically fragmented)?

Disk and RAM are equivalent. You can format RAM with a fragmentable file system and you can allocate disk space like how you allocate RAM. The difference is that file systems are designed, among other things, to allow arbitrary resizing without reallocation and copy.
So, no. A formatted SSD is fragmentable.

[chrisname]

A fragmented SSD is fine, though, since random access is what SSDs excel at.

[kfmfe04]

before I played with the large file, I thought a binary file and RAM would be exactly the same in this sense: in RAM, it doesn't matter if I offset a pointer forward 1 byte or 1GB - it is equally fast

I was surprised to see that it isn't the case with the binary file - I assume there is an underlying reason, otherwise, there is no reason to have both SEEK_SET and SEEK_END for fseek() - one can logically convert one to the other - btw, I am using ext3 so nothing out of the ordinary - I was guessing that it was the physical fragmentation that causes the difference between forward and backwards, but I really don't know - but there is a big difference when I tried it

150. choose the right Mysql engine
151. delete all indexes before LOAD DATA INFILE
152. don't do long-lived command-lines in ssh without adjustments to timeouts or nohup (like a LOAD DATA INFILE) - instead, do it via nxmachines

for #('"s and giggles, I thought I'd load a billion row CSV (equivalent to the aforementioned binary file) into Mysql - a very bad mistake is an attempted load with an index already in place - another mistake is using the wrong engine - MyISAM takes about 11min to load on my machine (faster than I expected) while InnoDB took much longer - had to abort since it was taking too long

[helios]

Of course I don't have a Linux VM when I need it.
I'll try this out with some huge GiB files and see what happens. Files are supposed to be random access regardless of how fragmented they are. Fragmentation only matters when reading them from beginning to end, not when seeking.

[1yR4jE8b]

Disk and RAM are equivalent.

Yup, Ram is just insanely faster. There's actually an application called RamDisk which actually lets you create a storage partition in your ram for uber fast access. It works pretty well, and it only adds a small amount of time to your startup/shutdown times so that the contents of the disk can be read to/written from disk so you don't lose data..

[helios]

Configuration: Debian 6.0.0 AMD64, ext3, gcc 4.4.5, libc 2.11.2
Dataset: 12 GiB+ file

Code:

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdio.h>
#include <time.h>

#ifndef O_LARGEFILE
#define O_LARGEFILE 0
#endif

int main(){
	unsigned char buffer[1024];
	clock_t t0,t1;
	int f;
	size_t a;
	
	f=open("bigfile",O_RDONLY|O_LARGEFILE);
	t0=clock();
	lseek64(f,13229844307ULL-1024,SEEK_SET);
	read(f,buffer,1024);
	t1=clock();
	for (a=0;a<1024;a++)
		printf("%02X",(int)buffer[a]);
	printf("\n%d\n",t1-t0);
	
	lseek64(f,0,SEEK_SET);
	t0=clock();
	lseek64(f,1024,SEEK_END);
	read(f,buffer,1024);
	t1=clock();
	for (a=0;a<1024;a++)
		printf("%02X",(int)buffer[a]);
	printf("\n%d\n",t1-t0);
	close(f);
	return 0;
}

Output:

root@debian:/# ./seektest
8F53AB841F43B67553AC85031230A2BC4DBB8F53AB841F43B67553AC8503124085694DBB8F53AB84 
1F43B67553AC85031251332B4DBB8F53AB841F43B67553AC8503126046554DBB8F53AB841F43B675 
53AC8503126F32144DBB8F53AB841F43B67553AC8503127B1EF94DBB8F53AB841F43B67553AC8503 
1288B1A94DBB8F53AB841F43B67553AC850312964C354DBB8F53AB841F43B67553AC850312A36DB0 
4DBB8F53AB841F43B67553AC850312AE410A4DBB8F53AB841F43B67553AC850312B86EBC4DBB8F53 
AB841F43B67553AC850312C1AB264DBB8F53AB841F43B67553AC850312CCCFA24DBB8F53AB841F43 
B67553AC850312DBA8FC4DBB8F53AB841F43B67553AC850312EC77374DBB8F53AB841F43B67553AC 
850312FF2D024DBB8F53AB841F43B67553AC8503130D99BC4DBB8F53AB841F43B67553AC85031319 
9B584DBB8F53AB841F43B67553AC85031324EB164DBB8F53AB841F43B67553AC8503132F848A4DBB 
8F53AB841F43B67553AC8503133BD2344DBB8F53AB841F43B67553AC8503134657CE4DBB8F53AB84 
1F43B67553AC850313515F7B4DBB8F53AB841F43B67553AC8503135C02924DBB8F53AB841F43B675 
53AC8503136743684DBB8F53AB841F43B67553AC850313754F504DBB8F53AB841F43B67553AC8503 
13814A1C4DBB8F53AB841F43B67553AC8503138DB4634DBB8F53AB841F43B67553AC8503139A3F5A 
4DBB8F53AB841F43B67553AC850313A56CF14DBB8F53AB841F43B67553AC850313AF88FC4DBB8F53 
AB841F43B67553AC850313B640C44DBB8F53AB841F43B67553AC850313BC30634DBB8F53AB841F43 
B67553AC850313C29D7F4DBB8F53AB841F43B67553AC850313C95CF34DBB8F53AB841F43B67553AC 
850313D03CA24DBB8F53AB841F43B67553AC850313D618EF4DBB8F53AB841F43B67553AC850313DC 
F3614DBB8F53AB841F43B67553AC850313E5CE934DBB8F53AB841F43B67553AC850313EF4DBC4DBB 
8F53AB841F43B67553AC850313F7AF6D4DBB8F53AB841F43B67553AC85031400C5774DBB8F53AB84 
1F43B67553AC8503140974DA4DBB8F53AB841F43B67553AC85031411BE4C4DBB8F53AB841F43B675 
53AC8503141A7AD04DBB8F53AB841F43B67553AC85031422AEF54DBB8F53AB841F43B67553AC8503 
142C03304DBB8F53AB841F43B67553AC85031433BEF74DBB8F53AB841F43B67553AC8503143C15DC 
4DBB8F53AB841F43B67553AC850314444B0F4DBB8F53AB841F43B67553AC8503144DFE644DBB8F53 
AB841F43B67553AC85031459D5514DBB8F53AB841F43B67553AC850314649F634DBB8F53AB841F43 
B67553AC8503146F95684DBB8F53AB841F43B67553AC85031478AF584DBB8F53AB841F43B67553AC 
8503147FE6814DBB8F53AB841F43B67553AC8503148675F5 
0
8F53AB841F43B67553AC85031230A2BC4DBB8F53AB841F43B67553AC8503124085694DBB8F53AB84 
1F43B67553AC85031251332B4DBB8F53AB841F43B67553AC8503126046554DBB8F53AB841F43B675 
53AC8503126F32144DBB8F53AB841F43B67553AC8503127B1EF94DBB8F53AB841F43B67553AC8503 
1288B1A94DBB8F53AB841F43B67553AC850312964C354DBB8F53AB841F43B67553AC850312A36DB0 
4DBB8F53AB841F43B67553AC850312AE410A4DBB8F53AB841F43B67553AC850312B86EBC4DBB8F53 
AB841F43B67553AC850312C1AB264DBB8F53AB841F43B67553AC850312CCCFA24DBB8F53AB841F43 
B67553AC850312DBA8FC4DBB8F53AB841F43B67553AC850312EC77374DBB8F53AB841F43B67553AC 
850312FF2D024DBB8F53AB841F43B67553AC8503130D99BC4DBB8F53AB841F43B67553AC85031319 
9B584DBB8F53AB841F43B67553AC85031324EB164DBB8F53AB841F43B67553AC8503132F848A4DBB 
8F53AB841F43B67553AC8503133BD2344DBB8F53AB841F43B67553AC8503134657CE4DBB8F53AB84 
1F43B67553AC850313515F7B4DBB8F53AB841F43B67553AC8503135C02924DBB8F53AB841F43B675 
53AC8503136743684DBB8F53AB841F43B67553AC850313754F504DBB8F53AB841F43B67553AC8503 
13814A1C4DBB8F53AB841F43B67553AC8503138DB4634DBB8F53AB841F43B67553AC8503139A3F5A 
4DBB8F53AB841F43B67553AC850313A56CF14DBB8F53AB841F43B67553AC850313AF88FC4DBB8F53 
AB841F43B67553AC850313B640C44DBB8F53AB841F43B67553AC850313BC30634DBB8F53AB841F43 
B67553AC850313C29D7F4DBB8F53AB841F43B67553AC850313C95CF34DBB8F53AB841F43B67553AC 
850313D03CA24DBB8F53AB841F43B67553AC850313D618EF4DBB8F53AB841F43B67553AC850313DC 
F3614DBB8F53AB841F43B67553AC850313E5CE934DBB8F53AB841F43B67553AC850313EF4DBC4DBB 
8F53AB841F43B67553AC850313F7AF6D4DBB8F53AB841F43B67553AC85031400C5774DBB8F53AB84 
1F43B67553AC8503140974DA4DBB8F53AB841F43B67553AC85031411BE4C4DBB8F53AB841F43B675 
53AC8503141A7AD04DBB8F53AB841F43B67553AC85031422AEF54DBB8F53AB841F43B67553AC8503 
142C03304DBB8F53AB841F43B67553AC85031433BEF74DBB8F53AB841F43B67553AC8503143C15DC 
4DBB8F53AB841F43B67553AC850314444B0F4DBB8F53AB841F43B67553AC8503144DFE644DBB8F53 
AB841F43B67553AC85031459D5514DBB8F53AB841F43B67553AC850314649F634DBB8F53AB841F43 
B67553AC8503146F95684DBB8F53AB841F43B67553AC85031478AF584DBB8F53AB841F43B67553AC 
8503147FE6814DBB8F53AB841F43B67553AC8503148675F5 
0
root@debian:/# 

The output was identical under NTFS.

[rapidcoder]

in RAM, it doesn't matter if I offset a pointer forward 1 byte or 1GB - it is equally fast

No, it is not true. Actually if you forward a pointer 1 byte, you can be almost 100% sure, the byte pointed by it resides in L1 cache (because of memory prefetching done by MMU). And L1 cache is insanely fast - typically 2 or 3 CPU cycles. On the other hand, when you request a random piece of memory, it can take several hundreds or even thousands of CPU cycles. And it can take ages if it was swapped out....

So the differences between disk and RAM are not really that huge, except the fact that RAM is simply orders of magnitude faster, both in terms of transfer rate and access time.

Yup, Ram is just insanely faster. There's actually an application called RamDisk which actually lets you create a storage partition in your ram for uber fast access. It works pretty well, and it only adds a small amount of time to your startup/shutdown times so that the contents of the disk can be read to/written from disk so you don't lose data..

RAM disks were faster in times of DOS. Now there is no point in using them. All moder OSes buffer disk accesses, free memory is used to store the most frequently accessed blocks. So if you have plenty of free memory, this is equivalent of having RAM disk.

[Cheraphy]

@rapidcoder

You're a bit of a know-it-all, aren't you? :P

[Mathhead200]

Luc Lieber wrote:

126. Accidentally Blending languages (I actually did #import a few times...)

Lol...

//You mean like
inculde java.util.*; //I did this a few times

public class MainClass {
   public static void main(String[] args) {

   }
}

Also when I was using C# for a while I started to forget to make my Java classes public. You think that type of problem would be easy to spot, but I didn't until I tried to load the class using reflection! http://www.facebook.com/topic.php?uid=2204806663&topic=17821

[anonymous23323124]

For 126, My biggest one was writing public or private before every member after switching from C#, or using default arguments after switching back...