I think it's Windows only, but that's probably not a problem (you don't mention what system you're using, but .ico files are most commonly associated with Windows?)
When I was looking around for an icon editor, I noticed that IcoFX seems to be quite popular, but I decided to stick with the more basic tool.
IcoFX is an award winning freeware icon editor. It is an all-in-one solution for icon creation, extraction and editing. It is designed to work with Windows XP, Windows Vista, Windows 7 and Macintosh icons supporting transparency.
@Catfish4 Yes and can you install it for me two? Also test it on my computer to make sure it works, thanks! :P
Actually, Sib Icon Creator and LiquidIcon look good, but they have a download or a purchase button. I am wondering if the download button is for a free trial or not.
GIMP can save images as .ico files. It can also save them in pretty much every other format known to man as well, and is a pretty powerful editor. It might be overkill for just a few icons, but you may end up using it for other things too.