RC4 encryption and newlines

Hello, I wrote a simple program that can encrypt/decrypt files using the stream cipher RC4:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
#include <algorithm>
#include <fstream>
#include <iostream>
#include <string>
int main(int argc, char *argv[])
{
    if (argc < 2)
    {
        std::cerr << "Must specify file!";
        return 0;
    }
    if (argc > 2)
    {
        std::cerr << "Too many arguments!";
        return 0;
    }
    unsigned char S[256];
    int i = 0;
    for (i = 0; i < 256; i++)
        S[i] = i;
    int j = 0, choice = 0;
    while (choice != 1 && choice != 2)
    {
        std::cout << "Encryption - 1, decryption - 2: ";
        std::cin >> choice;
    }
    std::string key;
    std::cout << "Enter the key: ";
    std::cin >> key;
    for (i = 0; i < 256; i++)
    {
        j = (j + S[i] + key.at(i % key.length())) % 256;
        std::swap(S[i], S[j]);
    }
    std::string argv1 = argv[1], printFile;
    std::ifstream read(argv1, std::ios::binary);
    if (choice == 1)
        printFile = argv1 + ".rc4";
    if (choice == 2)
        printFile = "1" + argv1.substr(0, argv1.length() - 4);
    std::ofstream print(printFile);
    char x;
    j = 0;
    i = 0;
    while (read.read(&x, 1))
    {
        i = (i + 1) % 256;
        j = (j + S[i]) % 256;
        std::swap(S[i], S[j]);
        char temp = S[(S[i] + S[j]) % 256] ^ x;
        print.write(&temp, 1);
    }
}


In the command line you have to pass in the file name you want to encrypt/decrypt.

This program, though, has one fatal flaw: the dreaded newlines. They use up 2 bytes instead of 1 and appear randomly as we XOR the bytes of the plaintext and the keystream. This is a major problem, because this means that the ciphertext will almost always be larger than the plaintext and wherever the first newline happens to appear, the plaintext from that point on will be messed up (because of it being 2 bytes long while we're reading the file byte by byte). Could anyone help me come up with a solution?

Also, is there any reason this would not work with any file type, like .jpg or .zip? From what I can understand, since it encrypts/decrypts the file byte by byte it should be able to recreate any file of any type, is my assumption correct? Thank you for your time!

P.S. To observe the bug in action, create a text file with the text Hello. Nice weather. inside and use a key abc to encrypt it. The ciphertext will have a newline pretty much in the middle of it and the decrypted text will look like this: Hello. NicbĄÄu‡{=iūĮ. Try this with the same plaintext and the key abcd. A newline doesn't appear and the decryption is successful.
Last edited on
Open both files in binary mode.
1
2
3
4
std::ifstream read(argv1, std::ios::binary);
// ...
// std::ofstream print(printFile);
std::ofstream print( printFile, std::ios::binary ) ;
Thank you, works perfectly!
Topic archived. No new replies allowed.