Well I don't consider myself a crypto expert by any means, but here's my stab at it:
* The algorithm does nothing to mask statistical properties of the passphrase or the plaintext. Subtracting adjacent bytes of the ciphertext (the various modulo 256 and the multiplication by 3/2 only adds about 1 or 2 bits of ambiguity per byte) reveals plaintext[i]*2+passphrase[i%n]. I was trying to crack it from this angle yesterday, but I gave up after a couple hours. Like I said, I'm not an expert.
* If for some reason the attacker knows the plaintext at a particular position, the ciphertext past that position, and probably before as well, becomes significantly weaker.
* If the plaintext changes partially or completely without changing the passphrase and the attacker knows both ciphertexts, I believe they can deduce the passphrase. I couldn't quite find the function to do this.
This is the kind of encryption a Roman general might have used to communicate, or that a novice programming student comes up with in ten minutes. It's painfully obvious that whoever wrote it either didn't know anything about the work in algorithm design and cryptanalysis that happened in the 20th century, or they didn't care. I mean, they didn't even try filtering the passphrase through a one-way function.
Feel free to correct any errors or add anything I've missed.
You have pretty much it it right on: it does not mask the statistical properties of the password or the plaintext.
The encryption comes down to modular arithmetic, with the only non-guaranteed fixed cycle being the plaintext, and the only unknown cycle length being the password length.
I imagine that watching a few thousand iterations would be enough to notice the pattern and give the password length. Or maybe use a fourier transform. IDK.
After getting the password length, it is really just unrolling password-length sections to match known text statistics, building a histogram on the password’s elements.
Once the password is cracked, game over.
To be honest, though, whether my method to crack it is actually valid (or even easy) is beyond my knowledge. I just studied Crypto 101 theory in University, and it has been some years, so how to actually do all this stuff is just me guessing. Regular cryptographers do it all the time, apparently, and could surely suggest a much smarter way.
[edit]
I suspect that Ang was the developer, by both how offended he became and also by a vocabulary slip (using “I” when speaking about the algorithm, IIRC).
If non-experts are able to point out problems with this encryption algorithm it makes me wonder what someone who lives and breathes crypto-analysis would think of this.
You'd think that would be exactly the sort of person whom a company developing a "highly effective app" for password encryption would employ for the task, right?
It doesn't take a cryptographer to know that one should not be designing a symmetric encryption algorithm, when there's no reason not to use Rijndael or Twofish.
It does take one to know how to do that, though.
Using those libraries is actually shockingly easy. More than that, Windows has strong encryption built right into the OS!
I did it myself last year, when I wanted a class to use AES over a stream. The actual code called about six OS functions and four variables, wrapped nicely into a generously-spaced 50-line header file and a 100-line implementation file (for both encryption and decryption functions).
My bad, last value was always the same because I sent the function a number 1 too small.
Thanks for the advice Duthomhas. I've been tinkering with it - learned a lot from this, still digging deeper. I don't think I'll get it any time soon, but we'll see how it goes.