How to structure a long program

[freddie1]

// cl StdLibParse1.cpp /O1 /Os /MT /EHsc
// 200,192 Bytes
#include <iostream>
#include <sstream>

int main()
{
 std::string input = "Zero, One, Two, Three, Four, Five, Six";
 std::istringstream ss(input);
 std::string token;
 while(std::getline(ss, token, ',')) 
 {
    std::cout << token << '\n';
 }
  
 return 0;
} 


#if 0

Output:
=======
Zero
 One
 Two
 Three
 Four
 Five
 Six

#endif 

Built with VC19 from Visual Studio 2015 we have a 200 k binary. Let me show a PowerBASIC rendition of the above…

#Compile Exe                                    ' No headers needed for this program.  All the dynamic memory allocation,
#Dim All                                        ' array handling, and string functionality are part of the core language.

Function PBMain() As Long
  Local iParseCount As Long
  Local strLine As WString
  Local pStrs() As WString
  Register i As Long

  StrLine = "Zero, One, Two, Three, Four, Five" ' Assign CSV string to dynamic OLE string variable
  IParseCount = ParseCount(strLine,",")         ' Determine  number of comma separated values
  Redim pStrs(iParseCount) As Wstring           ' Dynamic memory allocation of array string variable
  Parse strLine, pStrs(), ","                   ' Call PowerBASIC Parse command to Parse String based on comma
  For i=0 To UBound(pStrs, 1)-1                 ' Loop through pStrs array outputting parsed ‘tokens’.
    Console.Print LTrim$(pStrs(i))
  Next i
  Erase pStrs()                                 ' Free memory.  This is optional.  PowerBASIC automatically does it
  Waitkey$

  PBMain=0
End Function
                  

#if 0
Zero
One
Two
Three
Four
Five
#EndIf 

The above builds to 13,292 bytes with PowerBASIC, or about fifteen times smaller than the C++ version. But its possible to do even considerably better than that in C++ using my TCLib and my String Class, and eliminating static linkage to the C Runtime. Here would be that program using my String Class. You won’t be able to test this code yourself because my String Class as represented by Strings.cpp and Strings.h would be needed, as well as my TCLib. I’d be happy to give these to anyone that wants them, but I just wanted to point that out in case anyone would attempt to build this…

// cl Parse1.cpp Strings.cpp /O1 /Os /GS- /Zc:sizedDealloc- /link TCLib.lib kernel32.lib user32.lib
// 4,608 Bytes VC19 (VStudio 2015); x64; ansi
#include <windows.h>
#include "stdio.h"
#include "Strings.h"

int main()
{
 String s1 = "Zero, One, Two, Three, Four, Five, Six";        // Assign CSV String To Be Parsed To String Object
 int iParseCount = s1.ParseCount(',');                        // Determine Number Of CSVs
 String* pStrs   = new String[iParseCount];                   // Allocate Array To Hold Above Determined Number
 s1.Parse(pStrs, ',', iParseCount);                           // Parse The String
 for(int i=0; i<iParseCount; i++)                             // Output The CSVs
     pStrs[i].Print(true);                                    // ....
 delete [] pStrs;                                             // De-Allocate Dynamic Memory
 
 return 0;
}        

So now we’re down to an amazing 4,608 bytes in C++ with an x64 build in ansi. Wide character might bump it up 500 bytes or so.

#if 0

Output:
=======
Zero
One
Two
Three
Four
Five
Six

#endif

[freddie1]

Well, enough of that. But all I’ve written above applies to your question concerning where to go from Petzold to increase your knowledge of this type of coding. I don't believe there are any additional books that can be picked up, unfortunately. I’d have to say that most of what I’ve picked up I learned in the context of PowerBASIC and the PowerBASIC Forums. That community of folks is really more of a ‘community’ than any other place one is likely to find on the internet. The folks tend to be older – like 40 to 80 years old, and have been with computers since mainframe times – like me.

There are a number of web sites involved with PowerBASIC. The links I earlier provided about the function pointer stuff that you seem to have liked are at Jose Roca’s website. A fellow countryman of yours in Germany named Theo Gottwald provided Jose with the website’s server to host the site. Theo said something to the effect that when a genius is found capable of lifting the whole world up through his intelligence one must provide the genius with everything needed. I was extremely honored when Jose asked me to join his forum by providing me a board there. I’m the least knowledgeable of all the folks there. Other coders Jose asked to join him originally were Patrice Terrier of France, and Charles Pegge of Scotland, and Steve Hutcheson of Australia. For quite a while I was the only American. Recently James C. Fuller joined. He's my neighbor, relatively speaking; he lives in the U.S. state just north of me - New York.

Patrice is a graphics expert. I was glad I was able to help him get started with C and C++. He markets high end graphic software routines I believe mostly to C++ coders. He had to move to C++ because of the 64 bit issue with PowerBASIC.

Charles works on compilers mostly. He's the creator of OxygenBasic, which is a really interesting compiler.

Steve (Hutch) runs the main assembler Forum on the internet....

www.masm32.com

He's definitely hard core and has always been a big PowerBASIC supporter. He also has a PowerBASIC board on his site, and a lot of the active members of his MASM community (Microsoft Macro Assembler) are also PowerBASIC coders too. Hutch always has an interesting take on everything.

So you might be able to find some useful material there. Both Patrice and I have quite a lot of tutorial stuff there.

Here is a program you might find useful for learning material...

Speed Comparisons - x86 / x64 C++ verses PowerBASIC x86
July 31, 2015, 07:48:09 PM
http://www.jose.it-berater.org/smfforum/index.php?topic=5058.0

Its on my Discussion board. Let me tell you why you might find it useful. You had mentioned about displaying output and scrolling data. The genesis of the above referenced program is one of my main forestry applications we use at work to calculate timber sale volumes and create reports. There is a great deal of data processing involved in calculating one of our timber sales, and a lot of the stuff is over a network. Data from Microsoft Access databases and Excel spreadsheets is read, calculations are done, various data is written to databases on a server, and reports are written to Microsoft Word using COM/OLE. It can take a little while, that is, its not instantaneous, so some kind or hourglass, progress bar or whatever is needed to inform the user that work is being done, and that the software isn't 'locked up'.

But I decided I didn't want the typical solution of an hourglass cursor or a progress bar. It occurred to me that what would be really, really cool would be a scrolling output screen where the program would post progress reports of just what it is doing at the moment, and where intermediate results and tables of data would be output as they occurred. In other words, something like the major operating system updates from Microsoft where you can watch the progress of the updates in a window as they are being installed. Such a setup would beat the heck out of a dumb progress bar that tells nothing about the underlying processing taking place.

But oh how tricky it is to do!!! Believe it or not, one of the biggest stumbling blocks in developing software like that is coming up with test data for a mock up application. Computers are so fast its hard to come up with something that takes several seconds or minutes to do. And forget about trying to force the computer to run a dummy loop for billions of repetitions that does nothing. Modern C++ optimizing compilers are so darned smart they'll circumvent just about anything like that that you throw at them. In other words, they'll see a loop doesn't do anything, or they'll pre-calculate the end result of a series of arithmetic operations rather than running the loop!!!!! Its truly amazing. I know its hard to believe this is a problem, i.e., just coming up with test data to write the program, but it is.

Here's how I solved it. And its going to take a little digression on my part. Several years ago a new user of PowerBASIC posted in the PowerBASIC Forums a little C algorithm he had where he was testing the speed of C verses PowerBASIC in running this algorithm. And the C code was killing the PowerBASIC code. The C code was running like a hundred times faster than the algorithm converted to PowerBASIC. He wanted to know what was going on, as he had read that PowerBASIC was as fast as C, which is to say real fast. A number of members tested the results and found out this new user was right. This all caused quite a debate. Finally, one of the asm gurus at the PowerBASIC forum decompiled the C binary to see what the underlying assembler code was doing. What he found out was truly amazing. Microsoft's C compiler was smart enough to look at the code executing within the loop, which involved some simple additions, subtractions, multiplications, and divisions, and recognize that it was a mathematical operation which had a pre-determined result, that is, it was sort of a terminating series or something - mathematically. So the compiler wasn't even generating code to run the loop. It was pre-calculating the result. That's why it was so much faster. But PowerBASIC simply ran the loop and did the calculations. That's why it was being beaten so badly.

[freddie1]

Well, that raised all kinds of ruckus! Some of the PowerBASIC users felt the C compiler was cheating and the numbers didn't prove anything. Others said PowerBASIC was fairly beaten in that it wasn't smart enough to see what the C compiler was able to see right off.

Finally, one member - John Gleason, came up with the idea of developing a string manipulation test arduous and complicated enough that the C compiler would be forced to implement the algorithm and not 'cheat' like as described above. Here is a description of that algorithm he came up with...

1) Create a 15 MB string of dashes;
2) Change every 7th dash to a "P";
3) Replace every "P" with a "PU" (hehehe);
4) Replace every dash with an "8";
5) Put in a CrLf every 90 characters;
6) Output last 4K to Message Box;

This whole discussion was intriguing to me because I was a pretty competent C and C++ coder at that time and already had been using my C++ String Class for a pretty long time. But my reasons for developing my String Class didn't really involve performance or speed of the running algorithm, but rather my performance in writing C code where low level memory buffer manipulations of strings as bytes was nearly killing me. And I had never tested my String Class for performance so here looked like a good opportunity. And truth be told I was some worried my code would be unsatisfactory and run poorly. To make a very, very long story short I ended up having to do a total rewrite of my String Class over this, and I spent months working on this exercise. Before it was all through, I had written algorithms to implement the above in C, assembler, C and PowerBASIC using pointers, C++ with my String Class and the String Class from the C++ Standard Library, and a Replace algorithm from Bruce Eckel’s “Thinking In C++” book, Volume Two book.

So the reason I told you all that is that the algorithm I used in my attempt to come up with some test data where I could simulate a very long running process in my effort to create a continuously scrolling display is that algorithm as described above. However, instead of item #1 above 'Create a 15 MB string of dashes', I did something completely different.

My program doesn't just run that algorithm one time. I run it within a user configurable loop count, where the user gets to choose how many times the algorithm above is to run, e.g., fifty times, a hundred times, etc. And further, I didn't use 15 megabytes. Every time the loop is run the program generates a random number between one million bytes and twenty-five million bytes to process. So all of the above livens things up a bit.

Having covered all that, there is one more remaining bit of extreme trickiness to the above app and the algorithm it is trying to implement. To put it simply, it is absolutely impossible to implement the above described app as a single threaded app. Believe me, I've tried, and it can't be done. The reason is really tricky, and requires something of a feeling for how Windows works to understand. Let me start to explain.

Assume you have a main top level window on an application with a button on it. When the user clicks that button a code routine will run to implement the above algorithm, and lets say the algorithm is set to run 100 times within a loop. And lets further assume we're talking something like two minutes of heavy duty processing to finish. Our output window which is supposed to scroll is only large enough to hold twenty-five lines of output - not one hundred lines. For every one of the 100 iterations of the loop we'll text out a line of output to the output/display window stating how many bytes of data were processed, and how long it took. We're keeping track of processing time with GetTickCount(). In order to force a WM_PAINT message so as to display our output after each loop iteration we're calling InvalidateRect() from within our loop processing code. Sounds doable and workable, right??? Absolutely wrong! It won't work worth a crap.

When you click that command button to start the data processing, Windows sends a WM_COMMAND message to the Window Procedure. At that point within your WM_COMMAND message handler code you call your function to start the data processing. All is proceeding well in that code, bytes are being processed, and WM_PAINT messages are being put in the program's message queue. But the output window remains totally blank. No output whatsoever shows up, and in your despair if you attempt to even drag the output window on the screen it won't even move. The program appears to be locked up. If you are patient (few would be, most would go to Task Manager to end a seemingly locked up program) maybe two minutes later the output screen would instantaneously show up with all the output, that is, all 100 lines at once, and the screen would be scrolled all the way to the bottom. Not exactly what you wanted. You wanted to see the output as the program was running - not when it was over.

What happened is that once the process began churning through the string manipulation code it was stuck in that WM_COMMAND routine. It couldn't get out to do anything else! In other words, even though InvalidateRect() calls were being made from that routine, which is a call to Windows to send a WM_PAINT message, none of those calls could be executed because that's another Window Procedure call - and the process is still within the WM_COMMAND message, that is, the WM_COMMAND message hasn't returned yet. So you see the original plan of how this was to work was completely subverted by Windows!

And it even gets worse. Windows is totally diabolical. If all that wasn't bad enough Windows, in its desire for efficiency and speed, has internal routines to scan through the message queue amalgamating WM_PAINT messages together by combining invalid rectangles so that it only has to process one WM_PAINT instead of multitudes of them. So you're totally dead in the water with all this. Nothing's 'gonna work.

Except --- threads! What you have to do to implement scrolling output as the algorithm proceeds is create an application with two threads; the main or start up thread will consist of the GUI, that is, the start up window with the button on it that starts data processing, and the output/display window. The second or 'worker thread' will consist of the code to do the string/byte processing. And the operating system will thread switch continuously between these two threads as the application runs, so that when execution resumes in the GUI part of the application, if there are any WM_PAINT messages in the queue, they'll get processed and one or more lines of output will show up in the display window. When Windows thread switches back to the worker thread, more bytes will be processed. All this thread switching will occur many times per second, so to the user it will seem like the app's GUI is always responsive, and data is continuously being processed. In fact, there can be a 'Cancel' button on the main form to stop processing early at any time, and it will seem to work.

All in all Cal, I think this would be a very worthwhile application for you to examine at your stage in the learning process. I expect economics is like Forestry - lots of opportunities for lengthy data processing tasks.

[cal2004]

Freddie, I was wrong with super basic, complete nonsense due to my bad English skills. I was asking for power basic, of course (and not super basic). Excuse me for that.

Your posts are extremely interesting and helpful! When I wrote that I do not care for size, I meant because of my rather poor skills. Being an economist I do of course care for efficiency, very much. However at the moment and with my skills, I am happy if an app runs well. And the efficiency of power basic is very impressive.

Currently I am reading a lot of your stuff on Jose (thought it might be German somehow :-) and tried out your alternative architecture which I already love due to its ability to structure apps.

I would love to learn from you how to include word for reports and get an easy app for scrolls. However you must not hurry. At the moment, I have more than enough new stuff and things to read more carefully. And I also like very much to elaborate things on my own because thinking it through even with struggling a lot is the best way to learn...

You really enlighted my mind and opened a new world to me! You should consider writing a book about that stuff!

[freddie1]

I'd like to help you with that Word output Cal, but I'm tied up with other things now, and I guess you are too. Perhaps I should put together a tutorial about it, and maybe publish it here and at Jose's forum. Let me know when you are ready for it. By the way, if you are interested in PowerBASIC I believe they just updated their page where one buys stuff, and I think you can download their trial version of PB 9 for free. That would work with 99% of the stuff on my board at Jose's website.

I just remembered something. There is a German speaking PowerBASIC Forum. I've never gone there as my German skills aren't that great, in spite of all the studying of it I did once. I'm going to search for it now to see if it still exists.

[cal2004]

That is no problem with me. I have more than enough new stuff to study. And I also like to research for myself, e.g. Jose Roca Forum. I will let you know, when I am ready.

I already found the German forum about powerbasic (see link). It has been closed since 2013. But there is a link to the English forum and to the Jose Roca Forum, which is highly recommended. And there is apparently also a German guy called Kirschbaum who sells powerbasic in Germany.

I will download powerbasic for sure!

I hope I will read from you here some time later.

http://www.powerbasic.de/html/forum.html