Thursday, 4 May 2023

Web Tutorial: QBasic Filestitch

It's exceedingly rare that I ever get to use QBasic to solve a work problem. As such, when that happens, I'm inclined to commemorate it in true geek fashion - by writing a web tutorial.

Some context coming up!

When my company last year upgraded to Oracle Cloud, all the POS machines at various outlets were now sending CSV files to the relevant department. The problem was, whereas before the files were already consolidated before the department received it, with this new, ahem, upgraded system, the outlets were now sending the files separately. All twenty of them. As you can imagine, manually stitching the files together daily, quickly became a total pain in the ass.

What I eventually did was use QBasic to write an executable file that would perform the consolidation for them. All they needed to do was double-click on the executable, fill in a bit of information, and an entirely new CSV file would be produced in seconds, or less!

Today, I will show you how to write this code. Of course, because I don't want to give away any company secrets, certain things have been changed. But the idea remains the same!

We first have a series of CSV files. You can access them at my repository. Each of these files are named this way.

tx_{date in "YYYYMMDD" format}_{outlet four-digit id}.csv.

These have the following fields. A sample is shown here. The fields are for date, time, order number, item name, category, quantity and order type.

filestitch/tx_20221210_1122.csv
"2022-12-10", "11:16:12", "100001210015", "Strawberry Deluxe Surprise", "MAIN", "1", "DINE-IN"
"2022-12-10", "11:16:12", "100001210015", "Mango Slurpee", "DRINK", "2", "DINE-IN"
"2022-12-10", "11:18:09", "100001210016", "Chocolate Overload", "MAIN", "2", "TAKE-AWAY"
...


filestitch/tx_20221210_1123.csv
"2022-12-10","10:45:36","444002770055","Mixed Fruit Cheesecake","SIDE","1","DINE-IN"
"2022-12-10","10:45:36","444002770055","Hot Six","MAIN","1","DINE-IN"
"2022-12-10","10:45:36","444002770055","Chocolate Overload","MAIN","1","DINE-IN"
...


filestitch/tx_20221210_1124.csv
"2022-12-10","11:00:02","300005557800","Banana Sandwich Special","MAIN","4","TAKE-AWAY"
"2022-12-10","11:02:58","300005557801","Cream Cake","SIDE","1","TAKE-AWAY"
"2022-12-10","11:03:55","300005557802","Ice Lychee Tea","DRINK","2","TAKE-AWAY"
...


filestitch/tx_20221210_1125.csv
"2022-12-10","10:49:10","200000778889","Blueberry Muffin","SIDE","4","TAKE-AWAY"
"2022-12-10","10:55:11","200000778890","Blueberry Muffin","SIDE","2","TAKE-AWAY"
"2022-12-10","11:20:06","200000778891","Chocolate Overload","MAIN","1","TAKE-AWAY"
...


There are only four files, but in the case of my company, we had to deal with over twenty files. This is how we define each outlet, through the ids and outlets arrays.
CLS

DIM ids(4) AS STRING
DIM outlets(4) AS STRING

ids(0) = "1122"
outlets(0) = "TUAS"
ids(1) = "1123"
outlets(1) = "PASIR RIS"
ids(2) = "1124"
outlets(2) = "BALESTIER"
ids(3) = "1125"
outlets(3) = "LITTLE INDIA"


Here, we ask for the date, and ask for it in a certain format. We assign it to the variable FileDate. This is going to help us identify the correct file.
ids(3) = "1125"
outlets(3) = "LITTLE INDIA"

INPUT "Date of file? (YYYYMMDD format, eg, 20221101)", FileDate$


And then we ask for the directory, and assign it to the variable FileDir. Here, we're going to assume it's in the C drive.
INPUT "Date of file? (YYYYMMDD format, eg, 20221101)", FileDate$
INPUT "Directory?", FileDir$


This is pretty basic (hur hur) so far.




And here, we create a file in the directory specified by FileDir, with the name derived from FileDate. This will be the CSV file we write to. We will define this as #1.
OPEN "C:\" + FileDir$ + "\tx_" + FileDate$ + "_consolidated.csv" FOR OUTPUT AS #1


What we have now is a For loop to iterate through the outlets. As far as I know, there is no graceful way to grab the size of an array in QBasic, so let's just hardcode this. And after everything, we close #1 as a matter of good cleanup practices.
OPEN "C:\" + FileDir$ + "\tx_" + FileDate$ + "_consolidated.csv" FOR OUTPUT AS #1

FOR i = 0 TO 3 STEP 1

NEXT

CLOSE #1


We formulate a filename based on the element in the ids array pointed to by i, FileDate and FileDir. Needless to say, that directory actually has to exist. On the next line, we print FileName for better visibility.
FOR i = 0 TO 3 STEP 1
    FileName$ = "C:\" + FileDir$ + "\tx_" + FileDate$ + "_" + ids(i) + ".csv"
    PRINT FileName$

NEXT


Here, we open the file. It's defined as #2.
FOR i = 0 TO 3 STEP 1
    FileName$ = "C:\" + FileDir$ + "\tx_" + FileDate$ + "_" + ids(i) + ".csv"
    PRINT FileName$
    OPEN FileName$ FOR INPUT AS #2
NEXT


Now, what if that file isn't found or is empty? We trap this condition in an If block, checking if #2 has no data. We use the LOF() function, passing in 2 (for #2)as an argument for this. If it returns 0, either no such file exists or the file is empty.
FOR i = 0 TO 3 STEP 1
    FileName$ = "C:\" + FileDir$ + "\tx_" + FileDate$ + "_" + ids(i) + ".csv"
    PRINT FileName$
    OPEN FileName$ FOR INPUT AS #2
    IF LOF(2) = 0 THEN

    ELSE

    END IF

NEXT


And if so, we print out the name of the outlet (using the element in the outlets array pointed to by i) and use the KILL statement on #2. I know this sounds really violent, but all it does is delete the file, if found.
FOR i = 0 TO 3 STEP 1
    FileName$ = "C:\" + FileDir$ + "\tx_" + FileDate$ + "_" + ids(i) + ".csv"
    PRINT FileName$
    OPEN FileName$ FOR INPUT AS #2
    IF LOF(2) = 0 THEN
        PRINT outlets(i) + " OUTLET HAS NO DATA"
        KILL FileName$
    ELSE

    END IF
NEXT


If there is such a file and it has data, we use a Do-while loop, ending only when the end of the file is reached. This is checked using the EOF() function and passing in 2 (for #2) as an argument.
FOR i = 0 TO 3 STEP 1
    FileName$ = "C:\" + FileDir$ + "\tx_" + FileDate$ + "_" + ids(i) + ".csv"
    PRINT FileName$
    OPEN FileName$ FOR INPUT AS #2
    IF LOF(2) = 0 THEN
        PRINT outlets(i) + " OUTLET HAS NO DATA"
        KILL FileName$
    ELSE
        DO WHILE NOT EOF(2)

        LOOP
        CLOSE #2

    END IF
NEXT


So let's try it with an obviously wrong date and directory. You'll get a few of these, two for each file. Just keep clicking "Yes".




Here, it lists all the files it didn't find, which is all four of them.




Here, we use the INPUT statement to get the data from #2, the current file! After the first argument, it's all the data from the CSV files, separated by commas. Note that it has to be, for obvious reasons, in the same sequence!
FOR i = 0 TO 3 STEP 1
    FileName$ = "C:\" + FileDir$ + "\tx_" + FileDate$ + "_" + ids(i) + ".csv"
    PRINT FileName$
    OPEN FileName$ FOR INPUT AS #2
    IF LOF(2) = 0 THEN
        PRINT outlets(i) + " OUTLET HAS NO DATA"
        KILL FileName$
    ELSE
        DO WHILE NOT EOF(2)
            INPUT #2, COLDATE$, COLTIME$, COLNUM$, COLITEM$, COLCAT$, COLQTY$, COLTYPE$
        LOOP
        CLOSE #2
    END IF
NEXT


Then we have InputStr, which is initialized as an empty string. Then we append all the data to it. Note that we have to use the CHR$() function with 34 as an argument, to add the double-quotes.
FOR i = 0 TO 3 STEP 1
    FileName$ = "C:\" + FileDir$ + "\tx_" + FileDate$ + "_" + ids(i) + ".csv"
    PRINT FileName$
    OPEN FileName$ FOR INPUT AS #2
    IF LOF(2) = 0 THEN
        PRINT outlets(i) + " OUTLET HAS NO DATA"
        KILL FileName$
    ELSE
        DO WHILE NOT EOF(2)
            INPUT #2, COLDATE$, COLTIME$, COLNUM$, COLITEM$, COLCAT$, COLQTY$, COLTYPE$
            InputStr$ = ""
            InputStr$ = InputStr$ + CHR$(34) + COLDATE$ + CHR$(34) + ","
            InputStr$ = InputStr$ + CHR$(34) + COLTIME$ + CHR$(34) + ","
            InputStr$ = InputStr$ + CHR$(34) + COLNUM$ + CHR$(34) + ","
            InputStr$ = InputStr$ + CHR$(34) + COLITEM$ + CHR$(34) + ","
            InputStr$ = InputStr$ + CHR$(34) + COLCAT$ + CHR$(34) + ","
            InputStr$ = InputStr$ + CHR$(34) + COLQTY$ + CHR$(34) + ","
            InputStr$ = InputStr$ + CHR$(34) + COLTYPE$ + CHR$(34) + ","

        LOOP
        CLOSE #2
    END IF
NEXT


We also add this line to write the outlet name into a new column.
FOR i = 0 TO 3 STEP 1
    FileName$ = "C:\" + FileDir$ + "\tx_" + FileDate$ + "_" + ids(i) + ".csv"
    PRINT FileName$
    OPEN FileName$ FOR INPUT AS #2
    IF LOF(2) = 0 THEN
        PRINT outlets(i) + " OUTLET HAS NO DATA"
        KILL FileName$
    ELSE
        DO WHILE NOT EOF(2)
            INPUT #2, COLDATE$, COLTIME$, COLNUM$, COLITEM$, COLCAT$, COLQTY$, COLTYPE$
            InputStr$ = ""
            InputStr$ = InputStr$ + CHR$(34) + COLDATE$ + CHR$(34) + ","
            InputStr$ = InputStr$ + CHR$(34) + COLTIME$ + CHR$(34) + ","
            InputStr$ = InputStr$ + CHR$(34) + COLNUM$ + CHR$(34) + ","
            InputStr$ = InputStr$ + CHR$(34) + COLITEM$ + CHR$(34) + ","
            InputStr$ = InputStr$ + CHR$(34) + COLCAT$ + CHR$(34) + ","
            InputStr$ = InputStr$ + CHR$(34) + COLQTY$ + CHR$(34) + ","
            InputStr$ = InputStr$ + CHR$(34) + COLTYPE$ + CHR$(34) + ","
            InputStr$ = InputStr$ + CHR$(34) + outlets(i) + CHR$(34)
        LOOP
        CLOSE #2
    END IF
NEXT


Finally, we write InputStr into #1.
FOR i = 0 TO 3 STEP 1
    FileName$ = "C:\" + FileDir$ + "\tx_" + FileDate$ + "_" + ids(i) + ".csv"
    PRINT FileName$
    OPEN FileName$ FOR INPUT AS #2
    IF LOF(2) = 0 THEN
        PRINT outlets(i) + " OUTLET HAS NO DATA"
        KILL FileName$
    ELSE
        DO WHILE NOT EOF(2)
            INPUT #2, COLDATE$, COLTIME$, COLNUM$, COLITEM$, COLCAT$, COLQTY$, COLTYPE$
            InputStr$ = ""
            InputStr$ = InputStr$ + CHR$(34) + COLDATE$ + CHR$(34) + ","
            InputStr$ = InputStr$ + CHR$(34) + COLTIME$ + CHR$(34) + ","
            InputStr$ = InputStr$ + CHR$(34) + COLNUM$ + CHR$(34) + ","
            InputStr$ = InputStr$ + CHR$(34) + COLITEM$ + CHR$(34) + ","
            InputStr$ = InputStr$ + CHR$(34) + COLCAT$ + CHR$(34) + ","
            InputStr$ = InputStr$ + CHR$(34) + COLQTY$ + CHR$(34) + ","
            InputStr$ = InputStr$ + CHR$(34) + COLTYPE$ + CHR$(34) + ","
            InputStr$ = InputStr$ + CHR$(34) + outlets(i) + CHR$(34)
            PRINT #1, InputStr$
        LOOP
        CLOSE #2
    END IF
NEXT


Run this! This time, use "20221012" and "filestitch" as the inputs, to fit the file names.




And it's done.




A new file, tx_20221210_consolidated.csv, should appear in the directory. Its contents should be the combination of all the other files that have "20221210" in the filename. And at the end of each line, we have appended the outlet name!
...
"2022-12-10","15:21:02","100001210027","Strawberry Deluxe Surprise","MAIN","1","TAKE-AWAY","TUAS"
"2022-12-10","16:16:08","100001210028","Mixed Fruit Cheesecake","SIDE","1","TAKE-AWAY","TUAS"
"2022-12-10","17:09:14","100001210029","Mixed Fruit Cheesecake","SIDE","1","TAKE-AWAY","TUAS"
"2022-12-10","18:55:55","100001210030","Blueberry Muffin","SIDE","12","TAKE-AWAY","TUAS"
"2022-12-10","19:21:41","100001210031","Over-the-top Meltdown","MAIN","1","TAKE-AWAY","TUAS"
"2022-12-10","19:44:30","100001210032","Banana Sandwich Special","MAIN","10","TAKE-AWAY","TUAS"
"2022-12-10","19:51:02","100001210033","Strawberry Deluxe Surprise","MAIN","5","TAKE-AWAY","TUAS"
"2022-12-10","19:51:02","100001210033","Blueberry Muffin","SIDE","5","TAKE-AWAY","TUAS"
"2022-12-10","19:52:11","100001210034","Chocolate Cat And Mouse","MAIN","1","TAKE-AWAY","TUAS"
"2022-12-10","19:53:41","100001210035","Over-the-top Meltdown","MAIN","1","DINE-IN","TUAS"
"2022-12-10","19:53:41","100001210035","Chocolate Overload","MAIN","1","DINE-IN","TUAS"
"2022-12-10","19:55:58","100001210036","Cream Cake","SIDE","4","TAKE-AWAY","TUAS"
"2022-12-10","19:56:55","100001210037","Fruit Fritters","SIDE","10","TAKE-AWAY","TUAS"
"2022-12-10","19:58:10","100001210038","Strawberry Deluxe Surprise","MAIN","1","DINE-IN","TUAS"
"2022-12-10","19:58:10","100001210038","Hot Six","MAIN","1","DINE-IN","TUAS"
"2022-12-10","19:58:10","100001210038","Blueberry Muffin","SIDE","5","DINE-IN","TUAS"
"2022-12-10","19:58:10","100001210038","Cream Cake","SIDE","2","DINE-IN","TUAS"
"2022-12-10","19:58:10","100001210038","Ice Lychee Tea","DRINK","10","DINE-IN","TUAS"
"2022-12-10","19:58:10","100001210038","Ice Lemon Tea","DRINK","2","DINE-IN","TUAS"
"2022-12-10","10:45:36","444002770055","Mixed Fruit Cheesecake","SIDE","1","DINE-IN","PASIR RIS"
"2022-12-10","10:45:36","444002770055","Hot Six","MAIN","1","DINE-IN","PASIR RIS"
"2022-12-10","10:45:36","444002770055","Chocolate Overload","MAIN","1","DINE-IN","PASIR RIS"
"2022-12-10","10:58:09","444002770056","Over-the-top Meltdown","MAIN","2","TAKE-AWAY","PASIR RIS"
"2022-12-10","10:58:09","444002770056","Passionfruit Excitement","MAIN","2","TAKE-AWAY","PASIR RIS"
"2022-12-10","10:58:09","444002770056","Mixed Fruit Cheesecake","SIDE","6","TAKE-AWAY","PASIR RIS"
...


Happy programming!

It's not often I get to use QBasic professionally, much less write an entire web tutorial for it. I thoroughly enjoyed myself here.

By fair means or file,
T___T

No comments:

Post a Comment