Showing posts with label MySQL. Show all posts
Showing posts with label MySQL. Show all posts

Sunday, 7 December 2025

Idempotence in CRUD operations

Hello, readers. Today we're going to talk about idempotence.

The concept of idempotence is paramount in database security. By saying that, of course, I feel compelled to explain what idempotence is. That term tends to come up a lot with regard to data transactions. The official definition is as follows, with regard to data transactions.
Idempotency is the property of an operation where performing it multiple times produces the same result as performing it once.


What that means is that if you apply an operation over and over, and it makes no difference to the database after the first time, that operation is idempotent.

With regard to CRUD (CREATE, READ, UPDATE, DELETE) operations, it's important to understand which ones are idempotent, and which ones run the risk of undesirable results if accidentally executed multiple times. 

Please note that code examples are in MySQL.

CREATE

Idempotence: No (except in special circumstances)
CREATE operations are not idempotent by default. If you run a CREATE operation multiple times, you are going to get multiple created rows.

INSERT INTO categories (name, description) VALUES ("Cat A", "test")


Running this three times will result in these three rows being added.
id name description
1 Cat A test
2 Cat A test
3 Cat A test


What if the design of the table was structured this way? What if name was supposed to be unique?
CREATE table categories (
  id INT AUTO_INCREMENT PRIMARY KEY,
  name VARCHAR(100) UNIQUE,
  description TEXT
)


Being unique.

In that case, that particular CREATE operation would be idempotent because the database simply wouldn't allow subsequent CREATE operations with the same values in the name column.
id name description
1 Cat A test


Also, if you defined an UPSERT operation instead of a CREATE, the record would only get inserted the first time. Subsequent times, the record would already exist, and therefore it would get updated... with the same values. Thus, making it idempotent. There is an exception to this exception, but that will be discussed in the UPDATE operation further down.
INSERT INTO categories (name, description)
VALUES ("Cat A", "test")
ON DUPLICATE KEY UPDATE
description = VALUES(description);


READ

Idempotence: Yes
This is the most straightforward operation in the sense that it is always idempotent. The first READ request makes no change to the database. Subsequent READ requests also make no change.

SELECT * FROM categories WHERE name = "Cat A"

READ operations are not
about changing data.

No exceptions to the rule, no nothing. This is as open-and-shut a case of idempotence as you're ever going to get in the world of computer science.

UPDATE

Idempotence: Yes (except in special circumstances)
In most cases, running an UPDATE operation on a specific record multiple times, has no effect beyond the first time. It will end up updating that record to the same values. In the query below, row id 1's name and description values would just keep being updated to "Cat A" and "test", which is basically no change.

UPDATE categories
name = "Cat A",
description = "test"
WHERE id = 1


Thus, this is idempotent. Unless...

What if the table had an Audit Field that tracked when the record was updated?
CREATE table categories (
  id INT AUTO_INCREMENT PRIMARY KEY,
  name VARCHAR(100),
  description TEXT,
  updated DATETIME
)


And what if the UPDATE operation was like this? Then the updated field would be a different value every time this operation was run, and therefore it would no longer be idempotent!

Time being tracked.


UPDATE categories
name = "Cat A",
description = "test"
updated = now()
WHERE id = 1


Also, what if the UPDATE operation wasn't on a specific row? This query updates all rows where the name field fits a particular condition. Kind of like dropping a bomb on an entire country instead of surgically taking out one terrorist. (Boy, this got dark)

UPDATE categories
name = "Cat A",
description = "test"
WHERE name like "%Cat%"


What if, just after the UPDATE operation above was run, a row was coincidentally inserted?
INSERT INTO categories (name, description) VALUES ("Cat B", "test")


Then we'd have two rows like this...
id name description
1 Cat A test
2 Cat B test


...but if the UPDATE operation was run again, this would be the result!
id name description
1 Cat A test
2 Cat A test


So, to conclude, in principle, the UPDATE operation is idempotent. But only if the values updated are absolute, and only if a very specific record is being updated.

DELETE

Idempotence: Yes (except in special circumstances)
In principle, DELETE operations are idempotent. After all, you can only delete a record successfully, once. After that, it no longer exists, so rerunning the same DELETE operation, achieves nothing. To employ a grim analogy, you can't kill someone twice.

You can only kill
someone once.

DELETE FROM categories WHERE id = 1


However, the concerns that exist for the UPDATE operation also exist for the DELETE operation. What if the DELETE operation wasn't on a specific row? This deletes all rows where the name field fits a certain filter.
DELETE FROM categories WHERE name LIKE "%CAT%"


And if, after the first DELETE operation, a record was added like so, rerunning the above DELETE operation would remove this row!
INSERT INTO categories (name, description) VALUES ("Cat B", "test")


In conclusion

A lot of what I've outlined is context. In simple cases, the answer of idempotence are similarly straightforward. But this is software development, where things are rarely that simple.

Wishing you a categorically good day,
T___T

Thursday, 6 November 2025

Why Your Database Needs Audit Fields

In most relational database schemas, there exists a convention where timestamps and text strings are stored to record when a row was last inserted or updated. These are created in columns, and these columns are commonly referred to as Audit Fields.

Time for an audit!

Take for example this schema in MySQL, for the table Members.
CREATE TABLE Members (
    "id" INT NOT NULL AUTO_INCREMENT,
    "name" VARCHAR(100) NOT NULL,
    "createdAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
    "createdBy" VARCHAR(50) NOT NULL DEFAULT "system",
    "updatedAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    "updatedBy" VARCHAR(50) NOT NULL DEFAULT "system"
);


The last four fields are Audit Fields:
- createdAt: this is a timestamp that is set to the current date and time when the record is created, and never changed.
- createdBy: this is a string that is set to the user's login when the record is created, and never changed.
- updatedAt: this is a timestamp that is set to the current date and time when the record is created, and changed to the current date and time every time the record is updated.
- updatedBy: this is a string that is set to the user's login when the record is created, and changed to the current user login every time the record is updated.

How do Audit Fields work?

"Created" fields. These are less useful because they should never be updated, and serve as a static record of when the record was created, and by who. However, it's good to have because it provides an initial reference point to troubleshoot if needed.
INSERT INTO Members (name)
VALUES ("Ally Gator");


id name createdAt createdBy updatedAt updatedBy
1 Ally Gator 2024-05-05 16:05:42 admin1 2024-05-05 16:05:42 admin1


"Updated" fields. These are updated when the record is created and every time the record is updated. While it's not as useful as a full audit log, it at least lets you know when the last update was. Let's say you run this query.
UPDATE Members SET name = "Allie Gator" WHERE id = 1;


id name createdAt createdBy updatedAt updatedBy
1 Allie Gator 2024-05-05 16:05:42 admin1 2024-05-16 12:27:21 admin1


Why Audit fields?

At the risk of stating the obvious, these are useful when you need to perform an audit (hence the name) on a database as to what records were created or updated, when, and by who. The importance of this only becomes increasingly obvious the larger the database grows.

Audit Fields can be a pain to set up, especially if you're not used to it. Once you've done it enough, though, you may find yourself wondering how you ever managed without them. Yes, they take up space. Yes, you have to bear them in mind during INSERT and UPDATE operations.

But man, they're awfully useful.

Examining data.

Imagine you need to do some forensics on some data that got added out of nowhere. You can check the timestamps and the ids of whoever created them. If they don't match any user on record, you know you have a problem. Scratch that - there's (almost) always a problem, but at least you have a better idea where it's coming from.

Or even if it's not a security breach, perhaps there's a dispute as to who was responsible for a certain data update? If the timestamps and user ids are right there in the database, there's instant accountability.

And all this is not even considering the fact that depending on the prevailing laws of the land, the presence of Audit Fields may even be mandatory.

Ultimately...

Get comfortable with the concept of Audit Fields. It's not new. It's pretty much timeless, in fact. It always represents some extra work at the start, but will save you so much hassle in the long run.

See you later, Allie Gator!
T___T

Thursday, 5 September 2024

Five Hilariously Unfortunate Names in Tech

Naming things is one of the hardest things in the tech industry. This shouldn't be surprising; naming things is one of the hardest things in the world, period. One runs into a whole host of problems such as accuracy, contradicting an already existing name for something else, and unintended meaning.

In the last case, this takes the form of unfortunate comedy, esepcially when seen through the lens of another culture. I think it's fair to say that the people who named these things in tech, never considered the Asian perspective.

1. Microsoft

This is the most ubiquitous and arguably the most boring name in tech. "Micro" and "soft". It's two words that appear in every other sentence in software technology, or in the case of the former, just technology, period.

Feeling a little...
soft?

Until you think of it as a descriptor for penis size. And from there on, insinuate that Bill Gates was overcompensating for something in the bedroom.

Unfortunate name? Yeah, it's pretty unfortunate.

2. Debian

Debian is touted as a "complete free operating system", but that's not what we'll focus on today. You see, this name gave me a serious case of the giggles when I first saw it, and that's because the word "debian" looks like "da bian", which is the romanization of the Chinese phrase for "defecate".

Number Two.

Yep. In mandarin, "xiao bian" is Number One and "da bian" is Number Two.

Non-Chinese will probably draw a blank as to why this one tickled my funny bone. So, the naming is not that unfortunate. I do wonder how Debian does in China or Taiwan, though...

3. Erlang

Erlang is a programming language, and that's all I know about it. Never touched it, don't think I'm likely to in the future.

And on its own, the name "Erlang" isn't really remarkable.

The deity Erlang
and his celestial
watchdog.

However, if you were brought up in a Chinese household with traditional values, chances are you'd have heard of the three-eyed deity "Erlang". The coincidence is not exactly unfortunate, but it is amusing.

4. MariaDB

MariaDB is a database. With an engine similar to MySQL, MariaDB and MySQL are often mentioned in the same breath.

Now, on its own, the name "Maria" is a beautiful name. Think "Santa Maria" or the Italian last name "DiMaria". However, in sunny Singapore, that name has racist connotations.

"Housekeeping!"

Yes, "Maria" is a derogatory term for "Filipina". Especially with regard to Filipino domestic helpers. Don't ask me why; I have not a solitary clue.

5. xAI

Let me begin by saying that Number 2 on this list can be a little far-fetched. It requires both the ability to speak Mandarin and a slight leap of imagination. But ask any Hokkien speaker how to pronounce the name of Elon Musk's pet Artificial Intelligence company, and you might get a snigger.

A really shitty coincidence.

You see, while "debian" can be taken to mean "defecate" in Mandarin, saying "sai" in Hokkien literally means "shit". Dung. Poop. Excrement.

Yep, this is absolutely unfortunate.

Conclusion

This is not to say that any of these companies should change their names. In the case of Microsoft, that name is long entrenched. In the case of the others, it's inevitable that whatever words you use, it's going to have a completely different meaning from what you intended in some other language. And sometimes, yes, these incidences are hilarious.

This is also why I object to censoring words just because they sound offensive in the English language. Considering the fact that there are thousands of languages in use in the world today, this is an exercise in futility. 


Till we meet again, xAI-yonara!
T___T

Thursday, 8 August 2024

Software Review: XAMPP

My first experience in 2010 configuring PHP on a Windows laptop was a bit of a nightmare. I not only had to run the executable file, I had to grapple with every line from php.ini. And that's not to mention running code and then testing database connections.

So image my relief years later when I discovered XAMPP, a package that would set up an Apache server, just like that.



After all the steps I'd taken to install PHP on that first laptop, not having to do it all over again, over and over, on the subsequent machines that I owned, was palpable. I soon became a big fan.

And today, I'm going to show my appreciation through this software review.

The Premise

XAMPP is basically a package comprising of an Apache server, and optionally, MySQL (recent versions use MariaDB instead). Thus if you need to run PHP or Perl quickly, this is a great solution. In fact, "XAMPP" is an acronym that stands for "Cross-platform Apache MySQL PHP Perl".

You double-click to run the installer. Subsequently, whenever you want to run PHP or Perl code, you start up the server.

The Aesthetics

Meh, it's orange and grey. The whole thing looks basic. Nothing to shout about from an artistic viewpoint, really. But at the same time, there's something about the simplicity of it all that's really attractive.

The Experience

Overall, using XAMPP was a pleasure. Compare this to the hassle of setting up and maintaining your own Apache server? Not even close. Come on.

The Interface

Starting this thing up is easy. Configuring it, also easy.





Gone are the days of hunting for the exact file. XAMPP opens that file up for you and gently warms you to be careful when changing it.

What I liked

XAMPP condenses the horribly complicated process of setting up an Apache server with a database, regardless of whether it's on a Windows or Mac platform, into a few simple steps. What's not to love? It's almost a bit too simple, if I'm being honest. But too simple is usually better than not simple enough.

Sensible conventions. Things like the default deployment path, port number, and such, aren't outlandish. I can get behind "htdocs" as a root path, even if I've been trained to recognize other conventions such as ASP.NET's "wwwroot".

Looks charmingly retro. Now, this could be seen as a negative, but this section is titled "what I liked", so here it is.

What I didn't

It would've been nice if some configurables were changeable through either a desktop or web interface rather than having to open up the config text file. On the other hand, the need to change these things doesn't come up all that often, so...

In the MacOS version, the executable is manager-osx which isn't really intuitive. 



Conclusion

If you need a quick-and-dirty Apache setup tool, who you gonna call? That's right - XAMPP! This tool has been around for the last  couple decades, and hasn't ever really gone out of fashion. That's because XAMPP doesn't pretend to be anything more than just an Apache server setup tool, and in that it's a godsend for PHP and Perl devs.

My Rating

8 / 10

An XAMPP-lary piece of work!
T___T

Saturday, 16 December 2023

Functions that handle NULL values in databases

Not every value in a database has a well-defined value. Sometimes there is no value, or a NULL.

Empty values!

In these cases, you may need to handle these values, especially if there are calculations involved. Take the following table, TABLE_SALES, for example. There are missing values in the DISCOUNT column.

TABLE_SALES
DATETIME ITEM QTY SUBTOTAL DISCOUNT
2023-10-10 12:13:10 STRAWBERRY WAFFLE 2 20 0
2023-10-10 12:44:54 STRAWBERRY WAFFLE 1 10 0
2023-10-11 15:03:09 CHOCO DELIGHT 1 25 -2.5
2023-10-11 18:22:42 ORANGE SLICES 5 30
2023-10-12 10:56:01 STRAWBERRY WAFFLE 4 40 -3

Now let's say we tried this query.
SELECT DISCOUNT FROM TABLE_SALES

This does not present a problem.
DISCOUNT
0
0
-2.5

-3

But what if we wanted to use it as part of a calculation? You would have situations where we tried to add NULL values to the value of SUBTOTAL.
SELECT (SUBTOTAL + DISCOUNT) AS NETT FROM TABLE_SALES

Functions to handle NULL values

There are functions to handle these cases. They are named differently in different database systems. In SQLServer, it's IFNULL(). In MySQL, it's ISNULL(). In Oracle, it's NVL().

In all of these cases, two arguments are passed in. The first is the value that could be NULL. The second is the value to substitute it with if the value is NULL. Thus, for Oracle, it would be...
SELECT (SUBTOTAL + NVL(DISCOUNT, 0)) AS NETT FROM TABLE_SALES

This is nice and neat, but we can do better. The problem here is portability. If you had to move your data from Oracle to MySQL, for example, you would have to change all instances of NVL() to ISNULL().

The COALESCE() function

COALESCE() is a function that exists in all of the databases mentioned above. How does it work?

Well, you slip in any number of arguments to the COALSECE() function call, and the function will return the first non-NULL value. Thus...
COALSECE(NULL, NULL, 0.5, 1, NULL, 8)

.... will return this.
0.5

So if we did this...
SELECT (SUBTOTAL + COALESCE(DISCOUNT, 0)) AS NETT FROM TABLE_SALES

...it would return this. And you would be able to use that same function anywhere!
NETT
20
10
23.5
30
37

Finally...

Handling NULL values is important. Whether you choose to handle them at the data entry level (not allowing NULL values in a column) or in a calculation (using the COALESCE() function), at some point you have to handle them. I hope this helped!
NULL and forever,
T___T

Saturday, 21 November 2020

Spot The Bug: Whiling Away Your Data

They're bad, they're mind-boggling, they're bugs! And we're here today to catch another one of those sneaky little buggers in the latest installment of Spot The Bug.

I know you're
out there, bugs.


So lately, I've been fooling around with WordPress. Don't ask me why; that's neither here nor there for today. Also, this particular bug had nothing to do with WordPress and everything to do with PHP. You'll see why in a bit.

So I had set up a dummy WordPress site in a Mac environment with a whole bunch of Lorem Ipsum. I was poking around WordPress's database schema using MySQL and wrote a little bit of PHP code in my local environment to list out the titles of the wp_posts table.
<?php
    $conn = new mysqli("127.0.0.1", "root", "", "wptest");

    if ($conn->connect_error)
    {
        die("DB connection failed.");
    }

    $sql = "SELECT * FROM wp_posts WHERE post_type='post' AND post_status='publish' ORDER BY post_date DESC";
    $result = $conn->query($sql);

    if (mysqli_num_rows($result) === 0) die("No posts!");
    
    while($row = $result->fetch_assoc())
    {    
        echo "<h1>" . $row["post_title"] . "</h1>";            
    }            

    $conn->close();
?>


So it was going swimmingly so far.

And then next I decided to list out the comments on each post, from the wp_comments table. A little nested While loop should do the trick.
<?php
    $conn = new mysqli("127.0.0.1", "root", "", "wptest");

    if ($conn->connect_error)
    {
        die("DB connection failed.");
    }

    $sql = "SELECT * FROM wp_posts WHERE post_type='post' AND post_status='publish' ORDER BY post_date DESC";
    $result = $conn->query($sql);

    if (mysqli_num_rows($result) === 0) die("No posts!");
    
    while($row = $result->fetch_assoc())
    {    
        echo "<h1>" . $row["post_title"] . "</h1>";

        $sql = "SELECT * FROM wp_comments WHERE comment_post_id = " . $row["ID"];

        $result = $conn->query($sql);

        if (mysqli_num_rows($result) === 0) break;

        while($row_comments = $result->fetch_assoc())
        {
            echo "<p>" . $row_comments["comment_content"] . "</p>";
        } 
           
    }            

    $conn->close();
?>


And this happened! The first post title and its comments were laid out. No errors were seen, but the rest of the content disappeared!

What went wrong

As it turned out, it was me who was being an idiot. I'd appended "_comments" to the name of the associative array, row, to avoid confusing the program, right? Well, I forgot to do the same for result.
$result = $conn->query($sql);

if (mysqli_num_rows($result) === 0) break;

while($row_comments = $result->fetch_assoc())
{
    echo "<p>" . $row_comments["comment_content"] . "</p>";
}


Why it went wrong

So if result, within the very first iteration of the outer While loop, became the object that contained the dataset of the comments of the first post, then quite naturally there would be no next post to fetch!
while($row = $result->fetch_assoc())
{    
    echo "<h1>" . $row["post_title"] . "</h1>";

    $sql = "SELECT * FROM wp_comments WHERE comment_post_id = " . $row["ID"];

    $result = $conn->query($sql);

    if (mysqli_num_rows($result) === 0) break;

    while($row_comments = $result->fetch_assoc())
    {
        echo "<p>" . $row_comments["comment_content"] . "</p>";
    }            
}


How I fixed it

Quite easily done! Just add "_comments" to the name of result!
<?php
    $conn = new mysqli("127.0.0.1", "root", "", "wptest");

    if ($conn->connect_error)
    {
        die("DB connection failed.");
    }

    $sql = "SELECT * FROM wp_posts WHERE post_type='post' AND post_status='publish' ORDER BY post_date DESC";
    $result = $conn->query($sql);

    if (mysqli_num_rows($result) === 0) die("No posts!");
    
    while($row = $result->fetch_assoc())
    {    
        echo "<h1>" . $row["post_title"] . "</h1>";

        $sql = "SELECT * FROM wp_comments WHERE comment_post_id = " . $row["ID"];

        $result_comments = $conn->query($sql);

        if (mysqli_num_rows($result_comments) === 0) break;

        while($row_comments = $result_comments->fetch_assoc())
        {
            echo "<p>" . $row_comments["comment_content"] . "</p>";
        }            
    }            

    $conn->close();
?>


And there you go. That beautiful, beautiful Lorem Ipsum.


Moral of the story

In nested loops of any kind, repeating the declaration of a variable that was used in the outer loop is a very bad idea. Even if nothing untoward occurs now, it's an accident waiting to happen.

Thanks for reading. _comments welcome!
T___T

Saturday, 8 August 2020

Ten Problematic Tech Terms

The tech world has been a-buzz recently. Following the Black Lives Matter riots across the USA, some tech firms have declared their intention to help eradicate racism - by erasing problematic language from their code bases. The overall objective is to be inclusive and avoid insensitive references.

How this is going to help exactly, remains a mystery to many of us. It's tempting to simply dismiss all this as just another poorly-disguised attempt at virtue-signalling. But in the spirit of joining in the fun, let's take a look at some of the terms slated for erasure and their proposed replacements. And some terms that haven't yet had the dubious honor.

1. Master/Slave

This term is used in tech to describe situations where one process or entity controls another, or where one is an original and the others (the "slaves") take reference from it. Ostensibly, tech such as GitHub, Python and Twitter (and even MySQL!) have decided that they will no longer use the terms "master" or "slave" in their code repositories. Instead, terms such as "main" and "replica" will be used.

No more master-slave relationships!

It's a bit of a stretch of the imagination to equate a "Master" branch in GitHub with slavery, but what do I know, right? I'm not a marginalized race in the US. Hell, I don't even live in the US!

2. Black/White

Google claims that the terms "blacklist" and "blackhat" have negative connotations associated with the color, and this is somehow denigrating African-Americans. Instead, we should be using terms such as "rejectlist" and "allowlist" to replace "blacklist" and "whitelist", respectively.

We can't be blackhats
anymore?

To be fair, "rejectlist" and "allowlist" are a lot more obvious than "blacklist" and "whitelist". It's objectively a good change. I just think the reasons behind it feel kind of forced. It's almost like someone's trying a little too hard not to offend black people.

3. Chief Technical Officer

Hey, how about "Chief" Technical Officer? Or "Chief" anything? Isn't that insulting to Native Americans who actually earned that title? Non-native Americans, you can do better.

So Sioux me!

Native American cultural appropriation is a real thing, yo. Just ask Chris Hemsworth.


4. Ninja

Eradicating the word "ninja" from tech vocabulary will be welcome. That's also cultural appropriation. Imagine how the real ninjas feel, having that term co-opted by a bunch of computer geeks who probably couldn't throw a shuriken worth a damn.

Won't someone please
think of the ninjas?

Also, it's incredibly lame to describe yourself as a "code ninja". Please just fucking stop.

5. Kanban

Another case of cultural appropriation. The Kanban was originally used in manufacturing operations in Japan. Then this got taken to the USA, and some time later, the tech industry took to using this to manage software development processes. We can still use Kanban boards, though maybe we should start calling them something else?

Yeah, call 'em something else!

I seriously doubt the Japanese are anything less than smug that the Kanban got adopted by the Americans. But wait... does this whole movement actually have anything to do with the feelings of other cultures, or is it just an excuse to make tech companies feel all progressive and shit?

6. Sanity Check

The term "sanity check" is usually in the context of testing. However, the word "sanity" might be sensitive to people who suffer from mental illness. I'm no expert here, and far be it for me to sound unsympathetic, but could that be because they suffer from mental illness?

Who're you
calling insane?!

The term "smoke test" has been suggested as a substitute. But it appears that the term already exists, and there's actually a difference. You know what, this is crazy (no pun intended) and I'm just gonna let the experts sort this out.


7. Dummy

A "dummy" anything is usually used in software development as a stand-in for the real thing. A dummy account. A dummy file. Dummy content. Just like crash-test dummies are used in place of real people.

I surrender to the awesome power
of your Wokeness.

But no, what if it triggers people who are, say, not the brightest bulb in the chandelier? We want to be inclusive, right? Twitter has suggested "placeholder", which actually isn't that bad. Unlike the case of "blacklist" and whitelist", however,  the term "dummy" was actually pretty obvious already and I don't think it needed changing.

8. Throttling

In software, "throttling" is the process of regulating the rate of processing, because sometimes you gotta slow stuff down in order for things to go smoothly. After all, resources are limited, and we don't want the system to bite off more than it can chew.

Just choking, folks!

But geez, this term is just so violent. It brings to mind wringing of necks and MMA chokeholds. Why such a hostile term? How about "hugging"?

9. Penetration Testing

This is actually a term used in computer security, to assess the defenses and robustness of any particular system. But it's kind of lewd, isn't it? Penetration?

Such penetrating insight!

Think of all the locker room jokes we computer nerds could make if this term were still in use. How about we just scrap it so people don't feel, y'know,  uncomfortable?

10. Alpha/Beta

Alpha release. Beta release. These terms are commonly used to describe software versions. You know what else they're used to describe? Males.

That's Alpha AF.

Maybe usage of this term encourages toxic masculinity. Should we chance it? I mean, if we're going to deprecate the use of "master" and "slave", surely this is next!

Conclusion

Yes, I'm being really facetious here. But let's be real for a minute.

Naming things is one of the great struggles of software development. Congratulations, we just made it a whole lot harder.

It's not that I think tech terms are set in stone and shouldn't change at all. Obviously, some change is for the better. Even more obviously, it would be better if they were done for the right reasons. If done to improve clarity or sustainability of maintenance; some objectively beneficial metric, yes I'm all for it.

But if it's done just for the sake of appealing to the Social Justice mob, I think it's ill-advised. Because there's no end to this sort of thing. People are always going to be offended by something or other. The world doesn't revolve around the USA and their great struggle with racism and their history as slave-owners, and it's time people realized that.

Now that's a master main stroke!
T___T

Wednesday, 18 September 2019

Spot The Bug: Internet Explorer Strikes Again!

Hey guys, the Spot The Bug wagon just rolled back into town, and it's time to kick some bug butt!

Die, bugs, die!

Cross-browser compatibility can be a nightmare. And every once in a while, I'm forcibly reminded of that fact.

So here I was, using a jQuery AJAX call to populate a table. The endpoint led to a PHP file (named, imaginatively, getdata.php) which was grabbing data from a MYSQL database.

index.html
        <script>
            $(document).ready
            (
                $.ajax
                (
                    {
                        url: "getdata.php?type=quotes",
                        type: "GET",
                        success: function(result)
                        {
                            var data = JSON.parse(result);

                            $(data.quotes).each
                            (
                                function(i, x)
                                {
                                    $("#tblQuotes").append("<tr><td style='text-align:right'><b>" + x.person + "</b></td><td><i>" + x.quote + "</i></td></tr>");                                                      
                                }
                            )
                        }
                    }
                )
            );
        </script>


All was fine and dandy, until I detected a typo around the last row, and fixed it in the database. See that I spelled "Linus" wrongly? It's an easy mistake to make, given that "s" and "x" are just about right nest next to each other and he did create Linux. Anyways...


I ran the code again in Chrome, everything seemed fine. But once I tried to do the same in Internet Explorer, the typo came back!

What went wrong

It certainly wasn't the PHP. I ran the file directly in the browser, and even in Internet Explorer, it produced the correct results. This was what it was sending back to the AJAX call. Or was it?

getdata.php
$result = array("quotes" => $techQuotes);
echo json_encode($result);




It seemed a little suspicious that this was only happening in Internet Explorer. And since the PHP wasn't at fault, the next moving part was the AJAX call. Which meant that it was a front-end problem, which in turn gelled with the fact that it was only happening in Internet Explorer.

Why it went wrong

Apparently, Internet Explorer caches all GET requests by default. Therefore, since the endpoint was the same, it simply recycled the previous data! Nice going, Microsoft!

How I fixed it

I just explicitly set caching to off, right there.

index.html
                $.ajax
                (
                    {
                        url: "getdata.php?type=quotes",
                        type: "GET",
                        cache: false,
                        success: function(result)
                        {
                            var data = JSON.parse(result);

                            $(data.quotes).each
                            (
                                function(i, x)
                                {
                                    $("#tblQuotes").append("<tr><td style='text-align:right'><b>" + x.person + "</b></td><td><i>" + x.quote + "</i></td></tr>");                                                      
                                }
                            )
                        }
                    }
                )


And presto! That was how it looked in both Chrome and Internet Explorer now.

Conclusion

You know in this day and age it's so easy to forget that web developers are at the mercy of the browsers of the end-users, and what an utter pain it is to ensure cross-browser compatibility. You think using jQuery will solve all your problems, and surprise, surprise, it really doesn't. Sometimes it introduces new ones.

Cache you later!
T___T

Saturday, 16 March 2019

Why I don't call myself a Full-stack Developer

During some leisurely exploration of job sites, I came across an interesting trend. It seems there's an increase in companies requiring a "full-stack" web developer. For (gasp!) 3000 to 4000 SGD a month.

I have to wonder - do people even realize what they're asking for? Why "full stack"? The typical answer would be, as long as you've worked on two layers (namely, the front-end and the back-end), you're a full-stack developer. I strenuously disagree.

What does "full-stack" really mean?

In order to explain that, I would first have to begin with explaining the anatomy of a web development process. There are several layers of technology stacked (hence the term "full-stack") on top of each other. These may include, but are by no means limited to:

- Hosting/server/network
- Data Modelling/Systems Analysis
- Back-end programming/APIs
- Front-end design/HTML/CSS/JavaScript
- Marketing/SEO
- Project Management

Or to make things even simpler, what does the "stack" in "full-stack" actually refer to? Here's an example. Ever heard of LAMP?

No, not that lamp.

LAMP is a technological stack. It's an acronym comprising of all the technologies that make up its layers.

L is for Linux, which is the operating system.
A is for Apache, which is the server.
M is for MySQL, which is the database.
P is for PHP, which is the application scripting language.

And bear in mind that this is a much simpler example than the first. For this simplified example, a full-stack dev would need to be intimately familiar with only four layers. That's still double the number of layers most laypeople associate the term "full-stack" with.

A full-stack web developer has a good understanding of each layer and has attained a respectable degree of proficiency, if not mastery, over them. He has experienced web development in all these areas. He is the total package and can code your entire web application for you. All by his lonesome.

So, the question that remains in my mind is...

...why would such an awesomely multi-talented and experienced web wizard want to work for you for 3000 to 4000 SGD a month?

Oops, was I perhaps a little too honest there? But seriously, do these guys know what they're asking for when they say they want a "full-stack" web developer?

Football analogy

For those of us who watch football, here's an analogy.

The modern game has evolved. In the past, there was a goalkeeper, defenders, midfielders and strikers. Each specialized in their own area. Now, strikers have to backtrack to help out in defence. Defenders are expected to make the occasional foray up front. And midfielders - sorry guys - have to be everywhere. They have to provide assists, take shots, make tackles, link up with both defence and attack... yep, a lot of cross-training involved.

Let's talk football.

However, everyone is still a specialist. They may excel in more than one area, they may have to take on multiple roles on the pitch, but rare is the footballer who can be deployed anywhere on the field without thoroughly fucking up your game plan. No manager in his right mind is going to place Lionel Messi in defence, Sergio Ramos as central striker, or David Villa in goal.

Did that make sense, or did I just convince you that I know as little about football as I do about web development?

That said, David Villa could still make it as a goalkeeper. If the team in question was your typical High School first eleven. Because the bar for that would be significantly lower.

In other words, if you just need a generalist web developer for a relatively simple set-up to take care of all layers of the development process, fair enough. And if a small company or startup thinks they want to cut headcount by hiring a jack-of-all-trades, hey, I'm not about to judge.

But in this day and age, each layer of technology in the web development stack has grown. And is still growing. Is it realistic to expect anyone to be "full stack"? Web developers need to cross-train in several disciplines. They can't just get by with knowing only one layer of the entire process. At the very least, an understanding of how the current layer you are working with interacts with the next adjacent layer, or layers, is required.

So if a hiring company just wants cheap labor to take care of everything, here's a startling concept - just be honest about it. Stop abusing the term "full-stack".

Here's your stack, right there.

Because when we say "full-stack", we're talking about someone who has gone elbows-deep into each layer of the web development process. He's not just gotten his feet wet. Not just someone who has an interest in every layer and has poked around a bit, and knows how to throw fancy words like "Agile", "Scrum", "methodology" (and "full-stack", for that matter) around to sound like he knows his shit.

Speaking of which...

There are plenty of web developers who claim to be "full-stack" because they've done both front-end and back-end work. By that measure, almost all web developers are "full-stack". Have you ever met a front-end specialist who hasn't had to write back-end code and SQL queries? Do you know any back-end engineers who can't write a single line of HTML? I seriously doubt it.

Stop doing that. It's embarrassing to watch. We get it, you want to sound special. Stand out from the crowd.

But "has done both front-end and back-end work" is a requirement set by recruiters whose job is to sell candidates to companies, and claiming that their candidate is "full-stack" brings up the perceived value. I'm not going to be so crass as to claim that every recruiter knows diddly-squat, but more often than not, they are laypeople in the tech sector. Those are their standards. As a developer, you should be holding yourself to higher standards.

Calling someone like me a "full-stack developer" just because I've been employed by companies that made me do everything in the web development assembly line, is an insult to the real experts.

See you layer, alligator.
T___T

Sunday, 2 December 2018

Insertion the MySQL way

Today I want to talk about SQL's INSERT statement. For that, I'll be using this sample table.

CREATE TABLE TestTable (
    id int,
    name varchar(100),  
    address varchar(200),
    occupation varchar(100),
    gender varchar(1),
    rating decimal
);


There are a few very basic ways to use an INSERT statement, the most common of which would be:
INSERT INTO TestTable (id, name, address, occupation, gender, rating)
VALUES
(1, "Brews Lee", "11 Whompoa Lane", "Tea Merchant", "M", 0.5)

and the multiple insert version:
INSERT INTO TestTable (id, name, address, occupation, gender, rating)
VALUES
(1, "Brews Lee", "11 Whompoa Lane", "Tea Merchant", "M", 0.5),
(2, "Speedy Lee", "11 Whompoa Lane", "Football Coach", "M", 0.6),
(3, "Tan Jin Koo", "105 West Coast Drive", "Waiter", "M", 0.2),
(4, "Lara Bte Suparman", "65 Serangoon Avenue", "Cosplayer", "F", 2.5),
(5, "Jusdeep Singh", "7 Whitley Road", "Philosopher", "F", 1.5)

This is probably the safest and most generic way to insert a record into a table. You do it this way, it's almost impossible to screw up. Of course, it's a little verbose and in the case of tables with a lot of fields, specifying a value for each field can be a royal pain in the behind because you'll have to match the list of values against the list of parameters.

There's a shorter version of this.
INSERT INTO TestTable
VALUES
(1, "Brews Lee", "11 Whompoa Lane", "Tea Merchant", "M", 0.5)

Now this I don't recommend because while it takes less typing, it's way less robust than the first example.

Firstly, this means you'd need to specify values for all fields in the table. If you don't, like in the example below...
INSERT INTO TestTable
VALUES
(1, "Brews Lee", "11 Whompoa Lane", 0.5)

...this error occurs.
Column name or number of supplied values does not match table definition.

Secondly, if you altered the table by, for example, adding a field to it, the first example would still be valid (for the most part, depending on what mode you're operating in.) but the second example would cause things to break immediately.

Ditto if you change the order in which the fields appear in the table. Try this - delete the table and recreate it, but swop the positions of the last two fields.
CREATE TABLE TestTable (
    id int,
    name varchar(100),
    address varchar(200),
    occupation varchar(100),
    rating decimal,
    gender varchar(1)
);

Then try this SQL statement again. You'll get an error the system is now expecting a decimal and character value, but getting "M" and "0.5" respectively.
INSERT INTO TestTable (id, name, address, occupation, gender, rating)
VALUES
(1, "Brews Lee", "11 Whompoa Lane", "Tea Merchant", "M", 0.5)

The MySQL-specific version

Now, sometime years back, I discovered another way, quite by accident. I copied and pasted an UPDATE statement like the one below...

UPDATE TestTable SET
name = "Sincere Lee",
occupation = "Politician"
WHERE id = 2


...and modified it into an INSERT statement. This shouldn't have worked, but it did. Beautifully.

INSERT INTO TestTable SET
name = "Sincere Lee",
address = "11 Whompoa Lane",
occupation = "Politician",
gender = "M",
rating = 1.2


Upon further examination, this syntax is particular to MySQL only, and not standard SQL. And I have mixed feelings about this one. It's elegant, more robust than the other examples, and close enough to the UPDATE statement to make it relatable. It, of course, won't do multiple inserts like the second example. But for singular inserts, this is gold. Since you've specified what value goes into which column, there's no confusion even if you get the sequence wrong or miss out a value (provided, of course, the table is designed to allow empty or null values).

The main problem here is that it's non-standard syntax, which gives rise to two problems.

Using this too often will turn it into a habit, which may work against you if you have to work on other databases such as SQL Server or Oracle. It's generally encouraged to use standard SQL as much as possible.

If you should ever need to convert your existing back-end to some solution other than MySQL, this will be a sticking point. But, you know, if you chose MySQL to be your back-end, you probably had a really good reason to. So to commit to your decision, using MySQL-specific features would serve to reinforce that decision. Conversely, if you were just trying stuff out, the more standard the better.

Most Sincere Lee,
T___T

Tuesday, 28 June 2016

Spot The Bug: SQL weirdness

It's time for Spot The Bug again, so get your game face on.

I'm looking at you,
buster.

I was working on a very bare-bones login procedure. Quick, dirty, throw-away code. The front-end was in HTML, back-end in PHP and interfacing with a MySQL database. Upon clicking the Login button, the login.php script would fire off and query the database. The query returns the number of records that match the email address and password. If there is exactly one match, you're logged in. And if not, you have to try again.

Sounds simple enough. I must've done it a million times. But somehow, I just could not get a match. I always got the message "No such user found.". No syntax errors were in my PHP script.

But then I discovered something really strange with the query. This is the original code from login.php.
<?php
include "inc/config.php";

$login = $POST["txtLogin"];
$password = $POST["txtPassword"];

if (db_connect()->connect_errno)
{
    die("Database connection error.");
}
else
{
    $DBConn=db_connect();

    $strsql="SELECT COUNT(user_id) FROM tb_users WHERE user_login LIKE ? AND  user_password=MD5(?)";

    $sqlresult = $DBConn->prepare($strsql);

    if (!$sqlresult)
    {
        die("Database query error.");
    }
    else
    {
        $sqlresult->bind_param("ss",$login,$password);

        $sqlresult->execute();
        $sqlresult->bind_result($num);
        $sqlresult->fetch();

        if ($num==1)
        {
             //Logged in
        }
        else
        {
             die("No such user found.");
        }
    }

    $DBConn->close();
}
?>

When I changed it, like so, I got a match!
<?php
include "inc/config.php";

$login = $POST["txtLogin"];
$password = $POST["txtPassword"];

if (db_connect()->connect_errno)
{
    die("Database connection error.");
}
else
{
    $DBConn=db_connect();

    $strsql="SELECT COUNT(user_id) FROM tb_users WHERE user_login = ? AND  user_password=MD5(?)";

    $sqlresult = $DBConn->prepare($strsql);

    if (!$sqlresult)
    {
        die("Database query error.");
    }
    else
    {
        $sqlresult->bind_param("ss",$login,$password);

        $sqlresult->execute();
        $sqlresult->bind_result($num);
        $sqlresult->fetch();

        if ($num==1)
        {
              //Logged in
        }
        else
        {
              die("No such user found.");
        }
    }

    $DBConn->close();
}
?>

But it is not enough that things work; sometimes it is just as important to know why they are working. A SQL LIKE is supposed to be more flexible than the "=" operator. So why was the "=" operator matching while the LIKE wasn't?

What went wrong

The problem wasn't in my output. It was in my input. Apparently, while entering the email address in my HTML form, I had accidentally included a trailing space. Instead of "teochewthunder@gmail.com", the input was "teochewthunder@gmail.com ". And this had resulted in the SQL query returning a false positive. The Microsoft Knowledge Base provides a decent explanation. Apparently SQL Server suffers from the same quirk as MySQL. (https://support.microsoft.com/en-us/kb/316626).


How I fixed it

The problem arose because, unlike all the other times I had built this seemingly simple module, I had not written a utility to sanitize the input first. Which included eliminating leading and trailing spaces! So after I added this to my PHP script, it ran like a charm. It was a basic sanitization function using PHP's trim() function to remove leading and trailing spaces.
<?php
include "inc/config.php";

$login = sanitize($POST["txtLogin"]);
$password = sanitize($POST["txtPassword"]);

if (db_connect()->connect_errno)
{
    die("Database connection error.");
}
else
{
    $DBConn=db_connect();

    $strsql="SELECT COUNT(user_id) FROM tb_users WHERE user_login LIKE ? AND  user_password=MD5(?)";

    $sqlresult = $DBConn->prepare($strsql);

    if (!$sqlresult)
    {
        die("Database query error.");
    }
    else
    {
        $sqlresult->bind_param("ss",$login,$password);

        $sqlresult->execute();
        $sqlresult->bind_result($num);
        $sqlresult->fetch();

        if ($num==1)
        {
              //Logged in
        }
        else
        {
              die("No such user found.");
        }
    }

    $DBConn->close();
}

function sanitize($input)
{
    $temp=$input;
    $temp=trim($temp," ");

    return $temp;
}

?>


Moral of the story

Two things to take away from this.

1) Always, always sanitize user input. If I, the developer, can make such an elementary mistake on my own bloody form, who knows what the typical clueless end-user is capable of?

2) Beware of the SQL LIKE and "=" operator. They don't always function the way you'd expect.

That was, like, totally bogus, dudes.
T___T