Troubleshooting FTP

Posted by Justin A. Parr on November 29, 2020
Posted in: Rants, Tech Support. Leave a Comment

Let me start off by saying “I HATE FTP!”

It was developed in the early 1970’s, before the Internet was really the Internet, and before the existence of firewalls or any other type of network security.  Over the last nearly-50 years, it has picked up so many band-aids that the entire protocol is now just basically one big kludge.

Despite its shortcomings, and despite the availability of newer and far superior file transfer tools, banks, governments, and anyone else with a mainframe continues to use it.

So, unfortunately, people like you and I get stuck supporting it.

If you’re having an FTP problem, or are just morbidly curious to learn about an antiquated protocol, read on…

Continue Reading

Bad Design: WinZip – An Annoying Pop-Up Ad from an Irrelevant Product

Posted by Justin A. Parr on September 15, 2020
Posted in: Analyses and Responses, Good Design - Bad Design, Rants, Tech Recommendations. 2 comments

Bad Design: WinZip – An Annoying Pop-Up Ad from an Irrelevant Product

 

The company where I am currently employed uses WinZip as its desktop standard for archive software.

This isn’t my preferred option, but whatever.

One minute, I’m cruising around in a spreadsheet, and the next, this pops up:

They company in question has an enterprise license for WinZip, which means that I’m running a fully-licensed copy.

I will cover some of the history of WinZip, and why I think it’s irrelevant, and then I will tell you exactly why I think this pop-up is wrong.  I will also make some recommendations for both software designers and corporate desktop administrators.

 

Some History

…and, “Why I think WinZip is irrelevant”

Back in the DOS days, there were competing lossless file compression standards, and that’s still true today.

In the 1980’s and early 90’s, disk space was always at a premium, whether it was deciding what to delete off your hard drive so that you could install a new program, or how many programs you could back up to a floppy disk.

And, before the internet, you connected to Bulletin Board Systems (BBSs) through a very slow modem to find and download software.  Smaller files meant that a BBS could host more files, and also, that your download time would be significantly faster.

The first commonly-used compression program for microcomputers was simply called “ARC”.  Not only did ARC compress files, but it had the ability to compress multiple files in to a single “archive” file, and later versions could even span multiple floppy disks – handy if you were backing up your hard drive.

Due to its ease of use, single-file archive format, and itself being a small file that was easy to find and download, ARC quickly became standard among BBS operators and users.  Every PC in the 80’s had a “C:\UTILS” folder, and ARC was one of the programs inside of it.

Although ARC was commercial software developed by SEA corporation, it was distributed as “shareware” – a license that allowed both individual users and BBS operators to freely “share” the file by copying it, but requested that users send in some money and obtain a proper license if they decided to keep using it.  Of course, no one did that, and eventually, SEA released the source code for ARC.

A couple of years later, along came Phil Katz, who wrote a much faster and more efficient implementation of ARC, of course called PKARC, and rather than have both compression and extraction functions within the same executable, Phil Katz split the extraction function in to a separate program called PKXARC.  This meant that you could download the much more svelt PKXARC if all you needed to do was extract files without compressing them, but in either case, you would benefit from the much faster code.

PKARC soon became the de facto tool for using ARC files.

Later, Phil Katz co-developed the ZIP file format – the same one that’s in use today.  And, like ARC, everyone started using ZIP files, especially popular with BBS operators due to the smaller file sizes and faster extraction routines.

Soon, like ARC, every PC had PKZIP and PKUNZIP in their C:\UTILS folder alongside PKARC and PKXARC. Eventually, people just stopped using ARC altogether.

And thus, Phil Katz built the bulk of the PKWare software empire off of one little shareware utility.

However, PKZIP had one major drawback:  It was a command-line program, requiring some knowledge of what a command line is, how to use it, and how to use PKZIP within it.

As operating systems evolved toward Graphical User Interfaces (GUIs) and began to focus on ease-of-use, there was clearly a gap.

You could download a ZIP file from a BBS using your terminal program running in a window, but then you would have to shell out to a command prompt and run some arcane (pun intended) commands to extract the files within it, in to a usable format.

In the early 90’s WinZip emerged as a graphical front-end to PKZip.  With PKZip installed, and with WinZip properly installed and configured, a user could simply double-click on a zip file to extract its contents, or drag and drop some files to create a new ZIP file.  WinZip handled the user interface, and on the back end, it ran PKZip with the appropriate command-line switches.

Eventually, WinZip adopted a “clean-room” ZIP implementation, and dumped PKZip.  Now, there was no complicated dependencies nor configuration.  If a user needed to open a zip file, they could simply download and run WinZip as a single program.

As Windows 95 boosted the popularity of home computing, WinZip quickly became much more popular than PKZip.  PKWare made a feeble attempt to release a competing GUI product, but it was too-little, too-late.  WinZip had been on the market longer, most users were familiar with it, it had better desktop integration, and it was far easier to use.

By the late 90’s, WinZip was one of the first programs you needed to download after any new operating system was installed.

Similar to PKWare, WinZip had a “trialware” license that allowed you to freely use it for a couple of weeks, and then either delete it or license it.

To help drive revenue, unlicensed copies of WinZIp would pop up a friendly reminder.  Once you entered a license code, the reminder disappeared.  Further, a license was perpetual, and initially allowed for perpetual upgrades.  So for about $10 (if I recall), not only could you license WinZip to get rid of the annoying pop-up, but you could also freely download and upgrade to the latest version every couple of months when it was released.

WinZip prospered as it virtually ruled the market.  Over time they added new features and updated the GUI, but WinZip was still WinZip.

And then Microsoft released Windows XP, which had built-in support for zip files. 

It’s hard to believe that Microsoft turned a blind eye to this lucrative niche for nearly 10 years, but on the other hand, they were probably gun-shy after having recently fought off anti-trust litigation from Netscape.  In versions of Windows prior to XP, if you double-click on a ZIP file without WinZip (or another ZIP handler) installed, you more or less get a message that says “what the heck do you want me to do with this??”

Now, with Windows XP and onward, you could double-click a ZIP file to open it, or right-click some files and send them to a “compressed folder”, which is Microsoft’s codename for a ZIP archive file.

Even worse for WinZip, by the time Microsoft released XP, there were Free and Open Source Software (FOSS) alternatives such as WinRAR and 7-Zip.  Initially, each of these only supported a proprietary archive format (RAR and 7z respectively), but eventually they both ended up supporting multiple archive formats, including ZIP, and even the advanced features of the ZIP format, such as media spanning and AES encryption.

And, WinZip’s market began to dry up.

Casual users didn’t need to download anything – they simply used the operating system’s built-in ZIP file handler.

Power users could download FOSS utilities for free, such as WinRAR and 7-Zip, and simply avoid both the cost of WinZip and its increasingly clingy, laggy user interface and more frequent and annoying beg-ware pop-ups.

As it turns out, switching from a perpetual to an annual license model was a smart move for WinZip.

Rather than die out like ARC and PKZip, there is just enough recurring revenue from corporate licensees to keep it on life support.

So, despite being completely obsolete, WinZip is still running out there, with it’s ever-more kludgy and annoying user interface, limited features, and annoying pop-ups.

And, companies like the one I work for are still running it, because, hey, it works, and hey, they still have a license for it.

 

Why Pop-Up Ads are Bad

<RANT>

In case you don’t remember using a computer in the 2000’s, let me describe what it was like:

  • “Windows has some updates for you!”
  • “Update to the LATEST version of Adobe Flash to view this page!”
  • “Adobe Acrobat needs to be updated.”
  • “Your virus definitions are out of date.”
  • “You’re not running the latest version of WinDVD / WinAMP!  Download it now!”
  • 10 browser pop-ups for spray tanning you should buy online, software that can clean your PC, stock you must buy immediately, adult websites you HAVE to try right now, the truth about “x”, and whatever else you don’t care about.

Play this every couple of hours on endless repeat while you’re trying to edit a document or watch a movie.

PEOPLE.  HATE.  POP-UPS.

People hate pop-ups so much that every modern browser has a built-in pop-up blocker – code that specifically stops a website from opening new browser windows.

People hate pop-ups because they are pre-emptive.  You have to STOP doing what you WERE doing (which was important to YOU), and click “close” on a useless pop-up.  You can’t ignore a pop-up, because it’s right there, blocking whatever you were working on behind it.  It’s even worse when you’re typing, because you have to take your hands off the keyboard, which is both slow and distracting.

People HATE them.

So why have a completely random pop-up inside a fully-licensed product??

If I was running the trialware version (if that even still exists), then a pop-up is justified when I first launch the program.  “Hey… we see that you still haven’t registered…”   OK, fine.

BUT I HAVE registered.  It’s fully-licensed.  And, I DIDN’T EVEN HAVE WINZIP RUNNING!!  So it obviously has hooks in to the scheduler (Microsoft’s biggest architectural mistake, but that’s another story) for the sole purpose of trying to sell me more WinZip crapware, that I can only assume has even MORE pop-ups!

What’s even worse is that I don’t need any of the “new features”, and I don’t want them.  I don’t need some crappy product “running tasks in the background” or “sorting through my photos”.

All I need to be able to do is encrypt a ZIP file and read encrypted ZIP files.

BUT, because I’m on a corporate laptop, rather than downloading a superior product like 7-Zip, which is simple, fast, and free, I’m forced to run WinZip and deal with bulky, slow software and random crappy pop-ups.  What a user experience!

</RANT>

Just let this sink in for a minute… My recent experiences with WinZip have been so frustrating that I just spent 3 hours of my personal time writing this blog post.

 

Tips for Corporate Desktop Admins

Why do corporations still use WinZip?  Probably one of several reasons:

  • You have used it for years.  So what.  The current version is a kludgy piece of crap.  Move on.
  • You already own a license.  This falls under the sunk cost fallacy.  Once an investment significantly diminishes in utility, the best course of action is to STOP SPENDING MONEY ON IT.
  • We have to train our users.  No, you don’t.  The 7-Zip interface is similar to the WinZip classic interface, and anyone who has used a computer within the last decade will figure it out.
  • WinZip is secure.  So are 7-Zip and WinRAR.  But, now that no-one out in “the real world” is running WinZip, finding new vulnerabilities is much more difficult.  Existing WinZip deployments could sit on desktops for years in a vulnerable state.  In contrast, with 7-Zip being one of the most popular pieces of FOSS of all time, there is lots of active development, and any vulnerability would be quickly identified and patched.
  • “Open Source” licensing is questionable.  No, it’s not.  Have your legal department review and approve the GPL and move on.
  • Being open-source, there is little or no support.  IT’S AN ARCHIVE UTILITY.  HOW MUCH SUPPORT DO YOU REQUIRE?  Having said that, there’s probably more support out there for 7-Zip than any Microsoft or IBM product.  (Microsoft is the new IBM, they just don’t know it yet.)
  • It’s difficult to deploy.  You don’t even have to install it.  Just copy the files and run it.
  • It’s expensive.  The “F” in FOSS stands for “Free”.

Tip of the day:  DUMP WINZIP.  STOP WASTING MONEY ON MAINTENANCE AND LICENSING.  MOVE ON TO A SUPERIOR SOFTWARE PRODUCT.

 

Tip for Software Designers

If you are Microsoft:

If someone is making MILLIONS OF DOLLARS selling an add-on to your flagship operating system, hint, you might want to BUY THEM.  Loot the codebase, integrate the functionality, and move on.

If you are a startup company with a niche product:

  • Stay within your lane.  Don’t over-innovate.  Keep the feature set simple.
  • Keep the User Interface as simple as possible.  People are using your product out of necessity, not preference.  They don’t want 15 levels of menus, nor “innovative” user interface elements.  Keep it simple and fast.
  • Don’t get greedy.  You have a niche product, not a “desktop suite”.  Charge users a few bucks for each license, and keep the product as simple as possible.
  • Pivot when needed.  Despite being battered in to the dirt by WinZip, PKWare didn’t go out of business.  Currently, they are out of the desktop archiver market, and their current product offering is focused on security.
  • Sell out at the top, not the bottom.  Because you have a niche software product, someone will eventually come along who sells a better / faster / cheaper product.  Once you begin to lose market share, it’s already too late.  You WILL eventually have to sell – do you want to sell while your company is dominating the market, or as you walk out of bankruptcy court?
  • NO POP-UPS.  Remember your place.  I don’t “WinZip” on my computer daily.  I do a lot of things, and products like WinZip are a very small part of it.  The LESS interaction I have, the better.

Enough ranting, I think my points are clear.

NON-RECURSIVE Algorithm for Enumerating Permutations

Posted by Justin A. Parr on August 29, 2020
Posted in: Good Design - Bad Design, Math and Science. Leave a Comment

If you have a set of things, a permutation is a specific way that you can list those things.  Although algorithms exist to list every permutation, most of them rely on recursion – the calling of a function from within itself – which is horribly inefficient.

If you want a non-recursive algorithm that lists every permutation, keep reading…

Continue Reading

Thanks for Misusing My Personal Data!

Posted by Justin A. Parr on July 22, 2020
Posted in: Analyses and Responses, Good Design - Bad Design, Rants, The Light Side. Leave a Comment

Thanks for Misusing My Personal Data!

15 months ago, I bought a house.

During the course of that transaction, I had to disclose personal information to:

  • The finance company (two, since we dropped the first one)
  • The insurance company
  • The title company

I was required by Federal law to disclose information, including my:

  • Birthday and date
  • Social Security Number
  • Full Name
  • Address

You know, everything you might need to, you know, KNOW in order to steal my identity.

The business purpose for this was ostensibly to:

  • Obtain credit information
  • Review my financial records and assets
  • Report a financial transaction to the IRS

And, it was ostensibly to be used ONLY in the course of doing business.

A year passes…

I get a “Happy Birthday” e-mail from:

  • BOTH finance companies, even though I dropped one of them
  • The insurance company

In addition, I got an actual birthday card in the mail from the finance company that we ended up using.

Two decades ago, I would have thought “how quaint!” and moved on.

However, in the days of identity theft, YOUR BIRTHDAY is a significant piece of non-public personal data that should be closely guarded.

If I had gotten a birthday card at the beginning of the month with a note that says “Hey, happy birthday this month!” We know it’s your birthday, but we respect your privacy, so we’ve stored a generic representation of your personal data rather than your actual birthday.

Completely acceptable.

The reason storing my ACTUAL BIRTHDAY is NOT ACCEPTABLE, is twofold:

  1. YOU HAVE NO REASON TO STORE IT.  Once you’ve pulled my credit, sold me a house, and reported all of this to the government, there is no legitimate ongoing business need to continue to retain that information. 

    If your company stores data for which there is no valid, ongoing business purpose, you’re inviting a data breach.
  2. YOU PROBABLY AREN’T STORING IT SECURELY.  Is my birthday in a spreadsheet, stored on your laptop that you take to your house every night, which someone could steal from your house, or even worse, steal from the back seat of your car when you stop to pick up dinner on the way home? 

    Don’t laugh – I worked for a company where this exact situation happened – a spreadsheet containing personnel records, including social security numbers, was stored un-encrypted on the hard drive of a company-issued laptop that was stolen out of the back of someone’s car while parked in a restaurant parking lot.

    So hopefully not on a laptop, but, pursuant to GLBA or FCRA or HIPAA or a number of other laws, we should hope that my birthday is stored on a server that’s encrypted, logically-secured, physically-secured, logged, monitored, audited, sitting behind a firewall, etc.  More realistically, it’s stored “in the cloud” in your company’s sales system.

In addition to appearing completely unprofessional, the situation gives rise to the following, UNCOMFORTABLE QUESTIONS:

  • What else are you storing without my knowledge and consent?
  • Who do you share it with?
  • Is it all stored by social security number?  I hope not, but that’s how businesses were run 30 years ago.

<RANT>

And…

If WE NEVER DID BUSINESS AT ALL because MY WIFE FIRED YOU, then you have NO LEGITIMATE PURPOSE for storing my data, and ZERO REASONS to send me a birthday e-mail.

You know who you are…

</RANT>

 

Best Practices

Most data breaches result when companies store data that they don’t need, or store improperly, or both.

  • Identify and catalog all systems that store Personally-Identifiable Information (PII) / Non-Public Personal Information (NPPI) / Protected Health Information (PHI) and other sensitive, personal data.
  • Make sure these systems are secured properly – the Federal government provides guidance on securing financial and healthcare data.
  • Audit the data regularly to make sure you are only storing what is needed for legitimate business purposes.  This includes purging old data, as well as ensuring that you are not unnecessarily, permanently storing personal data fields.
    • In most cases, a business is only required to retain business records for 3 to 7 years, depending on the type of business.  If you have data older than that, you need to delete it!
    • If you have data fields that are necessary, say, to perform a credit check, you need to store them temporarily, and then delete them when no longer needed.  Those data fields should live only as long as the transaction, and no longer.  3 months to a year would be more than sufficient.
  • If you want to store demographic information, or, you know, send birthday cards in a quaint attempt to appear personable, then at least use legitimate techniques to anonymize the data.
    • Don’t store the birth year at all (if not needed for demographics)
    • If you DO need demographic information, Round the birth year to a multiple of 5
      • y’=int(y/5)*5
      • if y’=y then y’=y’+5
    • In your CRM system, set everyone’s birthday to the first of the month.  If my birthday is April 22, store 4/1.

Send me a birthday card at the first of the month, and let me know that because you respect my privacy, you DO NOT STORE MY ACTUAL BIRTHDAY.

 

 

SIMulation vs EMulation

Posted by Justin A. Parr on June 4, 2020
Posted in: Math and Science. Leave a Comment

SIMulation vs EMulation

This question came up in the context of another topic, and I immediately envisioned an excellent interview question:

Explain the difference between simulation and emulation

As someone who has written thousands of simulations, this is how I would answer that question…

 

SIMulation

Simulation is a closed system where the state “S” contains all relevant variables and their values, and S(i) describes those values at a specific iteration within the simulation.

S = { V0..Vk }

S(i) = {V0,i; V1,i; .. Vk,i}

Based on the variable values in S(i) and a system of rules A{R0… Rm}, the simulation calculates a new state, S(i+1).

In the simulation, we have a starting state, S(0) and an eventual end-state S(n), which is created after n iterations of the simulation.  We start with S(0) and apply the rules of A{} in order to obtain S(1), and this continues until we reach S(n).

So explicitly, a simulation requires feedback from the current state in order to create a subsequent state, given a specific set of variable values and a specific set of rules that operate on those values.

 

EMulation

Emulation is a simulation whose rules are copied from another system.

Given a set of rules that describe system B: {R0..Rm}, if system A is an emulation of B then it simply contains the same rules:

A = B = {R0..Rm}

Given any state S(i) and a set of rules A{}, if we apply those rules to the current state, we get a new state S(i+1):

S(i+1) = A{} -> S(i)

Given the same state S(i) and another set of rules B{}, if we apply the rules in B{} to the current state S(i), we get S'(i+1), which is an “alternate-reality copy” of S(i+1):

S'(i+1) = B{} -> S(i)

If two different sets of rules A{} and B{} applied to any possible state S(i) produce the same next state, then:

S'(i+1) = S(i+1)

And therefore:

A is an emulation of B or vice-versa.

The important detail is that both systems A and B are equivalent, but they might not be implemented the same way – for example, the rules of B might be implemented in hardware, while A might be a software emulation of B.

 

Comparison

Emulation is a specific simulation that answers the question: “how would some other specific system respond, given a state of S”.

Conversely, in a simulation, you can have a set of independent rules that aren’t tied to any other system – the simulation is simply answering the question: “what would happen, if…”

I’ve seen definitions of both simulation and emulation that tie to copying something from the real world, but I disagree.   For example, I could easily write a simulation of a 4D solid traversing 3D space, which is a completely hypothetical model that has no tie to the real world.

Likewise, if you copy the rules of some arbitrary simulation over to another system, then you’ve emulated the original simulation, with no ties to the real world – for example, you could emulate our clearly-contrived 4D solid simulation by copying its rules from a state diagram or a set of software commands to some other equivalent logical system – for example, the logical rules within a game simulation.

Which brings us to a final point:  Simulations (and by extension, emulations) can contain other simulations.  For example, you could have a simulation of a digital pet within a simulation of a digital world.  The state logic and iteration of the digital pet could be driven inside the world simulation, or it could be driven externally and simply rendered based on its current state.  If the digital pet is driven inside the world simulation, it could be driven within the code of the simulation (created at compile time), or it could be implemented as part of the logical rules of the world – for example, as a script or state machine that’s created using in-world tools during runtime.

 

Why the Sun does NOT track East-West

Posted by Justin A. Parr on May 19, 2020
Posted in: Math and Science. Leave a Comment

Why the Sun does NOT track East-West

(Most of the time)

 

We were brought up learning “the sun rises in the east, and sets in the west”, which is absolutely true.

Anyone with experience in the outdoors, including scouting, will probably have been told:

In the morning, follow your shadow to travel west.

In the afternoon, follow your shadow to travel east.

And… moss grows on the north sides of trees, etc…

 

HOWEVER….

That’s not always EXACTLY true…

 

About a year ago, I had a discussion about a road that runs north-south, and why the sun doesn’t track east-west.

It went like this:

Him: East is THAT way (points southwest)

Me: Um… No.  The road there runs north-south, so east is perpendicular to that road.

Him:  Well, I was always taught that the sun points east-west.

Me: That’s not exactly true, because the Earth is tilted on its access, causing certain parts of the Earth’s surface to be closer to the sun…

Him: I don’t believe all that…

Me:  It’s spring.  Go stand at the fence pole and walk from the fence to the house, following the sun’s path.  Then, do it again, same time of day, 6 months from now.

(To this day, we disagree on due east)

 

When I instinctively mentioned the Earth’s orbit, and the 23 degree tilt of Earth’s axis, I knew that to be correct, but I ended up dwelling on the actual mechanics.

So, here we go…

 

The Earth is tilted on its rotational axis by 23 degrees, relative to its orbital orientation around the sun.

 

As a result, over the course of a year, which is the Earth’s orbital period, that tilt causes the Earth’s orientation relative to its orbital plane to change.

In the winter and summer, if you were to slice through the center of the sun with a big knife, along the axis of the Earth’s rotation, the sun would be sliced vertically – perpendicular to the orbital plane, which is the path that the Earth follows as it orbits the sun.

However, in the spring and fall (and all other times), the Earth is canted with respect to the orbital plane, and thus if you were to slice through the sun’s center with a knife aligned with the Earth’s rotational axis, the sun would be cut at an angle with respect to the orbital plane.

 

In the summer, the Earth’s north pole is canted toward the sun.  From the sun’s perspective, its track is perfectly parallel to the Earth’s equator, which is perpendicular to the Earth’s rotational axis.

 

Likewise, in the winter, although the Earth’s north pole is canted away from the sun, the sun’s track is still parallel to the Earth’s equator.

 

In the spring and fall, the plane of the Earth’s rotational axis is tilted from the sun’s perspective.

 

In the spring, from the sun’s perspective, the Earth appears tilted to the right, causing the sun’s track to pass along a path that’s rotated counter-clockwise relative to the Earth’s equator and rotational axis.

 

Likewise, in the fall, the Earth is tilted in the opposite direction due to its orbit, and the sun’s track appears rotated clockwise relative to the Earth’s equator and rotational axis.

 

 

Thus, in winter or summer, the sun’s track follows a true east-west path.  At every other time, the Earth appears slightly tilted to the sun, causing the sun’s track to follow a northeast-southwest path in the spring, or a southwest-northeast path in the fall.

 

So, to all of you scouts out there…

  • In the winter or summer, your shadow points east-west.
  • In the fall, put the sun over your right shoulder to follow an east-west path.
  • Likewise, in the spring, put the sun over your left shoulder to follow an east-west path
  • When a map shows that a road runs north-south or east-west, it runs north-south or east-west, regardless of where the sun points.