Suppose for a moment that you’re a cartoonish caricature of an affluent individual- Say, a certain copyrighted duck.  And being this caricature, rather than keeping your wealth in real estate, investments and the like, you’ve opted for a good old fashioned room full of gold coins. Who among us wouldn’t love to take a physics-defying swim in such a “pool”? Of course, a large concentration of liquid assets like that is an extremely attractive target for those who might feel themselves more worthy of your fortune, regardless of the law. Some manner of protection is obviously needed for this vault of yours.

The obvious solution is a big, heavy door with a state of the art locking mechanism. But, being this affluent caricature, you have business that takes you around the world frequently, leaving your home unoccupied for significant periods of time- such caricatures are far too miserly to employ security staff. In practice, for as long as your home is unattended, that big fancy locking door is little more than a speed bump. Sure, it would take some time, but with you gone for weeks they have all the time they need to research the lock and pick it, or even just force their way through the door with explosives or a cutting torch.

What if, instead, you made the entrance to the vault hidden, perhaps accessed by pulling a few shelves out of the pantry and opening a wall panel behind it. That would-be wealth distributor might now spend weeks combing your home and never even find the vault to get started!

It’s a no-brainer, right? Well, not exactly. In practice, this caricature would at the very least need to employ a contractor to construct this vault and its access point. Bare minimum, even with zero leaks, there are at least two more people who know the secret of your vault. And as Benjamin Franklin once wrote, “Three can keep a secret if two of them are dead.”

This is what we call Security by Obscurity; a strategy that employs secrecy as the primary method to secure an asset- whether this is liquid wealth, documents, or data on a server somewhere. A strategy that has been recognized for hundreds of years now as being fundamentally flawed. If an asset is to have any actual utility, there must be a means of accessing it, and this will fundamentally entail multiple people knowing the secret, one way or another. Each individual who knows this secret is a potential vector for compromise, and as soon as the information is leaked, it is in the hands of people with neither the obligation nor the inclination to protect your assets.

This very topic was heavily discussed in the mid 1800s, specifically on the topic of maintaining secrecy for lock designs.  Alfred Charles Hobbs famously stated, in favor of disclosure: “Rogues are very keen in their profession, and know already much more than we can teach them.”  Essentially, the benefits of sharing the lock designs and getting more -benevolent- eyes looking for flaws significantly outweighed the risk inherent in the potential for the information being leaked to bad actors, as said bad actors happened to specialize in figuring out the workings of locks and circumventing them.

This same concept persists to this day, not only in the realm of physical locks, but in information security. We allow security professionals (like my team) to review our code, and attempt to circumvent our security and access or modify data that is supposed to be protected. We accept a little risk in doing this, as the end result is a system that is extremely difficult for a bad actor to break into. A small reduction in secrecy for a massive increase in security.

And yet, despite the well documented centuries old falliblity of the obscurity approach, many individuals and small businesses continue to rely on it as their sole line of defense, some not even bothering to secure their wifi! Relying on being a small party that no one is likely to target is fundamentally flawed, particularly when mechanisms like malware are considered. Ransomware alone has already cost billions of dollars in damages, without any need for its creators to target any given individual or company specifically.

I’m not saying that obscurity is useless- on the contrary, keeping protected information on a need to know basis is one of the core principles of security. But if used alone, it is essentially useless, as the approach inherently provides a large attack vector in the form of the human element which (as we know) is already the single largest attack vector even with thorough security measures in place. The information leak need not be intentional or even direct; simple human error can result in the leak of a secret, poor security discipline can lead to a malware infection leaking the same, and so on. Secrecy absolutely has its place as part of an overarching security strategy. It is simply not to be used alone.

So by all means, make your vault entrance a hidden door- but still make sure you spring for that locking mechanism too.

We are honored to have these awards

Tech Industry
Disruptive Tech Fast 50 Award
Inc 5000 Award
75 WD

Ever since the first mobile phone with a fingerprint scanner, surprisingly not the iPhone, has been released into the market in 2004 people have had mixed reviews about it. However, now that most smartphones, laptops, and security devices have the feature built-in or have an option for biometric scanners the question now becomes which is safer.

Along with biometric scanning, there is also the option of Multi-Factor Authentication which allows for a user to sign in with a PIN provided by a randomly generated application, in addition to their regular user name and password. Both methods provide an extra layer of protection and security with the applications that are being used. Another form of protection is through a private key. Private keys stay on your device and are never shared with anyone. This key can be used to unlock a device with a local gesture allowing for it to be unique and difficult to replicate.

In terms of securing information the different forms of biometric and MFA, both do the job well, by making it difficult for hackers to be able to gain access into individual users or a companies systems. What is better depends on the situation, since according to Alex Simmons the Corporate VP of Program Management, ‘99.9% of identity attacks have been thwarted by turning on MFA…'(Microsoft Security Passwordless Protection 2021).

A scenario where biometric scanning might be the better option would be for a health care worker or for someone who is constantly on the go and needs access to information without much hassle. In such situations, using a fingerprint scanner would be the most efficient way for more than one person to quickly access the system while maintaining security.

On the other hand, for someone who works in a lab or in an office where they tend to stay at their desk for extended periods of time then for such users there would not be much difference between a biometric scanner versus using a unique gesture or signing through a randomly generated PIN.

Both scenarios allow for the user to be able to access their information quickly and safely with no compromise to security. In addition, users would not have to worry as often about forgetting passwords or changing them every few weeks in fear of potential attacks. There would also be a reduction in the number of phishing attacks, since logging into a device would require the user to physically replicate different forms of biometrics or have the specific device that is used for MFA. These are only a few of the benefits of password-less authentication, and although it might not always be possible to use finger print or retina scanners on all devices, a combination of traditional authentication mixed with forms of MFA can allow for a very secure device.

Resources:

Microsoft Security Passwordless Protection. (2021). Retrieved December 10, 2021, from: https://query.prod.cms.rt.microsoft.com/cms/api/am/binary/RE2KEup.

We are honored to have these awards

Tech Industry
Disruptive Tech Fast 50 Award
Inc 5000 Award
75 WD

Coauthored by Chris “Cool” Hand “Luke” ft. Mathew “Big Money” Sella 

There’s a new and increasingly popular saying in software development that “memory is cheap” and optimizations are deferred or ignored because of the tooling available. Basic software development can get away with some bad habits, but if your system has the possibility of being challenged by volume or scale, you’ll likely wish you had put more thought into it. There’s a strong sense of satisfaction in building a system that can handle a tremendous amount of load. It’s similar to an engineer building a structure that can handle ten times the weight it was originally designed to hold, simply because it was well built. Here we’ll dive into some considerations any team should know of when they want to approach handling big data.  

What is big data? 

In the world of data, scale is relative. Big data to some teams may be hundreds of records, while for others it may be billions or trillions. Depending on the record in question, each of these scenarios can present their own challenges. If the data you handle is consistently growing, even if it isn’t an impressive amount right now, you’ll need to be smart about how you’ll handle it.  

Define Your Process

First, understand your data and process. 

What kind of data are you working with?  

You could be: 

  • Querying a database with millions of rows 
  • Storing thousands of files 
  • Extracting information from thousands of files 
  • Processing a batch request of hundreds of records that each require a multi-step workflow 

Each of these scenarios will have opportunities for optimization, but step one to having a good process up front is to recognize what kind of data you’ll be dealing with.  

After this, you need to understand your process, answer some questions such as: 

  • Is it (can it be) a background job? 
  • How quickly do you want to give a response to the user? 
  • Are you persisting information as a result of this job, or passing it straight through? 

Then, more practically and technically, you think to consider what kind of operations this process will require: 

  • Will your operation require a lot of I/O (input/output) operations? 
  • Are you going to be performing large, time-consuming queries? 
  • Do you have to store a tremendous amount of data as a result of this operation? 
  • Are you going to require storing a lot of information in memory? 

The answer to many of these questions will depend on the approach you take and play a large part in how efficient your process will be. For instance, some core optimization rules would be to limit your interactions with a database, such as inserting many records all at once and establishing a single connection that can be reused. However, this may mean that you increase the amount of information your process must hold in-memory, and these are the types of tradeoffs you’ll have to consider when defining your process. 

Try to define each step of your process and break them into small, discrete operations that can be examined and run independently. Good system design encourages this with any development, but when you’re dealing with heavy loads it becomes even more important. This also gives you the opportunity to have specific tools or optimizations for the steps that may become your bottleneck.  

Pick the right tools 

After having understood your process, it’s time to pick the tools you’ll be using to perform these operations. Be open minded when considering which language, framework, or platform may be the best fit for your operation. Some languages are better suited for specific operations, and even if you segment your codebase a little bit for the sake of being efficient, the tradeoff may be worth it!  

If you find you have a process with multiple steps, consider treating each of them as an opportunity to pick the right tool for the job. If you use something like AWS Lambda, each step of your process can be handled in a different language. You could have Python, Node, Go, and C# in the same workflow. Not necessarily something we would recommend, but you should explore which language fits the step you’re working on!   

Identify your bottlenecks 

Every process will end up having a single step that takes the most time. Often this one step will be the most time-consuming by a factor of 10 or more. This can present an opportunity to make significant progress in improving your workflow. Why spend time optimizing a step that takes seconds, when you have another that takes minutes? Focus on the steps with a potentially high return on investment. 

Another consideration when looking at your bottlenecks is to examine whether there are ways to organize your flow in such a way that some shorter processes can run in conjunction with your most time-consuming step. For instance, if your users are waiting for validation of data, you may be able to immediately respond with the validation of results and start a separate job to process the data.  

Practical Examples 

Choosing the right language 

Softrams just recently set a new benchmark for the number of records it was able to extract from almost 2500 files when it processed 4.26 billion records in a single data extraction. These operations involve a lot of Input/Output operations, and originally was written in NodeJS. Deciding to take advantage of the multi-threading capabilities of GoLang, we were able to reduce the time it took to process a single file from almost 16 minutes a file, to about 6 minutes a file. Over 2500 files, this type of change saved the process over 400 hours of processing time! 

Letting the database do the work 

One of our engineers worked at a company that needed to process between 200,000 and 500,000 records ever hour. Originally this job was written in PHP using a PostgresSQL database, and the process would load chunks of records into memory before processing them. This meant that they could only process a certain number of records at once, and the entire process took almost twenty minutes every hour. One engineer was able to move a core piece of processing logic to the query that fetched these records originally, which substantially reduced the computations that were required in-memory. This reduced the processing time from nearly 20 minutes to less than a minute per job.  

Summary

Memory is cheap, but don’t limit your system by neglecting a sound approach when dealing with large data. Be intentional about your process and set yourself up for success to scale 10x when you need to.

We are honored to have these awards

Tech Industry
Disruptive Tech Fast 50 Award
Inc 5000 Award
75 WD