Two Sides of the Same Coin: Data Storage and Algorithms

 

Algorithms and Data Structures are a powerful pair within the world of computers. First and foremost, Algorithms: 

What is an Algorithm? 

Most people only ever hear about algorithms from a scientific perspective. Therefore, a lot of people don’t understand what an algorithm is. Despite this, they are actually a general term for methods that complete tasks. From that perspective, everything you do is an algorithm. Cooking food, getting ready for the day, and doing your job at work; everything that requires tasks needs an algorithm to complete them. 

With respect to our discussion, the ability to perform work is meaningless when there is nothing to work on. Therefore, good algorithms are often used in tandem with Data Structures. 

What is a Data Structure?  

Simply put, it is a way of organizing data. Surprisingly, there is a method to how data should be stored, leading to many types of Data Structures: 

  • Arrays/Lists: Stores info contiguously. Data is physically ordered, like ordering data in Position 1, 2, 3, etc.  

  • Stacks: Storage using First-In-First-Out methods. Simply put, only the most recent element can be interacted with. 

  • Trees: The typical File Explorer solution. Data is organized in terms of Parents and Children; with children data nested below parents. 

Variety is the Spice of Life 

With everything previously discussed, it’s easy to say there is an abundance of Algorithms and Data structures. They are often designed based on use cases. Consider the differences between spreadsheet/database software. Both technically do the same thing: store, organize, and read data; but their scope is different. Databases like SQL can manage terabytes of data with ease. However, this requires technical knowledge to build and run; and can become a hassle for smaller datasets. Excel is prebuilt and has an easy to interact with GUI, but has physical limitations limiting storage and read speeds. 

Therefore, picking the correct Algorithm and Data structures comes down to a few things: 

Algorithms... 

  • Time Complexity: Ranks an algorithm based on its wait times. This naturally depends on the amount of data to comb through; however, clever algorithms can often sort/search through data efficiently, often never searching through some altogether. 

  • Space Complexity: Ranks algorithms based on the space they take up in memory. Data calls require memory space, so how an algorithms uses that memory space is important. A fast algorithm that takes up too much memory could halt other processes in the system, rendering it useless. 

Data Structures... 

  • Modification: Data is read and written all the time. This data is physically stored in memory. If modification of data is lazy or underdeveloped; fragmentation or extra, unneeded memory usage can result. 

  • Algorithms: The way data is structured is directly impacted by the algorithms that search through them. An algorithm that works best in an ordered data set could bring catastrophic results in an unordered set. 

So... What is the best Search/Sort/Storage solution? 

Two words: It depends. 

The answer to this question will depend on the answers to the following: 

  • What type of data am I storing? 

  • How much of it need to be stored? 

  • How quickly does this data need to be accessed? 

  • How much memory is available for storage and active reads/writes? 

Return to the Database vs Spreadsheet debate. Spreadsheets like Excel can afford to search each cell individually because data sets are often small. In addition, Excel is not built for (and therefore isn’t expected to perform) live and constant data retrieval. Databases like SQL can abuse search and sort algorithms to store and modify large data sets. The backbones of popular websites can run on a good implementation of SQL. 

 

References 

Database vs Spreadsheet: Full Comparison – 365 Data Science 

https://www.geeksforgeeks.org/dsa/introduction-to-data-structures/ 

https://learn.zybooks.com/zybook/CPT307:_Data_Structures_&_Algorithms_(INE2537A)/chapter/1/section/1 

Comments

Post a Comment

Popular posts from this blog

A Timeline of Hardware History

Image
As mentioned, my reasoning for making this blog is twofold. I wanted an excuse to deep dive into technology concepts. Simultaneously, I used it to persue an interest I have had with almost everything, history. People are only on Earth for a limited time. Knowing this, I have always been extremely invested in things older than me. This pursuit has been successful too as I currently own a Compaq Armada 1750.  All this to say, my choice of “Hardware components of computer systems” and “History and future of computers” is well founded. Not only is it something I am interested in, but it also allows me to cover many important topics within tech.  The Basics | How Hardware Developed The first area to discuss is the fundamentals of tech. This is a broad topic, so we will mainly look at the systems needed to make computers run. In a sense, the history of computers and understanding these fundamentals go hand in hand. Take a major hardware component, the CPU, for example, “CPUs are the...
Read more

Java: A Relic Still in use Today

Background Java was developed and released in 1990 and has come to be one of the most popular programming languages out there. A part of this popularity comes from its effortless compatibility between Operating Systems. Each Operating System is different. At its simplest, different OS’s look  different. Often times, the way you interact with them are different too. E ach OS has a unique style and way of interacting with files , creating a unique end user experience .   At its most complex, there are  differences in fundamental background tasks that allow each Operating System to run at all. These changes add up, resulting in programs requiring special versions for each operating system.   This is why download pages are typically organized by OS.   "Write Once, Run Anywhere"  Java programs are different though.  Usually, the code written for an app is translated into something physical hardware and the OS can understand. This is why different O...
Read more

Navigating a Network

Networking is one of the more difficult areas of IT to learn while starting out. The level of abstraction skyrockets and has caught me off guard many times. In an attempt to learning about networking, I booted up a command prompt and began pinging websites. I live on the east coast of Florida, so these are my results based on that.  Almost poetically, I had an interesting start to this project. Any attempt to ping a website would time out.  Although obvious to some of you already, I expected the regular IPV4 IP address and could not understand why this was pinging something different. I eventually figured it out during the research phase, but was stumped at the beginning. Given this issue, I loaded up a separate computer and swore to return to the issue.   Ping Results  Google     Pings Sent: 4      Pings Received: 4      Response Time Range: 33ms to 44ms      Average Response Time: 39ms      Home ...
Read more