When most people hear the words "machine learning", they imagine something very complex. They picture supercomputers, giant teams of scientists, and thousands of lines of confusing instructions. But the truth is much simpler than that.
At its heart, machine learning is just about one thing: teaching a computer to learn from examples. Instead of telling the computer exactly what to do in every situation, you show it a lot of data and let it figure out the patterns by itself.
In this blog, we are going to look at three of the most important machine learning algorithms. We will not write any code. Instead, we will understand the ideas behind each one, using simple everyday examples. Because before you can build something well, you need to understand what it is supposed to do.
The three algorithms we will explore are linear regression, K-Nearest Neighbours, and decision trees. Each one solves a different kind of problem, and together they cover a huge portion of what machine learning is used for in the real world.
What does "from scratch" really mean?
When developers say they are building an algorithm "from scratch", they mean they are starting with nothing. No shortcuts. No ready-made tools that do the heavy lifting. They are building the logic themselves, step by step, from the very beginning.
This matters because most beginners jump straight into using pre-built tools without understanding what is happening inside. That is fine for getting quick results. But if you want to truly understand machine learning, if you want to fix problems, improve your results, or build something new, you need to understand the foundation.
Think of it like cooking. You can order food from a restaurant every day and eat well. But if you learn to cook from scratch, you understand the ingredients, the heat, the timing, and the technique. Suddenly you can make anything, not just what is on the menu.
Building from scratch gives you that kind of understanding. And the good news is, the core ideas are not as complicated as they look.
Algorithm 1: Linear regression
Linear regression is one of the oldest ideas in all of statistics and machine learning. It answers a very simple question: if I know one thing, can I predict another?
Here is a real-world example. Imagine you are trying to predict the price of a house. You know the size of the house in square feet. The bigger the house, the higher the price, right? That relationship, between size and price, is exactly what linear regression tries to capture.
The algorithm looks at many past examples of house sizes and their prices. It then draws an imaginary straight line through all those data points, a line that fits them as closely as possible. Once that line exists, you can use it to predict the price of any new house just by looking at where its size falls on the line.
Think of it this way: if you plotted every house as a dot on a graph, with size on one side and price on the other, linear regression finds the one straight line that runs closest to all those dots at once.
The word "linear" simply means straight. The word "regression" comes from an old statistical idea about predicting values. Together, they just mean draw the best straight line to make predictions.
Where does this get used in real life? Everywhere. Businesses use it to predict sales based on spending. Doctors use it to estimate a patient's risk based on age and weight. Banks use it to predict loan repayment based on income. Any time you are trying to predict a number based on another number, linear regression is usually one of the first tools people reach for.
Its biggest strength is that it is simple and easy to explain. Its biggest limitation is that it only works well when the relationship between two things actually follows a straight line. Real life is often more complicated than that, which is why other algorithms exist.
Algorithm 2: K-Nearest Neighbors
K-Nearest Neighbours, often written as KNN, is one of the most intuitive ideas in all of machine learning. The logic behind it is something every human being already does naturally every single day.
Here is the idea. Imagine you move to a new city, and you want to know whether a particular neighbourhood is a good place to live. What do you do? You ask the people who already live nearby. If the five families closest to your potential new home all say it is a great neighbourhood, you trust that. If four out of five say it is not so good, you listen to that too. You are using the opinion of your nearest neighbours to make a decision.
KNN works exactly the same way. When you give it something new to classify, it looks at the data points that are closest to that new thing and takes a vote. Whatever the majority of those nearest neighbours say, that becomes the answer.
The "K" in K-Nearest Neighbours simply refers to how many neighbours you ask. If K is 3, you ask the 3 closest ones. If K is 10, you ask the 10 closest. Choosing the right K is one of the most important decisions when using this algorithm.
But how does a computer measure "closeness"? It uses distance. Just like measuring the straight-line distance between two points on a map, the algorithm calculates how far apart two data points are, based on their features. The ones with the smallest distance are the nearest neighbours.
KNN is used in many real-world applications. It helps recommendation systems suggest movies or products you might like based on people who have similar tastes to yours. It is used in medical diagnosis, where a patient's test results are compared to past patients to find the closest matches. It is even used in image recognition.
One thing to be careful about with KNN: if K is too small, one unusual data point can throw off your whole prediction. If K is too large, you start pulling in neighbours that are not really relevant. Finding the right balance is key.
Algorithm 3: Decision trees
A decision tree is perhaps the easiest algorithm to understand, because it works exactly the way human decision-making works. It asks a series of yes or no questions. Based on the answer to each question, it goes down a different path. At the end of the path, it gives an answer.
Think about how a doctor diagnoses a patient. They might start by asking, 'Do you have a fever?' If yes, they ask, 'Do you also have a cough?' If yes again, they ask, 'Have you been around anyone who was sick recently?' Each answer leads to a new question and eventually to a conclusion. That is a decision tree.
In machine learning, the computer builds this tree automatically by looking at training data. It tries to figure out which questions to ask and in what order to get to the right answer as quickly and accurately as possible.
The key idea behind a good decision tree is this: each question should split the data into groups that are as pure as possible. A "pure" group means everyone in that group has the same answer. The algorithm is always trying to create groups where all the members agree.
The way it measures purity is through something called Gini impurity. You do not need to memorise that term. Just understand the idea: a group where everyone agrees has zero impurity. A group that is perfectly split between two answers has the highest impurity. The algorithm picks the questions that reduce impurity the most, step by step.
Decision trees are used in an enormous number of fields. Banks use them to decide whether to approve a loan. Doctors use them for diagnosis. Businesses use them to figure out which customers are likely to leave. Online platforms use them to detect spam or fake accounts.
One thing to watch out for: a decision tree that is allowed to grow too deep will start memorising the training data instead of learning from it. It becomes so specific to the examples it has seen that it cannot handle new, slightly different situations. This is called overfitting. Keeping the tree at a reasonable depth helps prevent this.
How these three algorithms connect
These three algorithms might seem very different, but they all share the same fundamental goal: learn from past data and use that learning to make predictions about new data.
Linear regression does this for numbers. It predicts a quantity, like a price or a temperature. K-Nearest Neighbours does this for categories. It predicts which group something belongs to, based on what its neighbours look like. Decision trees do this through rules. They predict an answer by working through a chain of questions.
Together, they represent three completely different ways of thinking about the same problem. And understanding all three gives you a much broader and stronger foundation than knowing just one.
Common mistakes beginners make
The first mistake is skipping the understanding and jumping straight to tools. Tools are useful, but they are only as powerful as the person using them. If you do not understand what the algorithm is doing, you will not know when it is going wrong or how to fix it.
The second mistake is ignoring the data. Every algorithm is only as good as the data it learns from. If the data has errors, missing values, or hidden biases, the algorithm will learn those problems too. Always spend time understanding and cleaning your data before trusting the results.
The third mistake is expecting one algorithm to work for every problem. Linear regression is great for some problems and completely wrong for others. The same is true for KNN and decision trees. Part of learning machine learning is developing the judgement to know which tool fits which problem.
What to learn next
Now that you understand the ideas behind these three algorithms, you are ready to go deeper. The natural next step is to see how these concepts look when written as actual instructions for a computer. Even if you are not a programmer, reading simple code with a clear explanation beside it can be a powerful way to lock in the understanding.
After that, look into more advanced algorithms like random forests, which are just many decision trees working together, and support vector machines, which find boundaries between groups in clever ways. These build directly on the ideas you have already learnt here.
You can also explore free platforms like Google's Machine Learning Crash Course or fast.ai, both of which are designed to teach machine learning to people without a strong technical background. They take you from understanding to doing in a very approachable way.
The most important thing is to keep going. Machine learning rewards people who stay curious and keep asking, "But how does it actually work?"
Final thoughts
Machine learning is not magic. It is not something reserved for genius-level mathematicians or people who have been coding for twenty years. It is a set of ideas, each one logical and understandable, that together create something powerful.
Linear regression finds the best straight line through your data. K-Nearest Neighbours asks what the closest examples have in common. Decision trees ask the right questions in the right order. Three simple ideas. Used correctly, they can predict disease, power recommendations, detect fraud, and much more.
The best time to start understanding these ideas is right now, before you ever write a single line of code. Because once the concept is clear in your mind, everything else, the tools, the libraries, and the techniques, becomes much easier to learn and much easier to remember.
Start with curiosity. Build from there.