A Support Vector Machine (SVM) is a supervised machine learning algorithm primarily used for classification tasks, though it can also handle regression. The core idea of SVM is to find the optimal hyperplane that maximizes the margin between two classes in a dataset. This hyperplane acts as a decision boundary, distinguishing different classes in an N-dimensional space (N being the number of features).

Key to the SVM algorithm are support vectors, which are the data points nearest to the hyperplane. These points are critical in defining the position and orientation of the hyperplane, ensuring the largest possible margin between classes. By focusing on these support vectors, SVMs are robust against outliers and overfitting.

SVMs can handle both linear and non-linear data. For non-linear data, SVM uses a technique called the kernel trick, which transforms the data into a higher-dimensional space where a linear separator can be found. Common kernels include the linear, polynomial, and radial basis function (RBF).

Overall, SVMs are effective in high-dimensional spaces and versatile due to their ability to use different kernel functions. They are widely used in various applications, including image recognition, bioinformatics, and text classification, due to their robustness and accuracy.