Open Boundary
Assume there are 3 lines separating 3 areas:
line1: y=x , (x <= 10)
line2: y=20-x, (x>=10)
line3: x=10, y>=10
For each area, there are some data:
A:
[-3, -2, 0]
[-3, 1, 0]
[0, 40, 0]
[8, 90, 0]
[8, 10, 0]
[3, 40, 0]
B:
[11, 11, 1]
[15, 6, 1]
[13, 10, 1]
[13, 19, 1]
[23, -1, 1]
[25, -2, 1]
C:
[-2, -3, 2]
[3, 2, 2]
[10, 9, 2]
[15, 10, 2 ]
[20, -1, 2]
[25, -10, 2]
Run below code, we expect that [10, 5] will output value 2. Because point (10, 5) belongs to area B, which has value 2.
import numpy as np
import matplotlib.pyplot as plt
from sklearn import linear_model, datasets
X = np.array([
[-3, -2],
[-3, 1],
[0, 4],
[8, 9],
[8, 10],
[3, 4],
[11, 11],
[15, 6],
[13, 10],
[13, 19],
[23, -1],
[25, -2],
[-2, -3],
[3, 2],
[10, 9],
[15, 10],
[20, -1],
[25, -10]
])
Y = np.array([0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2])
logreg = linear_model.LogisticRegression(C=1e5)
# we create an instance of Neighbours Classifier and fit the data.
logreg.fit(X, Y)
print logreg.predict(np.array([[10, 5]]))
Run below code, it outputs the boundary and region:
h = .02 # step size in the mesh
# Plot the decision boundary. For that, we will assign a color to each
# point in the mesh [x_min, x_max]x[y_min, y_max].
x_min, x_max = X[:, 0].min() - .5, X[:, 0].max() + .5
y_min, y_max = X[:, 1].min() - .5, X[:, 1].max() + .5
xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))
Z = logreg.predict(np.c_[xx.ravel(), yy.ravel()])
# Put the result into a color plot
Z = Z.reshape(xx.shape)
plt.figure(1, figsize=(4, 3))
plt.pcolormesh(xx, yy, Z, cmap=plt.cm.Paired)
# Plot also the training points
plt.scatter(X[:, 0], X[:, 1], c=Y, edgecolors='k', cmap=plt.cm.Paired)
plt.xlabel('Sepal length')
plt.ylabel('Sepal width')
plt.xlim(xx.min(), xx.max())
plt.ylim(yy.min(), yy.max())
plt.xticks(())
plt.yticks(())
plt.show()
Enclosed Boundary
Suppose we have A, B parts
A: x^2 + y^2 <=2
B: x^2 + y^2 > 2
To solve this, we can use Polynomial Logistic Regression method. This is similar to Polynomial Linear Regression.
import numpy as np
import matplotlib.pyplot as plt
from sklearn.preprocessing import PolynomialFeatures
from sklearn import linear_model
# Area A: x^2 + y^2 <= 2
# Area B: x^2 + y^2 > 2
X = np.array([
# training set for A
[0, 0],
[1, 0],
[1.5, -0.2],
[1, 1.5],
[-1, -1.5],
[-1.9, 0],
[-0.5, -1],
[0, 1.9],
[0, -1.9],
[-1.9, 0],
[-1.5, 0.5],
# training set for B
[2, 2.5],
[2, 3],
[1, 5],
[1, -5],
[-3, 0],
[-2, -0.1],
[-2, 1],
[0, -2.1],
[0, 2.1],
[-2.1, 0],
[-0.6, -2]
])
poly = PolynomialFeatures(degree=2)
X_trans = poly.fit_transform(X)
Y = np.array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1])
logreg = linear_model.LogisticRegression(C=1e6)
# we create an instance of Neighbours Classifier and fit the data.
logreg.fit(X_trans, Y)
print logreg.coef_
print logreg.intercept_
print logreg.predict(np.array(poly.fit_transform([[0, 1.8]]))) # expect to return 0
print logreg.predict(np.array(poly.fit_transform([[0, -1.8]]))) # expect to return 0
print logreg.predict(np.array(poly.fit_transform([[0, -2.1]]))) # expect to return 1
And the result is:
Accordingly, we can build the hypothesis:
Check my code on github: open boundary, enclosed boundary