Assignment1

(1)
i=1
Consider the following small data set:
xy 13 -1 -2 24
• Solve for the value of θ1 that minimizes the cost function by substituting the values from the training set into the cost function, setting the derivative equal to zero, and solving for θ1. Show your work.
3
some θ0, θ1 such that J(θ0, θ1) = 0. Which of the statements below must then be true? (Check all that apply.)
A: This is not possible: By the definition of J(θ0, θ1), it is not possible for there to exist θ0 and θ1 so that J(θu,
v
)
2.
∂ ∂v
f
(u,
v)
3.
∂ ∂u
h(u,
v)
4.
∂ ∂v
h(u,
v
)
Partial Derivative Intuition Consider the contour plot of a function f (u, v) in Figure 1. For each of the following partial derivatives, state whether it is positive, negative, or equal to zero. Briefly explain. These questions can be answered from the contour plot without knowing the formula for the function.
Calculating Partial Derivatives Consider the following functions of the variables u, v, and w. Assume the variables x(i), y(i) are constants: they represent numbers that will not change during the execution of a
Figure 2: Contour of the function f (u, v). Given a value for z = f (u, v), lines are drawn for connecting the (u, v) coordinates where that z value occurs. Problem 4 Suppose that for some linear regression problem (say, predicting housing prices as in the lecture), we have some training set, and for our training set we managed to find
θ0 = 0) linear regression model from class, which has the following hypothesis and cost
function: Hypothesis :hθ(x) = θ1x
1 Cost Function :J(θ1) = 2
m
(hθ(x(i)) − y(i))2
B: We can perfectly predict the value of y even for new examples that we have not yet seen. (e.g., we can perfectly predict prices of even new houses that we have not yet seen.)
Problem 3 In Figure 2, the cost function J(θ0, θ1) has been plotted against θ0 and θ1 as shown in “Plot 2”. The contour plot for the same cost function is given in “Plot 1”. Based on the figure, choose the correct options (check all that apply).
A: If we start from point B, gradient descent with a well-chosen learning rate will eventually help us reach at or near point A, as the value of cost function J(θ0, θ1) is maximum at point A.
DS 310 Assignment 1
Please submit your solution to Canvas by 11:59pm, February 10th.
Problem 1 - Partial Derivatives These are designed to test your understanding of basic concepts.
• Check your answer by plugging in the training data from the previous problem into your expression for θ1. You should get the same value for θ1 that you got in that problem.
C: For these values of θ0 and θ1 that satisfy J(θ0, θ1) = 0, we have that hθ(x(i)) = y(i) for every training example (x(i), y(i))
D: For this to be true, we must have θ0 = 0 and θ1 = 0 so that hθ(x) = 0 Problem 5 - Scikit-Learn for Linear Regression We will use the implementation of Sklearn for linear regression. Please download the notebook file “Assignment1 Q5” from Canvas. You can either upload the notebook file to Google Colab to finish this question or run locally. Following the instructions in the notebook and fill in the key code. After that, in the browser, click on “File”, Select “Print”, and choose “Save as PDF”. Submit the pdf along with your solution to other problems to Canvas.
succinct but obfuscates the original variable names. A more explicit way to write the same
thing
is
∂ ∂u
f
(u,
v)|u=a,v=b)
1.
∂ ∂u
f
(−2,
−2)
2.
∂ ∂v
f
(−2,
−2)
3.
∂ ∂u
machine learning algorithm (e.g., the training data).
• f (u, v) = 10u2v3 + 3v2 + 4
• h(u, v) =
m i=1
1 2
(x(i)u
+
y(i)v)2
Write the following partial derivatives:
2
• Now do the same thing, but don’t substitute the values of the training set into the cost function. Instead, leave the x(i) and y(i) variables, take the derivative with respect to θ1, set it equal to zero, and solve for θ1. This will give you a general expression for θ1 in terms of the training data.
f
(3,
−3)
1
Figure 1: Contour of the function f (u, v). Given a value for z = f (u, v), lines are drawn for connecting the (u, v) coordinates where that z value occurs.
B: If we start from point B, gradient descent with a well-chosen learning rate will eventually help us reach at or near point A, as the value of cost function J(θ0, θ1) is minimum at A.