线性回归算法

线性回归

1. 代价函数最小化的方法: ● (批量)梯度下降法 ● 正归方程

2. 梯度下降法

先假设一个定点,然后按照一定的步长顺着这个点的梯度进行更新迭代下去,最后可以找到一个局部最优点,使代价函数在这个局部取得最小值

量(vector)

测

价

度

注:

1.是对θi的求偏导

2.批量梯度下降的每一步都用到了所有的训练样本

3.在多维问题中,要保证这些特征值都具有相近的维度,使得梯度下降

算法更快的收敛.

特征缩放公式:

1.除以最大值

2.

3.学习率的选择:

可以绘制迭代次数和代价函数的图表来观测算法在何时趋于收敛通常可以考虑尝试些学习率:α=0.01,0.03,0.1,0.3,1,3,10 规可以一次性求出最优解

①定义训练的参数(学习率训练次数打印步长)

②输入训练集(定义占位符X = tf.placeholder("float")Y = tf.placeholder("float"))

③随机生成w与b(初始化的方式很多种,方式不同可能会影响训练效果)

④创建线性模型(pred = tf.add(tf.multiply(X, W), b))

⑤用均方差计算training cost(cost = tf.reduce_sum(tf.pow(pred-Y,

2))/(2*n_samples))

⑥使用梯度下降进行优化(optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost))

⑦变量初始化与创建图

init = tf.global_variables_initializer()

with tf.Session() as sess:

sess.run(init)

⑧开始训练

Fit所有的训练数据

设定每50次的打印内容

⑨用测试集进行测试

计算testing cost

计算training cost 与testing cost之间的差值并输出

⑩画图

程序:

import tensorflow as tf

import numpy

import matplotlib.pyplot as plt

rng = numpy.random #产生随机数

# Parameters(参数学习率训练次数打印步长)

learning_rate = 0.01

training_epochs = 1000

display_step = 50

# Training Data

train_X = numpy.asarray([3.3,4.4,5.5,6.71,6.93,4.168,9.779,6.182,7.59,2.167,

7.042,10.791,5.313,7.997,5.654,9.27,3.1])

train_Y=

numpy.asarray([1.7,2.76,2.09,3.19,1.694,1.573,3.366,2.596,2.53,1.221,

2.827,

3.465,1.65,2.904,2.42,2.94,1.3])

n_samples = train_X.shape[0]

# tf Graph Input

X = tf.placeholder("float")

Y = tf.placeholder("float")

# Set model weights

W = tf.Variable(rng.randn(), name="weight")

b = tf.Variable(rng.randn(), name="bias")

# Construct a linear model

pred = tf.add(tf.multiply(X, W), b)

# Mean squared error

cost = tf.reduce_sum(tf.pow(pred-Y, 2))/(2*n_samples)

# Gradient descent

optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost) # Initialize the variables (i.e. assign their default value)

init = tf.global_variables_initializer()

# Start training

with tf.Session() as sess:

# Run the initializer

sess.run(init)

# Fit all training data

for epoch in range(training_epochs):

for (x, y) in zip(train_X, train_Y):

sess.run(optimizer, feed_dict={X: x, Y: y})

# Display logs per epoch step

if (epoch+1) % display_step == 0:

c = sess.run(cost, feed_dict={X: train_X, Y:train_Y})

print("Epoch:", '%04d' % (epoch+1), "cost=", "{:.9f}".format(c), \

"W=", sess.run(W), "b=", sess.run(b))

plt.plot(train_X, sess.run(W) * train_X + sess.run(b), label='Fitted line')

print("Optimization Finished!")

training_cost = sess.run(cost, feed_dict={X: train_X, Y: train_Y})

print("Training cost=", training_cost, "W=", sess.run(W), "b=", sess.run(b), '\n')

# Graphic display

plt.plot(train_X, train_Y, 'ro', label='Original data')

plt.plot(train_X, sess.run(W) * train_X + sess.run(b), label='Fitted line')

plt.legend()