梯度下降法解决线性模型

1，梯度下降法

给定一个数据集，x_data、y_data。寻找y=wx模型的w最优解。 代码如下：

import numpy as np
import matplotlib.pyplot as plt

x_data = [1.0, 2.0, 3.0]
y_data = [2.0, 4.0, 6.0]

scope_list = []
w_list = []

w = 60

#学习率
k = 0.01
# 

for i in range(200):
    # 计算cost（loss的和）
    loss_sum = 0
    for x_val, y_val in zip(x_data,y_data):
        loss_sum += 2 * x_val * (w*x_val - y_val)
    cost = loss_sum / 3
    
    # 计算本轮w
    w = w - k * cost
    print(w)
    scope_list.append(i)
    w_list.append(w)

plt.plot(scope_list,w_list)
plt.xlabel("scope")
plt.ylabel("W")
plt.show()

2，batch（随机梯度下降法）

代码如下：

import random
import numpy as np
import matplotlib.pyplot as plt

x_data = [1.0, 2.0, 3.0]
y_data = [2.0, 4.0, 6.0]

scope_list = []
w_list = []

w = 60

#学习率
k = 0.01

for i in range(200):
    # 计算cost(即随机一个loss当cost用)
    rand = random.randint(0,2)
    cost = 2 * x_data[rand] * (w*x_data[rand] - y_data[rand])
    
    # 计算本轮w
    w = w - k * cost
    print(w)
    scope_list.append(i)
    w_list.append(w)

plt.plot(scope_list,w_list)
plt.xlabel("scope")
plt.ylabel("W")
plt.show()