Skip to content

Commit 71d2124

Browse files
MorvanZhouMorvan Zhou
MorvanZhou
authored and
Morvan Zhou
committed
update
1 parent f33cfd5 commit 71d2124

12 files changed

+316
-53
lines changed

‎README.md

+39-34
Original file line numberDiff line numberDiff line change
@@ -18,69 +18,74 @@ In these tutorials, we will build our first Neural Network and try to build some
1818

1919
All methods mentioned below have their video and text tutorial in Chinese. Visit [莫烦 Python](https://morvanzhou.github.io/tutorials/) for more.
2020

21-
<!--
22-
* pyTorch basic
23-
* [torch and numpy](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/201_torch_numpy.py)
24-
* [Variable](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/202_variable.py)
25-
* [Activation](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/203_activation.py)
21+
22+
* Tensorflow basic
23+
* [Session](https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/201_session.py)
24+
* [Placeholder](https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/202_placeholder.py)
25+
* [Variable](https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/203_variable.py)
26+
* [Activation](https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/204_activation.py)
2627
* Build your first network
27-
* [Regression](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/301_regression.py)
28-
* [Classification](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/302_classification.py)
29-
* [An easy way](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/303_build_nn_quickly.py)
30-
* [Save and reload](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/304_save_reload.py)
31-
* [Train on batch](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/305_batch_train.py)
32-
* [Optimizers](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/306_optimizer.py)
28+
* [Regression](https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/301_simple_regression.py)
29+
* [Classification](https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/302_simple_classification.py)
30+
* [Save and reload](https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/303_save_reload.py)
31+
* [Optimizers](https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/304_optimizer.py)
32+
* [Tensorboard](https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/305_tensorboard.py)
3333
* Advanced neural network
34-
* [CNN](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/401_CNN.py)
35-
* [RNN-Classification](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/402_RNN_classifier.py)
36-
* [RNN-Regression](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/403_RNN_regressor.py)
37-
* [AutoEncoder](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/404_autoencoder.py)
38-
* [DQN Reinforcement Learning](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/405_DQN_Reinforcement_learning.py)
34+
* [CNN](https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/401_CNN.py)
35+
* [RNN-Classification](https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/402_RNN_classification.py)
36+
* [RNN-Regression](https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/403_RNN_regression.py)
37+
* [AutoEncoder](https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/404_AutoEncoder.py)
38+
* [DQN Reinforcement Learning](https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/405_DQN_reinforcement_learning.py)
39+
* [GAN (Generative Adversarial Nets)](https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/406_GAN.py) / [Conditional GAN](https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/406_conditional_GAN.py)
40+
3941
* Others (WIP)
40-
* [Why torch dynamic](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/501_why_torch_dynamic_graph.py)
41-
* [Train on GPU](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/502_GPU.py)
42-
* [Dropout](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/503_dropout.py)
43-
* [Batch Normalization](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/504_batch_normalization.py)
42+
* [Dropout](https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/501_dropout.py)
43+
* [Batch Normalization](https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/502_batch_normalization.py)
4444

45-
### [Regression](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/301_regression.py)
45+
### [Regression](https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/301_simple_regression.py)
4646

47-
<a href="https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/301_regression.py">
47+
<a href="https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/301_simple_regression.py">
4848
<img class="course-image" src="https://morvanzhou.github.io/static/results/torch/1-1-2.gif">
4949
</a>
5050

51-
### [Classification](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/302_classification.py)
51+
### [Classification](https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/302_simple_classification.py)
5252

53-
<a href="https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/302_classification.py">
53+
<a href="https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/302_simple_classification.py">
5454
<img class="course-image" src="https://morvanzhou.github.io/static/results/torch/1-1-3.gif">
5555
</a>
5656

57-
### [RNN](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/403_RNN_regressor.py)
57+
### [RNN](https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/403_RNN_regression.py)
5858

59-
<a href="https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/403_RNN_regressor.py">
59+
<a href="https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/403_RNN_regression.py">
6060
<img class="course-image" src="https://morvanzhou.github.io/static/results/torch/4-3-1.gif" >
6161
</a>
6262

63-
### [Autoencoder](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/404_autoencoder.py)
63+
### [Autoencoder](https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/404_AutoEncoder.py)
6464

65-
<a href="https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/403_RNN_regressor.py">
65+
<a href="https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/404_AutoEncoder.py">
6666
<img class="course-image" src="https://morvanzhou.github.io/static/results/torch/4-4-1.gif" >
6767
</a>
6868

69-
<a href="https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/403_RNN_regressor.py">
69+
<a href="https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/404_AutoEncoder.py">
7070
<img class="course-image" src="https://morvanzhou.github.io/static/results/torch/4-4-2.gif" >
7171
</a>
7272

73-
### [Dropout](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/503_dropout.py)
74-
<a href="https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/503_dropout.py">
73+
### [GAN (Generative Adversarial Nets)](https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/406_GAN.py)
74+
<a href="https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/406_GAN.py">
75+
<img class="course-image" src="https://morvanzhou.github.io/static/results/torch/4-6-1.gif" >
76+
</a>
77+
78+
### [Dropout](https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/501_dropout.py)
79+
<a href="https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/501_dropout.py">
7580
<img class="course-image" src="https://morvanzhou.github.io/static/results/torch/5-3-1.gif" >
7681
</a>
7782

78-
### [Batch Normalization](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/504_batch_normalization.py)
79-
<a href="https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/504_batch_normalization.py">
83+
### [Batch Normalization](https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/502_batch_normalization.py)
84+
<a href="https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/502_batch_normalization.py">
8085
<img class="course-image" src="https://morvanzhou.github.io/static/results/torch/5-4-2.gif" >
8186
</a>
8287

83-
-->
88+
8489

8590
# Donation
8691

‎tutorial-contents/session.py renamed to ‎tutorial-contents/201_session.py

File renamed without changes.

‎tutorial-contents/placeholder.py renamed to ‎tutorial-contents/202_placeholder.py

-2
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,6 @@
44
55
Dependencies:
66
tensorflow: 1.1.0
7-
matplotlib
8-
numpy
97
"""
108
import tensorflow as tf
119

‎tutorial-contents/203_variable.py

+20
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
"""
2+
Know more, visit my Python tutorial page: https://morvanzhou.github.io/tutorials/
3+
My Youtube Channel: https://www.youtube.com/user/MorvanZhou
4+
5+
Dependencies:
6+
tensorflow: 1.1.0
7+
"""
8+
import tensorflow as tf
9+
10+
var = tf.Variable(0) # our first variable in the "global_variable" set
11+
12+
add_operation = tf.add(var, 1)
13+
update_operation = tf.assign(var, add_operation)
14+
15+
with tf.Session() as sess:
16+
# once define variables, you have to initialize them by doing this
17+
sess.run(tf.global_variables_initializer())
18+
for _ in range(3):
19+
sess.run(update_operation)
20+
print(sess.run(var))

‎tutorial-contents/204_activation.py

+48
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
"""
2+
Know more, visit my Python tutorial page: https://morvanzhou.github.io/tutorials/
3+
My Youtube Channel: https://www.youtube.com/user/MorvanZhou
4+
5+
Dependencies:
6+
tensorflow: 1.1.0
7+
matplotlib
8+
"""
9+
import tensorflow as tf
10+
import numpy as np
11+
import matplotlib.pyplot as plt
12+
13+
# fake data
14+
x = np.linspace(-5, 5, 200) # x data, shape=(100, 1)
15+
16+
# following are popular activation functions
17+
y_relu = tf.nn.relu(x)
18+
y_sigmoid = tf.nn.sigmoid(x)
19+
y_tanh = tf.nn.tanh(x)
20+
y_softplus = tf.nn.softplus(x)
21+
# y_softmax = tf.nn.softmax(x) softmax is a special kind of activation function, it is about probability
22+
23+
sess = tf.Session()
24+
y_relu, y_sigmoid, y_tanh, y_softplus = sess.run([y_relu, y_sigmoid, y_tanh, y_softplus])
25+
26+
# plt to visualize these activation function
27+
plt.figure(1, figsize=(8, 6))
28+
plt.subplot(221)
29+
plt.plot(x, y_relu, c='red', label='relu')
30+
plt.ylim((-1, 5))
31+
plt.legend(loc='best')
32+
33+
plt.subplot(222)
34+
plt.plot(x, y_sigmoid, c='red', label='sigmoid')
35+
plt.ylim((-0.2, 1.2))
36+
plt.legend(loc='best')
37+
38+
plt.subplot(223)
39+
plt.plot(x, y_tanh, c='red', label='tanh')
40+
plt.ylim((-1.2, 1.2))
41+
plt.legend(loc='best')
42+
43+
plt.subplot(224)
44+
plt.plot(x, y_softplus, c='red', label='softplus')
45+
plt.ylim((-0.2, 6))
46+
plt.legend(loc='best')
47+
48+
plt.show()

‎tutorial-contents/301_simple_regression.py

+2-2
Original file line numberDiff line numberDiff line change
@@ -23,8 +23,8 @@
2323
plt.scatter(x, y)
2424
plt.show()
2525

26-
tf_x = tf.placeholder(tf.float32, x.shape, 'x') # input x
27-
tf_y = tf.placeholder(tf.float32, y.shape, 'y') # input y
26+
tf_x = tf.placeholder(tf.float32, x.shape) # input x
27+
tf_y = tf.placeholder(tf.float32, y.shape) # input y
2828

2929
# neural network layers
3030
l1 = tf.layers.dense(tf_x, 10, tf.nn.relu) # hidden layer

‎tutorial-contents/302_simple_classification.py

+2-2
Original file line numberDiff line numberDiff line change
@@ -27,8 +27,8 @@
2727
plt.scatter(x[:, 0], x[:, 1], c=y, s=100, lw=0, cmap='RdYlGn')
2828
plt.show()
2929

30-
tf_x = tf.placeholder(tf.float32, x.shape, 'x') # input x
31-
tf_y = tf.placeholder(tf.int32, y.shape, 'y') # input y
30+
tf_x = tf.placeholder(tf.float32, x.shape) # input x
31+
tf_y = tf.placeholder(tf.int32, y.shape) # input y
3232

3333
# neural network layers
3434
l1 = tf.layers.dense(tf_x, 10, tf.nn.relu) # hidden layer

‎tutorial-contents/303_save_reload.py

+80
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
"""
2+
Know more, visit my Python tutorial page: https://morvanzhou.github.io/tutorials/
3+
My Youtube Channel: https://www.youtube.com/user/MorvanZhou
4+
5+
Dependencies:
6+
tensorflow: 1.1.0
7+
matplotlib
8+
numpy
9+
"""
10+
import tensorflow as tf
11+
import matplotlib.pyplot as plt
12+
import numpy as np
13+
14+
tf.set_random_seed(1)
15+
np.random.seed(1)
16+
17+
# fake data
18+
x = np.linspace(-1, 1, 100)[:, np.newaxis] # shape (100, 1)
19+
noise = np.random.normal(0, 0.1, size=x.shape)
20+
y = np.power(x, 2) + noise # shape (100, 1) + some noise
21+
22+
23+
def save():
24+
print('This is save')
25+
# build neural network
26+
tf_x = tf.placeholder(tf.float32, x.shape) # input x
27+
tf_y = tf.placeholder(tf.float32, y.shape) # input y
28+
l = tf.layers.dense(tf_x, 10, tf.nn.relu) # hidden layer
29+
o = tf.layers.dense(l, 1) # output layer
30+
loss = tf.losses.mean_squared_error(tf_y, o) # compute cost
31+
train_op = tf.train.GradientDescentOptimizer(learning_rate=0.5).minimize(loss)
32+
33+
sess = tf.Session()
34+
sess.run(tf.global_variables_initializer()) # initialize var in graph
35+
36+
saver = tf.train.Saver() # define a saver for saving and restoring
37+
38+
for step in range(100): # train
39+
sess.run(train_op, {tf_x: x, tf_y: y})
40+
41+
saver.save(sess, 'params', write_meta_graph=False) # meta_graph is not recommended
42+
43+
# plotting
44+
pred, l = sess.run([o, loss], {tf_x: x, tf_y: y})
45+
plt.figure(1, figsize=(10, 5))
46+
plt.subplot(121)
47+
plt.scatter(x, y)
48+
plt.plot(x, pred, 'r-', lw=5)
49+
plt.text(-1, 1.2, 'Save Loss=%.4f' % l, fontdict={'size': 15, 'color': 'red'})
50+
51+
52+
def reload():
53+
print('This is reload')
54+
# build entire net again and restore
55+
tf_x = tf.placeholder(tf.float32, x.shape) # input x
56+
tf_y = tf.placeholder(tf.float32, y.shape) # input y
57+
l_ = tf.layers.dense(tf_x, 10, tf.nn.relu) # hidden layer
58+
o_ = tf.layers.dense(l_, 1) # output layer
59+
loss_ = tf.losses.mean_squared_error(tf_y, o_) # compute cost
60+
61+
sess = tf.Session()
62+
# don't need to initialize variables, just restoring trained variables
63+
saver = tf.train.Saver() # define a saver for saving and restoring
64+
saver.restore(sess, 'params')
65+
66+
# plotting
67+
pred, l = sess.run([o_, loss_], {tf_x: x, tf_y: y})
68+
plt.subplot(122)
69+
plt.scatter(x, y)
70+
plt.plot(x, pred, 'r-', lw=5)
71+
plt.text(-1, 1.2, 'Reload Loss=%.4f' % l, fontdict={'size': 15, 'color': 'red'})
72+
plt.show()
73+
74+
75+
save()
76+
77+
# destroy previous net
78+
tf.reset_default_graph()
79+
80+
reload()

‎tutorial-contents/304_optimizer.py

+69
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
"""
2+
Know more, visit my Python tutorial page: https://morvanzhou.github.io/tutorials/
3+
My Youtube Channel: https://www.youtube.com/user/MorvanZhou
4+
5+
Dependencies:
6+
tensorflow: 1.1.0
7+
matplotlib
8+
numpy
9+
"""
10+
import tensorflow as tf
11+
import matplotlib.pyplot as plt
12+
import numpy as np
13+
14+
tf.set_random_seed(1)
15+
np.random.seed(1)
16+
17+
LR = 0.01
18+
BATCH_SIZE = 32
19+
20+
# fake data
21+
x = np.linspace(-1, 1, 100)[:, np.newaxis] # shape (100, 1)
22+
noise = np.random.normal(0, 0.1, size=x.shape)
23+
y = np.power(x, 2) + noise # shape (100, 1) + some noise
24+
25+
# plot dataset
26+
plt.scatter(x, y)
27+
plt.show()
28+
29+
# default network
30+
class Net:
31+
def __init__(self, opt, **kwargs):
32+
self.x = tf.placeholder(tf.float32, [None, 1])
33+
self.y = tf.placeholder(tf.float32, [None, 1])
34+
l = tf.layers.dense(self.x, 20, tf.nn.relu)
35+
out = tf.layers.dense(l, 1)
36+
self.loss = tf.losses.mean_squared_error(self.y, out)
37+
self.train = opt(LR, **kwargs).minimize(self.loss)
38+
39+
# different nets
40+
net_SGD = Net(tf.train.GradientDescentOptimizer)
41+
net_Momentum = Net(tf.train.MomentumOptimizer, momentum=0.9)
42+
net_RMSprop = Net(tf.train.RMSPropOptimizer)
43+
net_Adam = Net(tf.train.AdamOptimizer)
44+
nets = [net_SGD, net_Momentum, net_RMSprop, net_Adam]
45+
46+
sess = tf.Session()
47+
sess.run(tf.global_variables_initializer())
48+
49+
losses_his = [[], [], [], []] # record loss
50+
51+
# training
52+
for step in range(300): # for each training step
53+
index = np.random.randint(0, x.shape[0], BATCH_SIZE)
54+
b_x = x[index]
55+
b_y = y[index]
56+
57+
for net, l_his in zip(nets, losses_his):
58+
_, l = sess.run([net.train, net.loss], {net.x: b_x, net.y: b_y})
59+
l_his.append(l) # loss recoder
60+
61+
# plot loss history
62+
labels = ['SGD', 'Momentum', 'RMSprop', 'Adam']
63+
for i, l_his in enumerate(losses_his):
64+
plt.plot(l_his, label=labels[i])
65+
plt.legend(loc='best')
66+
plt.xlabel('Steps')
67+
plt.ylabel('Loss')
68+
plt.ylim((0, 0.2))
69+
plt.show()

0 commit comments

Comments
 (0)