1 解決方案

【方案一】

載入模型結構放在全局，即tensorflow會話外層。

'''載入模型結構:最關鍵的一步'''saver = tf.train.Saver()'''建立會話'''with tf.Session() as sess: for i in range(STEPS): '''開始訓練''' _, loss_1, acc, summary = sess.run([train_op_1, train_loss, train_acc, summary_op], feed_dict=feed_dict) '''保存模型''' saver.save(sess, save_path="./model/path", i)

【方案二】

在方案一的基礎上，將模型結構放在圖會話的外部。

'''預測值'''train_logits= network_model.inference(inputs, keep_prob)'''損失值'''train_loss = network_model.losses(train_logits)'''優化'''train_op = network_model.train(train_loss, learning_rate)'''準確率'''train_acc = network_model.evaluation(train_logits, labels)'''模型輸入'''feed_dict = {inputs: x_batch, labels: y_batch, keep_prob: 0.5}'''載入模型結構'''saver = tf.train.Saver()'''建立會話'''with tf.Session() as sess: for i in range(STEPS): '''開始訓練''' _, loss_1, acc, summary = sess.run([train_op_1, train_loss, train_acc, summary_op], feed_dict=feed_dict) '''保存模型''' saver.save(sess, save_path="./model/path", i) 

2 時間測試

通過不同方法測試訓練程序，得到不同的訓練時間，每執行一次訓練都重新載入圖結構，會使每一步的訓練時間逐次增加，如果訓練步數越大，后面訓練速度越來越慢，最終可導致圖爆炸，而終止訓練。

【時間累加】

2019-05-15 10:55:29.009205: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMAstep: 0, time cost: 1.8800880908966064step: 1, time cost: 1.592250108718872step: 2, time cost: 1.553826093673706step: 3, time cost: 1.5687050819396973step: 4, time cost: 1.5777575969696045step: 5, time cost: 1.5908267498016357step: 6, time cost: 1.5989274978637695step: 7, time cost: 1.6078357696533203step: 8, time cost: 1.6087186336517334step: 9, time cost: 1.6123006343841553step: 10, time cost: 1.6320762634277344step: 11, time cost: 1.6317598819732666step: 12, time cost: 1.6570467948913574step: 13, time cost: 1.6584930419921875step: 14, time cost: 1.6765813827514648step: 15, time cost: 1.6751370429992676step: 16, time cost: 1.7304580211639404step: 17, time cost: 1.7583982944488525

【時間均衡】

2019-05-15 13:03:49.394354: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:1 with 7048 MB memory) -> physical GPU (device: 1, name: Tesla P4, pci bus id: 0000:00:0d.0, compute capability: 6.1)step: 0, time cost: 1.9781079292297363loss1:6.78, loss2:5.47, loss3:5.27, loss4:7.31, loss5:5.44, loss6:6.87, loss7: 6.84Total loss: 43.98, accuracy: 0.04, steps: 0, time cost: 1.9781079292297363step: 1, time cost: 0.09688425064086914step: 2, time cost: 0.09693264961242676step: 3, time cost: 0.09671926498413086step: 4, time cost: 0.09688210487365723step: 5, time cost: 0.09646058082580566step: 6, time cost: 0.09669041633605957step: 7, time cost: 0.09666872024536133step: 8, time cost: 0.09651994705200195step: 9, time cost: 0.09705543518066406step: 10, time cost: 0.09690332412719727