马春杰杰
[mcj]CNN中卷积核的卷积方式详解|CNN中卷积核是如何进行卷积的？
开局先祭出一张大神图： 比如我们有一张5*5*3的图作为输入，卷积核的尺寸为3*3，则卷积核的输入深度为3，这里的3对应着输入的3通道，卷积核的数量为2，则卷积核的参数应为：(3*3*3)*2。 卷积过程为，首先因为输入图像有3层，所以卷积 […]
[mcj]Keras优化for循环以及Keras节省GPU内存资源禁止Keras预分配GPU内存资源
最近在用Keras跑程序的时候，为了方便，想让它自动的进行循环测试，就用了for循环，结果跑了没几个循环就显示内存已满。
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 |
2019-04-24 13:56:03.386390: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 162 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:02:00.0, compute capability: 6.1) WARNING:tensorflow:From /home/ubuntu/anaconda3/envs/3dlstm/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:1062: calling reduce_prod (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version. Instructions for updating: keep_dims is deprecated, use keepdims instead 2019-04-24 13:56:14.278430: W tensorflow/core/common_runtime/bfc_allocator.cc:267] Allocator (GPU_0_bfc) ran out of memory trying to allocate 128.00MiB. Current allocation summary follows. 2019-04-24 13:56:14.278531: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (256): Total Chunks: 20, Chunks in use: 20. 5.0KiB allocated for chunks. 5.0KiB in use in bin. 584B client-requested in use in bin. 2019-04-24 13:56:14.278560: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (512): Total Chunks: 2, Chunks in use: 2. 1.0KiB allocated for chunks. 1.0KiB in use in bin. 1.0KiB client-requested in use in bin. 2019-04-24 13:56:14.278581: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (1024): Total Chunks: 4, Chunks in use: 4. 4.2KiB allocated for chunks. 4.2KiB in use in bin. 4.0KiB client-requested in use in bin. 2019-04-24 13:56:14.278602: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (2048): Total Chunks: 6, Chunks in use: 6. 12.0KiB allocated for chunks. 12.0KiB in use in bin. 11.9KiB client-requested in use in bin. 2019-04-24 13:56:14.278621: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (4096): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin. 2019-04-24 13:56:14.278640: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (8192): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin. 2019-04-24 13:56:14.278661: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (16384): Total Chunks: 2, Chunks in use: 2. 36.2KiB allocated for chunks. 36.2KiB in use in bin. 36.2KiB client-requested in use in bin. 2019-04-24 13:56:14.278679: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (32768): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin. 2019-04-24 13:56:14.278697: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (65536): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin. 2019-04-24 13:56:14.278715: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (131072): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin. 2019-04-24 13:56:14.278733: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (262144): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin. 2019-04-24 13:56:14.278754: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (524288): Total Chunks: 1, Chunks in use: 1. 975.5KiB allocated for chunks. 975.5KiB in use in bin. 864.0KiB client-requested in use in bin. 2019-04-24 13:56:14.278773: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (1048576): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin. 2019-04-24 13:56:14.278792: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (2097152): Total Chunks: 1, Chunks in use: 0. 3.99MiB allocated for chunks. 0B in use in bin. 0B client-requested in use in bin. 2019-04-24 13:56:14.278812: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (4194304): Total Chunks: 1, Chunks in use: 1. 4.00MiB allocated for chunks. 4.00MiB in use in bin. 3.38MiB client-requested in use in bin. 2019-04-24 13:56:14.278832: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (8388608): Total Chunks: 1, Chunks in use: 1. 8.00MiB allocated for chunks. 8.00MiB in use in bin. 6.75MiB client-requested in use in bin. 2019-04-24 13:56:14.278853: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (16777216): Total Chunks: 2, Chunks in use: 2. 43.00MiB allocated for chunks. 43.00MiB in use in bin. 40.50MiB client-requested in use in bin. 2019-04-24 13:56:14.278874: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (33554432): Total Chunks: 2, Chunks in use: 2. 69.00MiB allocated for chunks. 69.00MiB in use in bin. 54.00MiB client-requested in use in bin. 2019-04-24 13:56:14.278895: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (67108864): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin. 2019-04-24 13:56:14.278913: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (134217728): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin. 2019-04-24 13:56:14.278931: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (268435456): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin. 2019-04-24 13:56:14.278950: I tensorflow/core/common_runtime/bfc_allocator.cc:613] Bin for 128.00MiB was 128.00MiB, Chunk State: 2019-04-24 13:56:14.278970: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f6834000000 of size 28311552 2019-04-24 13:56:14.278986: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f6835b00000 of size 38797312 2019-04-24 13:56:14.279008: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f683de00000 of size 33554432 2019-04-24 13:56:14.279030: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f6850c00000 of size 16777216 2019-04-24 13:56:14.279046: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f6851c00000 of size 8388608 2019-04-24 13:56:14.279063: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f6880200000 of size 1024 2019-04-24 13:56:14.279078: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f6880200400 of size 1024 2019-04-24 13:56:14.279095: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f6880200800 of size 2048 2019-04-24 13:56:14.279110: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f6880201000 of size 2048 2019-04-24 13:56:14.279126: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f6880201800 of size 2048 2019-04-24 13:56:14.279142: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f6880202000 of size 2048 2019-04-24 13:56:14.279158: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Free at 0x7f6880202800 of size 4184064 2019-04-24 13:56:14.279174: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f6890400000 of size 4194304 2019-04-24 13:56:14.279190: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4400000 of size 256 2019-04-24 13:56:14.279206: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4400100 of size 256 2019-04-24 13:56:14.279222: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4400200 of size 256 2019-04-24 13:56:14.279238: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4400300 of size 512 2019-04-24 13:56:14.279253: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4400500 of size 256 2019-04-24 13:56:14.279269: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4400600 of size 256 2019-04-24 13:56:14.279284: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4400700 of size 256 2019-04-24 13:56:14.279300: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4400800 of size 256 2019-04-24 13:56:14.279315: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4400900 of size 1024 2019-04-24 13:56:14.279331: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4400d00 of size 256 2019-04-24 13:56:14.279346: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4400e00 of size 256 2019-04-24 13:56:14.279361: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4400f00 of size 256 2019-04-24 13:56:14.279377: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4401000 of size 256 2019-04-24 13:56:14.279394: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4401100 of size 2048 2019-04-24 13:56:14.279410: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4401900 of size 256 2019-04-24 13:56:14.279425: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4401a00 of size 256 2019-04-24 13:56:14.279440: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4401b00 of size 256 2019-04-24 13:56:14.279456: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4401c00 of size 256 2019-04-24 13:56:14.279472: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4401d00 of size 16384 2019-04-24 13:56:14.279487: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4405d00 of size 256 2019-04-24 13:56:14.279503: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4405e00 of size 256 2019-04-24 13:56:14.279518: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4405f00 of size 2048 2019-04-24 13:56:14.279534: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4406700 of size 256 2019-04-24 13:56:14.279550: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4406800 of size 256 2019-04-24 13:56:14.279566: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4406900 of size 1280 2019-04-24 13:56:14.279582: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4406e00 of size 256 2019-04-24 13:56:14.279598: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a4406f00 of size 20736 2019-04-24 13:56:14.279614: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a440c000 of size 512 2019-04-24 13:56:14.279630: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0x7f68a440c200 of size 998912 2019-04-24 13:56:14.279646: I tensorflow/core/common_runtime/bfc_allocator.cc:638] Summary of in-use Chunks by size: 2019-04-24 13:56:14.279666: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 20 Chunks of size 256 totalling 5.0KiB 2019-04-24 13:56:14.279685: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 2 Chunks of size 512 totalling 1.0KiB 2019-04-24 13:56:14.279702: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 3 Chunks of size 1024 totalling 3.0KiB 2019-04-24 13:56:14.279720: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 1 Chunks of size 1280 totalling 1.2KiB 2019-04-24 13:56:14.279738: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 6 Chunks of size 2048 totalling 12.0KiB 2019-04-24 13:56:14.279756: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 1 Chunks of size 16384 totalling 16.0KiB 2019-04-24 13:56:14.279774: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 1 Chunks of size 20736 totalling 20.2KiB 2019-04-24 13:56:14.279793: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 1 Chunks of size 998912 totalling 975.5KiB 2019-04-24 13:56:14.279811: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 1 Chunks of size 4194304 totalling 4.00MiB 2019-04-24 13:56:14.279828: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 1 Chunks of size 8388608 totalling 8.00MiB 2019-04-24 13:56:14.279846: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 1 Chunks of size 16777216 totalling 16.00MiB 2019-04-24 13:56:14.279864: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 1 Chunks of size 28311552 totalling 27.00MiB 2019-04-24 13:56:14.279882: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 1 Chunks of size 33554432 totalling 32.00MiB 2019-04-24 13:56:14.279900: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 1 Chunks of size 38797312 totalling 37.00MiB 2019-04-24 13:56:14.279934: I tensorflow/core/common_runtime/bfc_allocator.cc:645] Sum Total of in-use chunks: 125.01MiB 2019-04-24 13:56:14.279960: I tensorflow/core/common_runtime/bfc_allocator.cc:647] Stats: Limit: 170393600 InUse: 131082240 MaxInUse: 131082240 NumAllocs: 41 MaxAllocSize: 38797312 2019-04-24 13:56:14.279989: W tensorflow/core/common_runtime/bfc_allocator.cc:271] ******************************************xxxxxxx**********************xxx***********x* 2019-04-24 13:56:14.280057: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at random_op.cc:202 : Resource exhausted: OOM when allocating tensor with shape[8192,4096] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc |
出这个错误的意思其实就是说内存已满，为了解决这个问题 […]
[mcj]使用Matlab/Python合并mat文件并存为mat
在关联问题中，我们讲过如何利用python将mat文件存为excel，但是excel有个问题，那就是最多存储255列数据，当我们数据很多的时候怎么办呢？ 在这里，我选择mat文件。 关联问题： 应用需求： 在这里，我有多个ma […]
[mcj]使用Python读取mat文件并存为excel
关联问题： 因工作需要，需要将.mat文件存到excel中，基本思想就是，先读取mat文件，然后读取mat的索引头，将该索引的数据存为list，然后把该list存到excel中去。 读取mat： [crayon-6864b99ee87385 […]
[mcj]基于Opencv-DNN模块的YOLOv3目标检测并保存视频(C++)|YOLOV3修改检测物体为特定类别如飞机行人
在这篇文章中，我们将学习如何使用YOLOv3（一种最先进的物体探测器）与OpenCV。 YOLOv3是流行的物体检测算法YOLO的最新变种- 你只看一次。已发布的模型可识别图像和视频中的80个不同对象，但最重要的是它具有超快速且几乎与Sin […]
[mcj]ubuntu16.04无法访问网页：ping: unknown host www.baidu.com
这个问题真是来得莫名其妙，突然就不能上网了。打不开任何网页，尝试着ping了一下百度，提示
|
1 2 |
ubuntu@mcj:/root$ ping www.baidu.com ping: unknown host www.baidu.com |
找了很多答案，都不适用于我这个，我这里有两个虚拟网卡eth0和eth1，平时都是用 […]
[mcj]Anaconda安装Opencv之后不能用：error: (-2:Unspecified error) The function is not implemented. Rebuild ..
利用Anaconda创建了很多需要的环境，有些环境需要opencv的支持，不过直接使用conda安装opencv的话，可能会遇到不能使用的尴尬情况~这是因为conda安装的opencv是阉割版的，功能比较少。甚至连imshow都不能用。会出 […]
[mcj]Ubuntu下测试OpenCV是否安装成功|OpenCV使用Python/C++读取并显示图片
C++版： 读取一张图片，并显示：
|
1 2 3 4 5 6 7 8 |
#include<opencv2/opencv.hpp> using namespace cv; int main() { Mat picture = imread("dog.png"); imshow("显示图片", picture); waitKey(0); } |
然后进行编译
|
1 2 |
g++ -c -o cpp.o ./cpp.cpp g++ -std=c++11 -o cpp cpp.o -lopencv_core -lopencv_imgproc -lopencv_imgcodecs -l opencv_highgui |
这里需要注意，有些人可能会报错： [crayon-6864 […]
[mcj]cudaGetDevice() failed. Status: CUDA driver version is insufficient for CUDA runtime version解决
目前在用anaconda进行各种包的管理，确实很方便，不过使用中还是有一些小问题。 关于anaconda管理cuda的问题： anaconda的方便之处在于它很方便的管理各种包，比如我们在不同的项目使用不同的cuda版本，这时就可以建立不同 […]
[mcj]pytorch基本函数理解【持续更新】
这篇博文主要讲pytorch的基本函数理解。 1 torch.arange(min,max,stride) 类似于python中的range
|
1 2 |
print(torch.arange(1,10,3)) > tensor([1, 4, 7]) |
可以看到，这相当于从1~10，每 […]
