Deephotoshoptream学习 TX1模块C++ Sample application 2详细测试

机器人虎哥

2024-04-24 帮助1人

大家好，我是虎哥，使用NVIDIA Jeston TX1 也有很长一段时间了，由于这是基本停产的一个模块，其实自己也担心有很多官方的demo无法适配跑起来了，所以花了点时间，进一步研究发挥其GPU性能，使用各种硬件协处理器来加速。今天周末，开始续测试一下DeepStream自带的C DEMO，开始入门学习DeepStream的使用，讲通过详细分享官方自带的5个典型例子，来完成入门学习，分享给大家，也是自己做个笔记总结。

学新通

0、C/C Sample Apps Source Details

参考资料

1、Sample test application 2

0、C/C Sample Apps Source Details

官网入口：https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_C_Sample_Apps.html

DeepStream SDK 包含插件、库、应用程序和源代码的存档。对于Debian安装（在Jetson或dGPU上）和SDK管理器安装，sources目录位于/opt/nvidia/deepstream/deepstream-6.2 sources。对于tar包，源文件位于提取的deepstream包中。DeepStream Python绑定和示例应用程序作为单独的包提供。有关详细信息，请参阅GitHub - NVIDIA-AI-IOT/deepstream_python_apps: DeepStream SDK Python bindings and sample applications.

使用Graph Composer创建的DeepStream图列在参考图部分。有关详细信息，请参见Graph Composer简介。

Reference test application	Path inside sources directory	Description
Sample test application 1	apps/sample_apps/deepstream-test1	Sample of how to use DeepStream elements for a single H.264 stream: filesrc → decode → nvstreammux → nvinfer or nvinferserver (primary detector) → nvdsosd → renderer. This app uses resnet10.caffemodel for detection.
Sample test application 2	apps/sample_apps/deepstream-test2	Sample of how to use DeepStream elements for a single H.264 stream: filesrc → decode → nvstreammux → nvinfer or nvinferserver (primary detector) → nvtracker → nvinfer or nvinferserver (secondary classifier) → nvdsosd → renderer. This app uses resnet10.caffemodel for detection and 3 classifier models (i.e., Car Color, Make and Model).
Sample test application 3	apps/sample_apps/deepstream-test3	Builds on deepstream-test1 (simple test application 1) to demonstrate how to:Use multiple sources in the pipeline.Use a uridecodebin to accept any type of input (e.g. RTSP/File), any GStreamer supported container format, and any codec.Configure Gst-nvstreammux to generate a batch of frames and infer on it for better resource utilization.Extract the stream metadata, which contains useful information about the frames in the batched buffer.This app uses resnet10.caffemodel for detection.
Sample test application 4	apps/sample_apps/deepstream-test4	Builds on deepstream-test1 for a single H.264 stream: filesrc, decode, nvstreammux, nvinfer or nvinferserver, nvdsosd, renderer to demonstrate how to:Use the Gst-nvmsgconv and Gst-nvmsgbroker plugins in the pipeline.Create NVDS_META_EVENT_MSG type metadata and attach it to the buffer.Use NVDS_META_EVENT_MSG for different types of objects, e.g. vehicle and person.Implement “copy” and “free” functions for use if metadata is extended through the extMsg field.This app uses resnet10.caffemodel for detection.
Sample test application 5	apps/sample_apps/deepstream-test5	Builds on top of deepstream-app. Demonstrates:Use of Gst-nvmsgconv and Gst-nvmsgbroker plugins in the pipeline for multistream.How to configure Gst-nvmsgbroker plugin from the config file as a sink plugin (for KAFKA, Azure, etc.).How to handle the RTCP sender reports from RTSP servers or cameras and translate the Gst Buffer PTS to a UTC timestamp.For more details refer the RTCP Sender Report callback function `test5_rtcp_sender_report_callback()` registration and usage in `deepstream_test5_app_main.c`. GStreamer callback registration with rtpmanager element’s “handle-sync” signal is documented in `apps-common/src/deepstream_source_bin.c`.This app uses resnet10.caffemodel for detection.

接下来的时间，我们就通过系统学习这5个示例程序。

test1: DeepStream的Hello World。介绍如何基于多种DeepStream 插件来构建一个Gstream 管道。这个样例中输入的是一个视频文件，经过解码、批处理、目标检测，并将检测信息显示在屏幕中。
test2: 在test1的基础上，将二级网络级联到一级网络。图中我们也能看到，在目标检测之后多了一个图像分类的模块。
test3: 在test1的基础上，如何实现多数据源。比如同时接入4路视频，实现对4路视频数据的同时推理。
test4: 在test1的基础上，展示如何使用message broker插件创建物联网服务。

参考资料

DeepStream SDK开发指南：NVIDIA DeepStream SDK Developer Guide — DeepStream 6.2 Release documentation

DeepStream 概况： https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_Overview.html

DeepStream 数据结构：https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_plugin_metadata.html

GStreamer 学习笔记： https://www.cnblogs.com/phinecos/archive/2009/06/07/1498166.html

B站DeepStream 相关视频合集：jetson nano中如何下载deepstream-哔哩哔哩_Bilibili

1、Sample test application 2

学新通

接着看一下deepstream-test2的范例，这是基于test1的基础上去添加“多级检测器”的功能，由于这个功能与DeepStream的追踪功能相捆绑，因此也许一并启动。

编程语言：C/C
代码量：506行（含注解）
输入源：单个H264/H265视频文件
智能推理：单个的4类别（car, person, bicycle, roadsign）主检测器，加上3个基于“Car”类别的次级检测器，包括颜色、品牌、车型等等，这里还必须打开“追踪器（tracker）”的功能
显示输出：显示器
插件流：filesrc -> h264parse -> nvv4l2decoder -> nvstreammux -> nvinfer (primary detector) -> nvtracker -> nvinfer(secondary classifier) -> nvvideoconvert -> nvdsosd -> nvegltransform -> nveglglessink

1.1 进入目录，找到代码：

cd /opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps
cd deepstream-test2

学新通

1.2 按照指导编译文件

从 README文件种可以看到编译步骤：

Compilation Steps:

  $ Set CUDA_VER in the MakeFile as per platform.
      For Jetson, CUDA_VER=10.2
      For x86, CUDA_VER=11.4
  $ sudo make

确认自己系统CUDA_VER，我自己就是CUDA_VER=10.2

学新通

打开MakeFile文件修改

学新通

保存退出后编译：

sudo make

nvidia@ubuntu:/opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-test2$ sudo make
cc -c -o deepstream_test2_app.o -DPLATFORM_TEGRA -I../../../includes -I /usr/local/cuda-10.2/include -pthread -I/usr/include/gstreamer-1.0 -I/usr/include/glib-2.0 -I/usr/lib/aarch64-linux-gnu/glib-2.0/include deepstream_test2_app.c
cc -o deepstream-test2-app deepstream_test2_app.o -lgstreamer-1.0 -lgobject-2.0 -lglib-2.0 -L/usr/local/cuda-10.2/lib64/ -lcudart -L/opt/nvidia/deepstream/deepstream-6.0/lib/ -lnvdsgst_meta -lnvds_meta -Wl,-rpath,/opt/nvidia/deepstream/deepstream-6.0/lib/

编译完成。

学新通

1.3 运行测试

README文件有如何运行的说明

To run:

  $ ./deepstream-test2-app <h264_elementary_stream>

NOTE: To compile the sources, run make with "sudo" or root permission.

官方自带的样本种有H264视频：

cd /opt/nvidia/deepstream/deepstream-6.0/samples/streams

学新通

先用官方的示例视频测试，其所在目录：

cd /opt/nvidia/deepstream/deepstream-6.0/samples/streams

回到测试目录：

cd /opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-test2

执行测试命令：在NoMachine终端执行

./deepstream-test2-app /opt/nvidia/deepstream/deepstream-6.0/samples/streams/sample_720p.h264

学新通

之后就是你泡杯茶，等着，就好了，等挺长时间，直到出现下列打印：

学新通

学新通我们可以回顾一下test1中的推理结果：

学新通

这里明显看到test2不仅能为每个检测到的物体标上编号，这就是“追踪”的功能，在“car”物体上，还有“颜色”、“品牌”、“车型”等信息。

学新通

1.4 报错尝试解决

报错信息：

ERROR: Deserialize engine failed because file path: /opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-test2/../../../../samples/models/Secondary_VehicleTypes/resnet18.caffemodel_b16_gpu0_int8.engine open error

ERROR: Deserialize engine failed because file path: /opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-test2/../../../../samples/models/Secondary_CarMake/resnet18.caffemodel_b16_gpu0_int8.engine open error

ERROR: Deserialize engine failed because file path: /opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-test2/../../../../samples/models/Secondary_CarColor/resnet18.caffemodel_b16_gpu0_int8.engine open error

ERROR: Deserialize engine failed because file path: /opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-test2/../../../../samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine open error

所以，报错是打开文件失败，找这几文件

/opt/nvidia/deepstream/deepstream-6.0/samples/models/Secondary_VehicleTypes/resnet18.caffemodel_b16_gpu0_int8.engin
/opt/nvidia/deepstream/deepstream-6.0/ssamples/models/Secondary_CarMake/resnet18.caffemodel_b16_gpu0_int8.engine
/opt/nvidia/deepstream/deepstream-6.0/ssamples/models/Secondary_CarColor/resnet18.caffemodel_b16_gpu0_int8.engine
/opt/nvidia/deepstream/deepstream-6.0/ssamples/models/Secondary_CarColor/resnet18.caffemodel_b16_gpu0_int8.engine
/opt/nvidia/deepstream/deepstream-6.0/samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine

进入一个文件夹看看

#进入
cd /opt/nvidia/deepstream/deepstream-6.0/samples/models/Secondary_VehicleTypes/
#
ll

发现根本没有我们需要的：“resnet18.caffemodel_b16_gpu0_int8.engin”这个文件，取而代之的是：“resnet18.caffemodel_b16_gpu0_fp16.engine”，这个时候发现是文件名不对，所以我果断的使用了新的文件名称：

#进入测试目录
cd /opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-test2
#依次打开配置文件
sudo vim dstest2_sgie2_config.txt
sudo vim dstest2_sgie1_config.txt
sudo vim dstest2_sgie3_config.txt
sudo vim dstest2_pgie_config.txt
#核对所有需要的engin文件位置和名称和实际是否对应

dstest2_pgie_config.txt 修改后

学新通 dstest2_sgie1_config.txt修改后

学新通

dstest2_sgie2_config.txt 修改后

学新通

dstest2_sgie3_config.txt修改后

学新通

改完之后我们再测试一下：执行测试命令：在NoMachine终端执行

./deepstream-test2-app /opt/nvidia/deepstream/deepstream-6.0/samples/streams/sample_720p.h264

学新通 无报错，而且可以感觉到加载很快，也就是之前我们的测试应该是一样的问题，这个报错导致加载很慢。

学新通

终于找到了这个问题的原因，并且解决了！可以安心睡觉了。

1.5 尝试输入不同的场景视频

上面的测试，我们都输入的事官方提供的视频，我自己转码了一些H.264存储的流文件，进行额外的一些测试看看：

学新通

学新通测试视频1：

#进入测试目录
cd /opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-test2
#测试视频1：/home/nvidia/lsrobot_worksapce/Deepstream/mysamples_streams/H264-Streat-1.h264
./deepstream-test2-app /home/nvidia/lsrobot_worksapce/Deepstream/mysamples_streams/H264-Streat-1.h264

学新通

测试视频2：

#进入测试目录
cd /opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-test2
#测试视频2：/home/nvidia/lsrobot_worksapce/Deepstream/mysamples_streams/H264-Streat-2.h264
./deepstream-test2-app /home/nvidia/lsrobot_worksapce/Deepstream/mysamples_streams/H264-Streat-2.h264

学新通

这篇好文章是转载于：学新通技术网

Deephotoshoptream学习 TX1模块C++ Sample application 2详细测试

0、C/C Sample Apps Source Details

参考资料

1、Sample test application 2

1.1 进入目录，找到代码：

1.2 按照指导编译文件

1.3 运行测试

1.4 报错尝试解决

1.5 尝试输入不同的场景视频

photoshop保存的图片太大微信发不了怎么办

Android 11 保存文件到外部存储，并分享文件

word里面弄一个表格后上面的标题会跑到下面怎么办

《学习通》视频自动暂停处理方法

photoshop扩展功能面板显示灰色怎么办

微信公众号没有声音提示怎么办

excel下划线不显示怎么办

excel打印预览压线压字怎么办

怎样阻止微信小程序自动打开

TikTok加速器哪个好免费的TK加速器推荐