基于pytorch實(shí)現(xiàn)bert4ner作箍、圖像分類(lèi)、yolv5目標(biāo)檢測(cè)小工具

代碼地址

壓縮后放在百度網(wǎng)盤(pán)
鏈接: https://pan.baidu.com/s/1r4JhMSGD5XyFIJVvZ2zjMQ 提取碼: i418

總體界面

image.png
預(yù)測(cè)結(jié)果
image.png

目錄

總體思路

項(xiàng)目目錄結(jié)構(gòu)

image.png

模型預(yù)測(cè)地址(訓(xùn)練和刪除功能被屏蔽)

鏈接: http://124.222.36.194/

部署架構(gòu)

image.png

基本流程

  1. 創(chuàng)建項(xiàng)目


    image.png
  2. 添加對(duì)應(yīng)項(xiàng)目的數(shù)據(jù)集


    image.png
  3. 訓(xùn)練


    image.png
  4. 預(yù)測(cè)橄登,可以調(diào)用對(duì)應(yīng)的rest api


    image.png

BERT實(shí)現(xiàn)的NER句子內(nèi)容提取

數(shù)據(jù)格式

使用 BOI 格式
image.png
數(shù)據(jù)目錄
image.png
預(yù)測(cè)
image.png

圖像分類(lèi)

數(shù)據(jù)目錄
image.png
預(yù)測(cè)

預(yù)測(cè)的案例圖片

http://124.222.36.194/images/dog.jpg
http://124.222.36.194/images/cat.jpg

yolov5目標(biāo)檢測(cè)

數(shù)據(jù)目錄
image.png
image.png
預(yù)測(cè)

預(yù)測(cè)的案例圖片

http://124.222.36.194/images/hat_person.jpg
預(yù)測(cè)結(jié)果
image.png

VUE-ELEMENT-ADMIN前端

來(lái)自VUE-ELEMENT-ADMIN,模板修改
使用nginx做前端服務(wù)器和后臺(tái)服務(wù)代理

nginx.conf配置

server {
        listen       80;
    listen  [::]:80;
        server_name  localhost;

        #charset koi8-r;

        #access_log  logs/host.access.log  main;

        location / {
            root   /app/www;
            autoindex on;  
        }
        
        location /images/ {
            alias /app/images/;
            autoindex on;
        }
    
        location /ner/ {
            add_header Cache-Control no-cache;
            add_header Cache-Control private;
            expires -1s;
            proxy_pass http://localhost:28582/;
        }

        location /detection/ {
            add_header Cache-Control no-cache;
            add_header Cache-Control private;
            expires -1s;
            proxy_pass http://localhost:28382/;
        }

        location /img/ {
            add_header Cache-Control no-cache;
            add_header Cache-Control private;
            expires -1s;
            proxy_pass http://localhost:28482/;
        }

docker容器

Dockerfile

FROM nvidia/cuda:11.4.0-base-ubuntu20.04
RUN sed -i 's/archive.ubuntu.com/mirrors.ustc.edu.cn/g' /etc/apt/sources.list
RUN mkdir /app
RUN apt update
RUN apt upgrade -y
RUN apt install build-essential -y
RUN apt install libtool -y
RUN apt-get install libpcre3 libpcre3-dev -y
RUN apt-get install zlib1g-dev -y
RUN apt-get install openssl -y
ADD nginx-1.20.0.tar.gz /usr/local/src/
RUN apt-get install gcc -y
RUN apt-get install g++ -y
RUN apt install libgl1-mesa-glx -y
RUN apt-get install make -y
RUN apt install wget -y
RUN apt install vim -y
WORKDIR /usr/local/src/nginx-1.20.0/
RUN ./configure --prefix=/usr/local/nginx --without-http_gzip_module && make && make install
ENV PATH /usr/local/nginx/sbin:$PATH
WORKDIR /app
RUN apt-get install -y python3 python3-pip
RUN ln -s /usr/bin/python3 /usr/bin/python
COPY requirements.txt .
COPY torch-1.8.2+cu111-cp38-cp38-linux_x86_64.whl .
COPY torchaudio-0.8.2-cp38-cp38-linux_x86_64.whl .
COPY torchvision-0.9.2+cu111-cp38-cp38-linux_x86_64.whl .
RUN pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
RUN pip install torch-1.8.2+cu111-cp38-cp38-linux_x86_64.whl
RUN pip install torchaudio-0.8.2-cp38-cp38-linux_x86_64.whl
RUN pip install torchvision-0.9.2+cu111-cp38-cp38-linux_x86_64.whl
RUN pip install virtualenv==20.0.23 -i https://pypi.tuna.tsinghua.edu.cn/simple
RUN /bin/bash -c "source /etc/profile"
RUN /bin/bash -c "source ~/.bashrc"
RUN virtualenv /app/photo-categories-venv
RUN virtualenv /app/dd-content-venv
RUN virtualenv /app/yolov5-prj-venv
COPY ./photo-categories ./photo-categories
COPY ./dd-content ./dd-content
COPY ./yolov5-prj ./yolov5-prj
COPY ./dd-deep-front/vue-element-admin/dist ./www
ADD ./nginx.conf /usr/local/nginx/conf/
COPY start.sh .
RUN chmod +x start.sh
ENTRYPOINT ["/app/start.sh"]
EXPOSE 80

構(gòu)建鏡像

docker build -t dddeep:v1 .
Removing intermediate container 866de083c375
 ---> f0803340cb54
Step 38/46 : COPY ./photo-categories ./photo-categories
 ---> 0d917f6795d5
Step 39/46 : COPY ./dd-content ./dd-content
 ---> 88f8565cb740
Step 40/46 : COPY ./yolov5-prj ./yolov5-prj
 ---> 9f441e497fe6
Step 41/46 : COPY ./dd-deep-front/vue-element-admin/dist ./www
 ---> 0ee9dbcc63f5
Step 42/46 : ADD ./nginx.conf /usr/local/nginx/conf/
 ---> 875a0d8fe504
Step 43/46 : COPY start.sh .
 ---> 1cad813647db
Step 44/46 : RUN chmod +x start.sh
 ---> Running in 355ad926fd74
Removing intermediate container 355ad926fd74
 ---> 488b58ac5c68
Step 45/46 : ENTRYPOINT ["/app/start.sh"]
 ---> Running in 935b571763b5
Removing intermediate container 935b571763b5
 ---> c27a972960a0
Step 46/46 : EXPOSE 80
 ---> Running in 609c2736b3d5
Removing intermediate container 609c2736b3d5
 ---> 588251e4a32f
Successfully built 588251e4a32f
Successfully tagged dddeep:v1

創(chuàng)建容器

docker run --name dddeep_1 -d --gpus all -p 80:80 dddeep:v1

進(jìn)入容器

  1. docker exec -it dddeep_1 /bin/bash
  2. 查看啟動(dòng)的python服務(wù),缺少yolov5的服務(wù)
root@f74abc4faa27:/app# ps -ef|grep python3
root           8       1 21 12:10 ?        00:00:06 /usr/bin/python3 /usr/local/bin/uvicorn main:app --host 0.0.0.0 --port 28582
root          10       1 14 12:10 ?        00:00:04 /usr/bin/python3 /usr/local/bin/uvicorn main:app --host 0.0.0.0 --port 28482
  1. 嘗試啟動(dòng)yolov5服務(wù)
cd /app/yolov5-prj
source /app/yolov5-prj-venv/bin/activate
export PYTHONIOENCODING=utf-8
export PYTHONPATH=/app/yolov5-prj/backend/src:/app/yolov5-prj/yolov5
uvicorn main:app --host 0.0.0.0 --port 28382

出現(xiàn)缺少庫(kù)錯(cuò)誤

root@f74abc4faa27:/app# cd /app/yolov5-prj
root@f74abc4faa27:/app/yolov5-prj# source /app/yolov5-prj-venv/bin/activate
(yolov5-prj-venv) root@f74abc4faa27:/app/yolov5-prj# export PYTHONIOENCODING=utf-8
(yolov5-prj-venv) root@f74abc4faa27:/app/yolov5-prj# export PYTHONPATH=/app/yolov5-prj/backend/src:/app/yolov5-prj/yolov5
(yolov5-prj-venv) root@f74abc4faa27:/app/yolov5-prj# uvicorn main:app --host 0.0.0.0 --port 28382
Traceback (most recent call last):
  File "/usr/local/bin/uvicorn", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/uvicorn/main.py", line 425, in main
    run(app, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/uvicorn/main.py", line 447, in run
    server.run()
  File "/usr/local/lib/python3.8/dist-packages/uvicorn/server.py", line 68, in run
    return asyncio.run(self.serve(sockets=sockets))
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/uvicorn/server.py", line 76, in serve
    config.load()
  File "/usr/local/lib/python3.8/dist-packages/uvicorn/config.py", line 448, in load
    self.loaded_app = import_from_string(self.app)
  File "/usr/local/lib/python3.8/dist-packages/uvicorn/importer.py", line 24, in import_from_string
    raise exc from None
  File "/usr/local/lib/python3.8/dist-packages/uvicorn/importer.py", line 21, in import_from_string
    module = importlib.import_module(module_str)
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 848, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/app/yolov5-prj/backend/src/main.py", line 2, in <module>
    from api import api_yolov5
  File "/app/yolov5-prj/backend/src/api/api_yolov5.py", line 16, in <module>
    import detect
  File "/app/yolov5-prj/yolov5/detect.py", line 14, in <module>
    import cv2
  File "/usr/local/lib/python3.8/dist-packages/cv2/__init__.py", line 180, in <module>
    bootstrap()
  File "/usr/local/lib/python3.8/dist-packages/cv2/__init__.py", line 152, in bootstrap
    native_module = importlib.import_module("cv2")
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: libgthread-2.0.so.0: cannot open shared object file: No such file or directory
  1. 安裝庫(kù)libglib2.0-dev
(yolov5-prj-venv) root@f74abc4faa27:/app/yolov5-prj# apt-get install libglib2.0-dev -y

安裝過(guò)程需要選擇時(shí)區(qū)歹河,選擇“亞洲”> “上海”

Please select the geographic area in which you live. Subsequent configuration questions will narrow this down by presenting a list of cities, representing the time zones in which
they are located.

  1. Africa  2. America  3. Antarctica  4. Australia  5. Arctic  6. Asia  7. Atlantic  8. Europe  9. Indian  10. Pacific  11. SystemV  12. US  13. Etc
Geographic area: 6

Please select the city or region corresponding to your time zone.

  1. Aden      10. Bahrain     19. Chongqing  28. Harbin       37. Jerusalem    46. Kuala_Lumpur  55. Novokuznetsk  64. Qyzylorda      73. Taipei         82. Ulaanbaatar
  2. Almaty    11. Baku        20. Colombo    29. Hebron       38. Kabul        47. Kuching       56. Novosibirsk   65. Rangoon        74. Tashkent       83. Urumqi
  3. Amman     12. Bangkok     21. Damascus   30. Ho_Chi_Minh  39. Kamchatka    48. Kuwait        57. Omsk          66. Riyadh         75. Tbilisi        84. Ust-Nera
  4. Anadyr    13. Barnaul     22. Dhaka      31. Hong_Kong    40. Karachi      49. Macau         58. Oral          67. Sakhalin       76. Tehran         85. Vientiane
  5. Aqtau     14. Beirut      23. Dili       32. Hovd         41. Kashgar      50. Magadan       59. Phnom_Penh    68. Samarkand      77. Tel_Aviv       86. Vladivostok
  6. Aqtobe    15. Bishkek     24. Dubai      33. Irkutsk      42. Kathmandu    51. Makassar      60. Pontianak     69. Seoul          78. Thimphu        87. Yakutsk
  7. Ashgabat  16. Brunei      25. Dushanbe   34. Istanbul     43. Khandyga     52. Manila        61. Pyongyang     70. Shanghai       79. Tokyo          88. Yangon
  8. Atyrau    17. Chita       26. Famagusta  35. Jakarta      44. Kolkata      53. Muscat        62. Qatar         71. Singapore      80. Tomsk          89. Yekaterinburg
  9. Baghdad   18. Choibalsan  27. Gaza       36. Jayapura     45. Krasnoyarsk  54. Nicosia       63. Qostanay      72. Srednekolymsk  81. Ujung_Pandang  90. Yerevan
Time zone: 70
  1. 安裝成功后花吟,再執(zhí)行啟動(dòng)yolov5服務(wù)
uvicorn main:app --host 0.0.0.0 --port 28382
(yolov5-prj-venv) root@f74abc4faa27:/app/yolov5-prj# uvicorn main:app --host 0.0.0.0 --port 28382
INFO:     Started server process [332]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:28382 (Press CTRL+C to quit)
^CINFO:     Shutting down
INFO:     Waiting for application shutdown.
INFO:     Application shutdown complete.
INFO:     Finished server process [332]
  1. 退出容器秸歧,停止容器,重新啟動(dòng)容器
(yolov5-prj-venv) root@f74abc4faa27:/app/yolov5-prj# exit

docker ps
CONTAINER ID   IMAGE       COMMAND           CREATED          STATUS          PORTS                               NAMES
f74abc4faa27   dddeep:v1   "/app/start.sh"   16 minutes ago   Up 16 minutes   0.0.0.0:80->80/tcp, :::80->80/tcp   dddeep_1

docker stop dddeep_1 

docker start dddeep_1
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
  • 序言:七十年代末衅澈,一起剝皮案震驚了整個(gè)濱河市键菱,隨后出現(xiàn)的幾起案子,更是在濱河造成了極大的恐慌今布,老刑警劉巖经备,帶你破解...
    沈念sama閱讀 211,123評(píng)論 6 490
  • 序言:濱河連續(xù)發(fā)生了三起死亡事件,死亡現(xiàn)場(chǎng)離奇詭異部默,居然都是意外死亡侵蒙,警方通過(guò)查閱死者的電腦和手機(jī),發(fā)現(xiàn)死者居然都...
    沈念sama閱讀 90,031評(píng)論 2 384
  • 文/潘曉璐 我一進(jìn)店門(mén)傅蹂,熙熙樓的掌柜王于貴愁眉苦臉地迎上來(lái)纷闺,“玉大人算凿,你說(shuō)我怎么就攤上這事±绻Γ” “怎么了氓轰?”我有些...
    開(kāi)封第一講書(shū)人閱讀 156,723評(píng)論 0 345
  • 文/不壞的土叔 我叫張陵,是天一觀(guān)的道長(zhǎng)浸卦。 經(jīng)常有香客問(wèn)我署鸡,道長(zhǎng),這世上最難降的妖魔是什么镐躲? 我笑而不...
    開(kāi)封第一講書(shū)人閱讀 56,357評(píng)論 1 283
  • 正文 為了忘掉前任储玫,我火速辦了婚禮,結(jié)果婚禮上萤皂,老公的妹妹穿的比我還像新娘撒穷。我一直安慰自己,他們只是感情好裆熙,可當(dāng)我...
    茶點(diǎn)故事閱讀 65,412評(píng)論 5 384
  • 文/花漫 我一把揭開(kāi)白布端礼。 她就那樣靜靜地躺著,像睡著了一般入录。 火紅的嫁衣襯著肌膚如雪蛤奥。 梳的紋絲不亂的頭發(fā)上,一...
    開(kāi)封第一講書(shū)人閱讀 49,760評(píng)論 1 289
  • 那天僚稿,我揣著相機(jī)與錄音凡桥,去河邊找鬼。 笑死蚀同,一個(gè)胖子當(dāng)著我的面吹牛缅刽,可吹牛的內(nèi)容都是我干的。 我是一名探鬼主播蠢络,決...
    沈念sama閱讀 38,904評(píng)論 3 405
  • 文/蒼蘭香墨 我猛地睜開(kāi)眼衰猛,長(zhǎng)吁一口氣:“原來(lái)是場(chǎng)噩夢(mèng)啊……” “哼!你這毒婦竟也來(lái)了刹孔?” 一聲冷哼從身側(cè)響起啡省,我...
    開(kāi)封第一講書(shū)人閱讀 37,672評(píng)論 0 266
  • 序言:老撾萬(wàn)榮一對(duì)情侶失蹤,失蹤者是張志新(化名)和其女友劉穎髓霞,沒(méi)想到半個(gè)月后卦睹,有當(dāng)?shù)厝嗽跇?shù)林里發(fā)現(xiàn)了一具尸體,經(jīng)...
    沈念sama閱讀 44,118評(píng)論 1 303
  • 正文 獨(dú)居荒郊野嶺守林人離奇死亡酸茴,尸身上長(zhǎng)有42處帶血的膿包…… 初始之章·張勛 以下內(nèi)容為張勛視角 年9月15日...
    茶點(diǎn)故事閱讀 36,456評(píng)論 2 325
  • 正文 我和宋清朗相戀三年分预,在試婚紗的時(shí)候發(fā)現(xiàn)自己被綠了。 大學(xué)時(shí)的朋友給我發(fā)了我未婚夫和他白月光在一起吃飯的照片薪捍。...
    茶點(diǎn)故事閱讀 38,599評(píng)論 1 340
  • 序言:一個(gè)原本活蹦亂跳的男人離奇死亡笼痹,死狀恐怖配喳,靈堂內(nèi)的尸體忽然破棺而出,到底是詐尸還是另有隱情凳干,我是刑警寧澤晴裹,帶...
    沈念sama閱讀 34,264評(píng)論 4 328
  • 正文 年R本政府宣布,位于F島的核電站救赐,受9級(jí)特大地震影響涧团,放射性物質(zhì)發(fā)生泄漏。R本人自食惡果不足惜经磅,卻給世界環(huán)境...
    茶點(diǎn)故事閱讀 39,857評(píng)論 3 312
  • 文/蒙蒙 一泌绣、第九天 我趴在偏房一處隱蔽的房頂上張望。 院中可真熱鬧预厌,春花似錦阿迈、人聲如沸。這莊子的主人今日做“春日...
    開(kāi)封第一講書(shū)人閱讀 30,731評(píng)論 0 21
  • 文/蒼蘭香墨 我抬頭看了看天上的太陽(yáng)。三九已至炭晒,卻和暖如春待逞,著一層夾襖步出監(jiān)牢的瞬間,已是汗流浹背网严。 一陣腳步聲響...
    開(kāi)封第一講書(shū)人閱讀 31,956評(píng)論 1 264
  • 我被黑心中介騙來(lái)泰國(guó)打工识樱, 沒(méi)想到剛下飛機(jī)就差點(diǎn)兒被人妖公主榨干…… 1. 我叫王不留,地道東北人震束。 一個(gè)月前我還...
    沈念sama閱讀 46,286評(píng)論 2 360
  • 正文 我出身青樓牺荠,卻偏偏與公主長(zhǎng)得像,于是被迫代替她去往敵國(guó)和親驴一。 傳聞我的和親對(duì)象是個(gè)殘疾皇子,可洞房花燭夜當(dāng)晚...
    茶點(diǎn)故事閱讀 43,465評(píng)論 2 348

推薦閱讀更多精彩內(nèi)容