출처: http://www.linuxdata.org/bbs/board.php?bo_table=TensorFlow&wr_id=8&sst=wr_hit&sod=desc&sop=and&page=1
[AI] LLM RAG 환경 구성 - 1. Ollama 설치
안녕하세요.
꿈꾸는여행자입니다.
LLM RAG 환경 구성을 다루고자 합니다.
이번 항목에서는 Ollama 설치 까지 실습을 진행해 보도록 하겠습니다.
상세 내역은 아래와 같습니다.
감사합니다.
> 아래
________________
목차
1. Overview 3
1.1. 분석 3
1.1.1. 관련 검색 3
1.1.2. Url 3
1.2. 설계 3
2. Ollama 5
2.1. Prepare 5
2.1.1. Setting - sudo user 5
2.1.2. Package 5
2.1.3. cuda 9
2.2. Install Ollama 11
2.2.1. Download and Install Ollama 11
2.2.2. Install Ollama Web-UI [Options] 11
2.3. HuggingFace-Hub 12
2.3.1. HuggingFace-Hub 설치 12
2.3.2. Download GGUF 파일 13
2.3.3. Modelfile 준비 14
2.4. Run Ollama 15
2.4.1. List 15
2.4.2. Create 15
2.4.3. Run 16
3. LangServe 17
3.1. Prepare 17
3.1.1. git clone 17
3.1.2. Install Python Packages 17
3.1.3. Check Source 18
3.2. LangServe에서 Ollama 체인 생성 19
3.2.1. Run - langserve 19
3.3. Check - Browser 19
3.3.1. LangServe 19
3.3.2. Fast API 20
4. streamlit 21
4.1. Prepare 21
4.1.1. Install Python Packages 21
4.2. Setting 21
4.2.1. main.py - LANGSERVE_ENDPOINT 21
4.3. Run 22
4.3.1. Run - streamlit 22
4.3. Check - Browser 23
________________
1. Overview
1.1. 분석
1.1.1. 관련 검색
* Word
* ollama langserve streamlit
1.1.2. Url
* RAG
* 무료로 한국어 파인튜닝 모델 받아서 나만의 로컬 LLM 호스팅 하기(LangServe) + RAG 까지!!
* https://github.com/teddylee777/langserve_ollama
* https://www.youtube.com/watch?v=VkcaigvTrug
* BEST
* Ollama
* 로컬에서 무료로 사용할 수 있는 LLM 도구, Ollama 활용 가이드
* https://anpigon.tistory.com/434
* Linux에서 Ollama 기반으로 LLM 실행하기
* Ollama 를 활용해서 개인형 LLM 서버 구성하기
* https://devocean.sk.com/blog/techBoardDetail.do?ID=165685
* [LLM] Ollama Web-UI 설치
* https://pasus.tistory.com/322
* Amazon EC2에서 Ollama 서버 및 Ollama 웹 UI를 사용하여 로컬 LLM을 배포하는 방법
* Docker 기반
* Cuda
* Installing CUDA on Rocky Linux
* https://doc.flowhub.com.cn/en/FAQ/RockylinuxInstallCuda/
1.2. 설계
________________
2. Ollama
2.1. Prepare
2.1.1. Setting - sudo user
* 대상 계정에 sudo 권한 설정
* usermod -aG wheel lds
su - lds
[root@llm ~]# cat /etc/group | grep wheel
wheel:x:10:
[root@llm ~]# usermod -aG wheel lds
[root@llm ~]# cat /etc/group | grep wheel
wheel:x:10:lds
[root@llm ~]# su - lds
[lds@llm ~]$ sudo ls -la /root
[sudo] password for lds:
total 36
dr-xr-x---. 5 root root 4096 May 2 17:23 .
dr-xr-xr-x. 18 root root 235 Jan 17 14:38 ..
-rw-------. 1 root root 1023 Jan 17 14:26 anaconda-ks.cfg
-rw-------. 1 root root 512 May 2 17:21 .bash_history
-rw-r--r--. 1 root root 18 May 11 2022 .bash_logout
-rw-r--r--. 1 root root 141 May 11 2022 .bash_profile
-rw-r--r--. 1 root root 429 May 11 2022 .bashrc
drwx------. 3 root root 17 Apr 3 09:51 .cache
-rw-r--r--. 1 root root 100 May 11 2022 .cshrc
drwxr-xr-x. 6 root root 87 Apr 3 11:31 llama2env
drwx------. 2 root root 6 Jan 17 14:19 .ssh
-rw-r--r--. 1 root root 129 May 11 2022 .tcshrc
-rw-------. 1 root root 1358 May 2 17:23 .viminfo
[lds@llm ~]$
*
2.1.2. Package
* Install git
* git 설치
sudo dnf install git git-core
[root@llm ~]# id
uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
[root@llm ~]# dnf install git git-core
Last metadata expiration check: 0:06:24 ago on Thu 02 May 2024 05:09:38 PM KST.
Dependencies resolved.
======================================================================================================================
Package Architecture Version Repository Size
======================================================================================================================
Installing:
git x86_64 2.39.3-1.el9_2 appstream 61 k
git-core x86_64 2.39.3-1.el9_2 appstream 4.2 M
Installing dependencies:
git-core-doc noarch 2.39.3-1.el9_2 appstream 2.6 M
perl-Error noarch 1:0.17029-7.el9 appstream 41 k
perl-Git noarch 2.39.3-1.el9_2 appstream 37 k
perl-TermReadKey x86_64 2.38-11.el9 appstream 36 k
perl-lib x86_64 0.65-480.el9 appstream 15 k
Transaction Summary
======================================================================================================================
Install 7 Packages
Total download size: 7.0 M
Installed size: 36 M
Is this ok [y/N]: y
Downloading Packages:
(1/7): perl-lib-0.65-480.el9.x86_64.rpm 76 kB/s | 15 kB 00:00
(2/7): perl-Error-0.17029-7.el9.noarch.rpm 202 kB/s | 41 kB 00:00
(3/7): perl-Git-2.39.3-1.el9_2.noarch.rpm 933 kB/s | 37 kB 00:00
(4/7): perl-TermReadKey-2.38-11.el9.x86_64.rpm 66 kB/s | 36 kB 00:00
(5/7): git-2.39.3-1.el9_2.x86_64.rpm 253 kB/s | 61 kB 00:00
(6/7): git-core-doc-2.39.3-1.el9_2.noarch.rpm 1.7 MB/s | 2.6 MB 00:01
(7/7): git-core-2.39.3-1.el9_2.x86_64.rpm 1.5 MB/s | 4.2 MB 00:02
----------------------------------------------------------------------------------------------------------------------
Total 1.5 MB/s | 7.0 MB 00:04
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : git-core-2.39.3-1.el9_2.x86_64 1/7
Installing : git-core-doc-2.39.3-1.el9_2.noarch 2/7
Installing : perl-TermReadKey-2.38-11.el9.x86_64 3/7
Installing : perl-lib-0.65-480.el9.x86_64 4/7
Installing : perl-Error-1:0.17029-7.el9.noarch 5/7
Installing : git-2.39.3-1.el9_2.x86_64 6/7
Installing : perl-Git-2.39.3-1.el9_2.noarch 7/7
Running scriptlet: perl-Git-2.39.3-1.el9_2.noarch 7/7
Verifying : perl-Error-1:0.17029-7.el9.noarch 1/7
Verifying : perl-lib-0.65-480.el9.x86_64 2/7
Verifying : perl-TermReadKey-2.38-11.el9.x86_64 3/7
Verifying : perl-Git-2.39.3-1.el9_2.noarch 4/7
Verifying : git-core-doc-2.39.3-1.el9_2.noarch 5/7
Verifying : git-core-2.39.3-1.el9_2.x86_64 6/7
Verifying : git-2.39.3-1.el9_2.x86_64 7/7
Installed:
git-2.39.3-1.el9_2.x86_64 git-core-2.39.3-1.el9_2.x86_64 git-core-doc-2.39.3-1.el9_2.noarch
perl-Error-1:0.17029-7.el9.noarch perl-Git-2.39.3-1.el9_2.noarch perl-TermReadKey-2.38-11.el9.x86_64
perl-lib-0.65-480.el9.x86_64
Complete!
[root@llm ~]#
*
* python
* python과 pip 확인
sudo dnf install python3.11
python3.11 --version
pip --version
[lds@llm ~]$ sudo dnf install python3.11
Last metadata expiration check: 0:41:48 ago on Thu 02 May 2024 05:09:38 PM KST.
Dependencies resolved.
======================================================================================================================
Package Architecture Version Repository Size
======================================================================================================================
Installing:
python3.11 x86_64 3.11.5-1.el9_3 appstream 26 k
Installing dependencies:
libnsl2 x86_64 2.0.0-1.el9.0.1 appstream 30 k
mpdecimal x86_64 2.5.1-3.el9 appstream 85 k
python3.11-libs x86_64 3.11.5-1.el9_3 appstream 9.3 M
python3.11-pip-wheel noarch 22.3.1-4.el9_3.1 appstream 1.4 M
python3.11-setuptools-wheel noarch 65.5.1-2.el9 appstream 712 k
Transaction Summary
======================================================================================================================
Install 6 Packages
Total download size: 12 M
Installed size: 47 M
Is this ok [y/N]: y
Downloading Packages:
(1/6): libnsl2-2.0.0-1.el9.0.1.x86_64.rpm 179 kB/s | 30 kB 00:00
(2/6): python3.11-3.11.5-1.el9_3.x86_64.rpm 79 kB/s | 26 kB 00:00
(3/6): mpdecimal-2.5.1-3.el9.x86_64.rpm 173 kB/s | 85 kB 00:00
(4/6): python3.11-setuptools-wheel-65.5.1-2.el9.noarch.rpm 617 kB/s | 712 kB 00:01
(5/6): python3.11-pip-wheel-22.3.1-4.el9_3.1.noarch.rpm 850 kB/s | 1.4 MB 00:01
(6/6): python3.11-libs-3.11.5-1.el9_3.x86_64.rpm 2.1 MB/s | 9.3 MB 00:04
----------------------------------------------------------------------------------------------------------------------
Total 1.9 MB/s | 12 MB 00:05
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : python3.11-pip-wheel-22.3.1-4.el9_3.1.noarch 1/6
Installing : python3.11-setuptools-wheel-65.5.1-2.el9.noarch 2/6
Installing : mpdecimal-2.5.1-3.el9.x86_64 3/6
Installing : libnsl2-2.0.0-1.el9.0.1.x86_64 4/6
Installing : python3.11-3.11.5-1.el9_3.x86_64 5/6
Installing : python3.11-libs-3.11.5-1.el9_3.x86_64 6/6
Running scriptlet: python3.11-libs-3.11.5-1.el9_3.x86_64 6/6
Verifying : libnsl2-2.0.0-1.el9.0.1.x86_64 1/6
Verifying : mpdecimal-2.5.1-3.el9.x86_64 2/6
Verifying : python3.11-libs-3.11.5-1.el9_3.x86_64 3/6
Verifying : python3.11-3.11.5-1.el9_3.x86_64 4/6
Verifying : python3.11-setuptools-wheel-65.5.1-2.el9.noarch 5/6
Verifying : python3.11-pip-wheel-22.3.1-4.el9_3.1.noarch 6/6
Installed:
libnsl2-2.0.0-1.el9.0.1.x86_64 mpdecimal-2.5.1-3.el9.x86_64
python3.11-3.11.5-1.el9_3.x86_64 python3.11-libs-3.11.5-1.el9_3.x86_64
python3.11-pip-wheel-22.3.1-4.el9_3.1.noarch python3.11-setuptools-wheel-65.5.1-2.el9.noarch
Complete!
[lds@llm ~]$
[lds@llm ~]$ python3.11 --version
Python 3.11.5
[lds@llm ~]$
[lds@llm ~]$ pip --version
pip 21.2.3 from /home/lds/Works/llm/env/lib64/python3.9/site-packages/pip (python 3.9)
[lds@llm ~]$
*
* python - env
* python 환경 구성 및 실행
* python3.11 로 실행
python3.11 -m ven env
source env/bin/activate
[lds@llm llm]$ pwd
/home/lds/Works/llm
[lds@llm llm]$ python3.11 -m venv env
[lds@llm llm]$ source env/bin/activate
(env) [lds@llm llm]$
*
2.1.3. cuda
https://doc.flowhub.com.cn/en/FAQ/RockylinuxInstallCuda/
Update the system version and kernel
sudo dnf update && sudo dnf upgrade -y
Install epel-release
sudo dnf install epel-release
Add the NVIDIA repository
sudo dnf config-manager --add-repo \
https://developer.download.nvidia.com/compute/cuda/repos/rhel8/x86_64/cuda-rhel8.repo
dnf clean all
dnf list
Install the corresponding development tools and header files
sudo dnf install kernel-devel-$(uname -r) kernel-headers-$(uname -r)
Install the NVIDIA driver
sudo dnf install nvidia-driver nvidia-settings
Install the CUDA driver
sudo dnf install cuda-driver
2.2. Install Ollama
2.2.1. Download and Install Ollama
* Install with one command:
* sudo 권한이 있는 User 또는 root로 실행
* 30분 소요
curl -fsSL https://ollama.com/install.sh | sh
[root@llm ~]# curl -fsSL https://ollama.com/install.sh | sh
>>> Downloading ollama...
######################################################################## 100.0%##O#-# ######################################################################## 100.0%
>>> Installing ollama to /usr/local/bin...
>>> Creating ollama user...
>>> Adding ollama user to render group...
>>> Adding ollama user to video group...
>>> Adding current user to ollama group...
>>> Creating ollama systemd service...
>>> Enabling and starting ollama service...
Created symlink /etc/systemd/system/default.target.wants/ollama.service → /etc/systemd/system/ollama.service.
>>> The Ollama API is now available at 127.0.0.1:11434.
>>> Install complete. Run "ollama" from the command line.
WARNING: No NVIDIA/AMD GPU detected. Ollama will run in CPU-only mode.
[root@llm ~]#
[AI] LLM RAG 환경 구성 - 2 Ollama 실행 및 모델 등록 > 머신러닝/AI 자료실 | LinuxDataSystem Community
사이트 정보 회사명 : (주)리눅스데이타시스템 대표이사 : 정정모 본사 : 강남구 봉은사로 114길 40 홍선빌딩 2층 - tel : 02-6207-1160 대전지사 : 유성구 노은로174 도원프라자 5층 - tel : 042-331-1161 --> -->
www.linuxdata.org
출처: https://3604.tistory.com/manage/newpost/?type=post&returnURL=%2Fmanage%2Fposts%2F
[AI] LLM RAG 환경 구성 - 2 Ollama 실행 및 모델 등록
안녕하세요.
꿈꾸는여행자입니다.
LLM RAG 환경 구성을 다루고자 합니다.
이번 항목에서는 Ollama 실행 및 모델 등록 실습을 진행해 보도록 하겠습니다.
상세 내역은 아래와 같습니다.
감사합니다.
> 아래
________________
2.3. HuggingFace-Hub
* HuggingFace gguf 파일을 Ollama 로딩
2.3.1. HuggingFace-Hub 설치
* 설치
* gguf 환경에서 사용하기 위해 huggingface 구성
* 3분 소요
pip install huggingface-hub
(env) [lds@llm llm]$ pip install huggingface-hub
Collecting huggingface-hub
Using cached huggingface_hub-0.22.2-py3-none-any.whl (388 kB)
Collecting filelock
Using cached filelock-3.14.0-py3-none-any.whl (12 kB)
Collecting fsspec>=2023.5.0
Using cached fsspec-2024.3.1-py3-none-any.whl (171 kB)
Collecting packaging>=20.9
Using cached packaging-24.0-py3-none-any.whl (53 kB)
Collecting pyyaml>=5.1
Downloading PyYAML-6.0.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (757 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 757.7/757.7 kB 2.2 MB/s eta 0:00:00
Collecting requests
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Collecting tqdm>=4.42.1
Using cached tqdm-4.66.2-py3-none-any.whl (78 kB)
Collecting typing-extensions>=3.7.4.3
Using cached typing_extensions-4.11.0-py3-none-any.whl (34 kB)
Collecting charset-normalizer<4,>=2
Downloading charset_normalizer-3.3.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (140 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 140.3/140.3 kB 1.4 MB/s eta 0:00:00
Collecting idna<4,>=2.5
Using cached idna-3.7-py3-none-any.whl (66 kB)
Collecting urllib3<3,>=1.21.1
Using cached urllib3-2.2.1-py3-none-any.whl (121 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2024.2.2-py3-none-any.whl (163 kB)
Installing collected packages: urllib3, typing-extensions, tqdm, pyyaml, packaging, idna, fsspec, filelock, charset-normalizer, certifi, requests, huggingface-hub
Successfully installed certifi-2024.2.2 charset-normalizer-3.3.2 filelock-3.14.0 fsspec-2024.3.1 huggingface-hub-0.22.2 idna-3.7 packaging-24.0 pyyaml-6.0.1 requests-2.31.0 tqdm-4.66.2 typing-extensions-4.11.0 urllib3-2.2.1
[notice] A new release of pip available: 22.3.1 -> 24.0
[notice] To update, run: pip install --upgrade pip
(env) [lds@llm llm]$
(env) [lds@llm llm]$
*
2.3.2. Download GGUF 파일
* 사용되는 파일
* 아래의 예시는 EEVE-Korean-Instruct-10.8B-v1.0
* HF: https://huggingface.co/yanolja/EEVE-Korean-Instruct-10.8B-v1.0
* GGUF: https://huggingface.co/heegyu/EEVE-Korean-Instruct-10.8B-v1.0-GGUF
* GGUF 파일을 다운로드 받기 위하여 https://huggingface.co/heegyu/EEVE-Korean-Instruct-10.8B-v1.0-GGUF 에서 원하는 .gguf 모델을 다운로드 받습니다.
* 30분 소요
huggingface-cli download \
heegyu/EEVE-Korean-Instruct-10.8B-v1.0-GGUF \
ggml-model-Q5_K_M.gguf \
--local-dir [PATH_DOWNLOAD] \
--local-dir-use-symlinks False
huggingface-cli download \
heegyu/EEVE-Korean-Instruct-10.8B-v1.0-GGUF \
ggml-model-Q5_K_M.gguf \
--local-dir ./ \
--local-dir-use-symlinks False
(env) [lds@llm llm]$ huggingface-cli download \
heegyu/EEVE-Korean-Instruct-10.8B-v1.0-GGUF \
ggml-model-Q5_K_M.gguf \
--local-dir ./ \
--local-dir-use-symlinks False
Consider using `hf_transfer` for faster downloads. This solution comes with some limitations. See https://huggingface.co/docs/huggingface_hub/hf_transfer for more details.
downloading https://huggingface.co/heegyu/EEVE-Korean-Instruct-10.8B-v1.0-GGUF/resolve/main/ggml-model-Q5_K_M.gguf to /home/lds/.cache/huggingface/hub/tmpaz24aq7z
ggml-model-Q5_K_M.gguf: 100%|████████████████████████████████████████████████████| 7.65G/7.65G [18:59<00:00, 6.72MB/s]
./ggml-model-Q5_K_M.gguf
(env) [lds@llm llm]$
*
2.3.3. Modelfile 준비
* EEVE-Korean-Instruct-10.8B-v1.0 예시
* 각 GGUF 마다 사용되는 Template 존재
* SOLAR Template 내용 활용시 https://huggingface.co/upstage/SOLAR-10.7B-Instruct-v1.0
* vi Modelfile
FROM ggml-model-Q5_K_M.gguf
TEMPLATE """{{- if .System }}
<s>{{ .System }}</s>
{{- end }}
<s>Human:
{{ .Prompt }}</s>
<s>Assistant:
"""
SYSTEM """A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions."""
PARAMETER stop <s>
PARAMETER stop </s>
(env) [lds@llm llm]$ pwd
/home/lds/Works/llm
(env) [lds@llm llm]$
(env) [lds@llm llm]$ vi Modelfile
(env) [lds@llm llm]$ cat Modelfile
FROM ggml-model-Q5_K_M.gguf
TEMPLATE """{{- if .System }}
<s>{{ .System }}</s>
{{- end }}
<s>Human:
{{ .Prompt }}</s>
<s>Assistant:
"""
SYSTEM """A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions."""
PARAMETER stop <s>
PARAMETER stop </s>
(env) [lds@llm llm]$
2.4. Run Ollama
2.4.1. List
* Ollama 모델 목록
* ollama list
(env) [lds@llm llm]$ ollama list
NAME ID SIZE MODIFIED
(env) [lds@llm llm]$
2.4.2. Create
* Ollama 모델 생성
* 3분 소요
ollama create EEVE-Korean-10.8B -f EEVE-Korean-Instruct-10.8B-v1.0-GGUF/Modelfile
ollama create EEVE-Korean-10.8B -f ./Modelfile
ollama list
(env) [lds@llm llm]$ ollama create EEVE-Korean-10.8B -f ./Modelfile
transferring model data
creating model layer
creating template layer
creating system layer
creating parameters layer
creating config layer
using already created layer sha256:b9e3d1ad5e8aa6db09610d4051820f06a5257b7d7f0b06c00630e376abcfa4c1
writing layer sha256:6b70a2ad0d545ca50d11b293ba6f6355eff16363425c8b163289014cf19311fc
writing layer sha256:1fa69e2371b762d1882b0bd98d284f312a36c27add732016e12e52586f98a9f5
writing layer sha256:fc44d47f7d5a1b793ab68b54cdba0102140bd358739e9d78df4abf18432fb3ea
writing layer sha256:fd9b55ebd209ef8ef07f55be3fa0f624f8781cf08d6f0931875cffd76fcdbcd7
writing manifest
success
(env) [lds@llm llm]$
(env) [lds@llm llm]$ ollama list
NAME ID SIZE MODIFIED
EEVE-Korean-10.8B:latest 254d40e8ab74 7.7 GB About a minute ago
(env) [lds@llm llm]$
*
2.4.3. Run
* Ollama 실행
* model 기준으로 cli로 문의 답변 가능
* 1분안에 답변하지만 Text가 느리게 나옴
* CPU 4Core/ Memory 16GB 기준
ollama run EEVE-Korean-10.8B:latest
(env) [lds@llm app]$ ollama run EEVE-Korean-10.8B:latest
>>> Hello
Hello! I'm here to assist you in any way I can. Please go ahead and ask your question or provide a topic for me
to help with. If I don't know the answer to something, I will do my best to point you in the right direction or
find the information together. I strive to provide accurate and positive responses while ensuring safety and
respect for all individuals involved. Let's start our conversation!
>>> /?
Available Commands:
/set Set session variables
/show Show model information
/load <model> Load a session or model
/save <model> Save your current session
/bye Exit
/?, /help Help for a command
/? shortcuts Help for keyboard shortcuts
Use """ to begin a multi-line message.
>>> /bye
(env) [lds@llm app]$
[AI] LLM RAG 환경 구성 - 3 LangServe 구성
안녕하세요.
꿈꾸는여행자입니다.
LLM RAG 환경 구성을 다루고자 합니다.
이번 항목에서는
LangServe 구성 실습을 진행해 보도록 하겠습니다.
상세 내역은 아래와 같습니다.
감사합니다.
> 아래
3. LangServe
3.1. Prepare
3.1.1. git clone
* github에서 Source를 Download 받음
* git clone https://github.com/teddylee777/langserve_ollama
mv langserve_ollama/* langserve_ollama/.* ./
rmdir langserve_ollama/
(env) [lds@llm llm]$ git clone https://github.com/teddylee777/langserve_ollama
Cloning into 'langserve_ollama'...
remote: Enumerating objects: 105, done.
remote: Counting objects: 100% (105/105), done.
remote: Compressing objects: 100% (71/71), done.
remote: Total 105 (delta 54), reused 76 (delta 27), pack-reused 0
Receiving objects: 100% (105/105), 374.26 KiB | 4.11 MiB/s, done.
Resolving deltas: 100% (54/54), done.
(env) [lds@llm llm]$
(env) [lds@llm llm]$ mv langserve_ollama/* langserve_ollama/.* ./
mv: cannot move 'langserve_ollama/.' to './.': Device or resource busy
mv: cannot move 'langserve_ollama/..' to './..': Device or resource busy
(env) [lds@llm llm]$ rmdir langserve_ollama/
(env) [lds@llm llm]$ ls -al
total 7474620
drwxr-xr-x. 8 lds lds 4096 May 2 17:35 .
drwxr-xr-x. 3 lds lds 17 May 2 17:31 ..
drwxr-xr-x. 2 lds lds 125 May 2 17:33 app
drwxr-xr-x. 5 lds lds 74 May 2 16:17 env
drwxr-xr-x. 3 lds lds 4096 May 2 17:33 example
-rw-r--r--. 1 lds lds 7653486272 May 2 16:42 ggml-model-Q5_K_M.gguf
drwxr-xr-x. 8 lds lds 163 May 2 17:33 .git
-rw-r--r--. 1 lds lds 50 May 2 17:33 .gitignore
drwxr-xr-x. 2 lds lds 41 May 2 17:33 images
-rw-r--r--. 1 lds lds 318 May 2 16:46 Modelfile
drwxr-xr-x. 6 lds lds 141 May 2 17:33 ollama-modelfile
-rw-r--r--. 1 lds lds 481043 May 2 17:33 poetry.lock
-rw-r--r--. 1 lds lds 659 May 2 17:33 pyproject.toml
-rw-r--r--. 1 lds lds 3343 May 2 17:33 README.md
-rw-r--r--. 1 lds lds 14983 May 2 17:33 requirements.txt
(env) [lds@llm llm]$
*
3.1.2. Install Python Packages
* Install
* https://python.langchain.com/docs/get_started/installation/
* requirements.txt 파일 Package 설치시 너무 오래 걸림 1시간 넘게
pip install -r requirements.txt
pip install "langserve[all]"
pip install langchain_community
pip install langchain_openai
(env) [lds@llm llm]$ pip install "langserve[all]"
Collecting langserve[all]
Downloading langserve-0.1.1-py3-none-any.whl (1.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 3.8 MB/s eta 0:00:00
Collecting fastapi<1,>=0.90.1
Downloading fastapi-0.111.0-py3-none-any.whl (91 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 92.0/92.0 kB 160.6 kB/s eta 0:00:00
Collecting httpx>=0.23.0
Using cached httpx-0.27.0-py3-none-any.whl (75 kB)
…
Successfully uninstalled packaging-24.0
Successfully installed MarkupSafe-2.1.5 annotated-types-0.6.0 anyio-4.3.0 click-8.1.7 dnspython-2.6.1 email_validator-2.1.1 fastapi-0.111.0 fastapi-cli-0.0.2 h11-0.14.0 httpcore-1.0.5 httptools-0.6.1 httpx-0.27.0 jinja2-3.1.3 jsonpatch-1.33 jsonpointer-2.4 langchain-core-0.1.50 langserve-0.1.1 langsmith-0.1.53 markdown-it-py-3.0.0 mdurl-0.1.2 orjson-3.10.2 packaging-23.2 pydantic-2.7.1 pydantic-core-2.18.2 pygments-2.17.2 python-dotenv-1.0.1 python-multipart-0.0.9 rich-13.7.1 shellingham-1.5.4 sniffio-1.3.1 sse-starlette-1.8.2 starlette-0.37.2 tenacity-8.2.3 typer-0.12.3 ujson-5.9.0 uvicorn-0.29.0 uvloop-0.19.0 watchfiles-0.21.0 websockets-12.0
[notice] A new release of pip available: 22.3.1 -> 24.0
[notice] To update, run: pip install --upgrade pip
(env) [lds@llm llm]$
3.1.3. Check Source
* pycharm community 사용
* sh pycharm.sh
[lds@host bin]$ pwd
/home/lds/Work/pycharm-community-2023.3.5/bin
[lds@host bin]$ sh pycharm.sh
CompileCommand: exclude com/intellij/openapi/vfs/impl/FilePartNodeRoot.trieDescend bool exclude = true
2024-05-03 09:22:47,109 [ 312] WARN - #c.i.s.ComponentManagerImpl - `preload=true` must be used only for core services (service=com.intellij.ae.database.core.baseEvents.fus.AddStatisticsEventLogListenerTemporary, plugin=com.jetbrains.ae.database)
2024-05-03 09:22:47,110 [ 313] WARN - #c.i.s.ComponentManagerImpl - `preload=true` must be used only for core services (service=com.jetbrains.rdserver.statistics.BackendStatisticsManager, plugin=com.jetbrains.codeWithMe)
2024-05-03 09:22:47,556 [ 759] WARN - #c.i.s.ComponentManagerImpl - com.intellij.psi.search.FilenameIndex initializer requests com.intellij.ide.plugins.PluginUtil instance
*
3.2. LangServe에서 Ollama 체인 생성
3.2.1. Run - langserve
* app 폴더 진입 후 server.py 실행
* python server.py
[lds@llm ~]$ cd /home/lds/Works/llm/
[lds@llm llm]$ source env/bin/activate
(env) [lds@llm llm]$ cd app/
(env) [lds@llm app]$ pwd
/home/lds/Works/llm/app
(env) [lds@llm app]$ python server.py
(env) [lds@llm app]$ python server.py
INFO: Started server process [2043]
INFO: Waiting for application startup.
__ ___ .__ __. _______ _______. _______ .______ ____ ____ _______
| | / \ | \ | | / _____| / || ____|| _ \ \ \ / / | ____|
| | / ^ \ | \| | | | __ | (----`| |__ | |_) | \ \/ / | |__
| | / /_\ \ | . ` | | | |_ | \ \ | __| | / \ / | __|
| `----./ _____ \ | |\ | | |__| | .----) | | |____ | |\ \----. \ / | |____
|_______/__/ \__\ |__| \__| \______| |_______/ |_______|| _| `._____| \__/ |_______|
LANGSERVE: Playground for chain "/prompt/" is live at:
LANGSERVE: │
LANGSERVE: └──> /prompt/playground/
LANGSERVE:
LANGSERVE: Playground for chain "/llm/" is live at:
LANGSERVE: │
LANGSERVE: └──> /llm/playground/
LANGSERVE:
LANGSERVE: Playground for chain "/xionic/" is live at:
LANGSERVE: │
LANGSERVE: └──> /xionic/playground/
LANGSERVE:
LANGSERVE: Playground for chain "/chat/" is live at:
LANGSERVE: │
LANGSERVE: └──> /chat/playground/
LANGSERVE:
LANGSERVE: Playground for chain "/translate/" is live at:
LANGSERVE: │
LANGSERVE: └──> /translate/playground/
LANGSERVE:
LANGSERVE: See all available routes at /docs/
…
*
3.3. Check - Browser
3.3.1. LangServe
* Browser
* http://192.168.50.130:8000
* http://192.168.50.130:8000/xionic/playground/
3.3.2. Fast API
* Browser
* http://192.168.50.130:8000/docs#/
[AI] LLM RAG 환경 구성 - 4 streamlit 구성
안녕하세요.
꿈꾸는여행자입니다.
LLM RAG 환경 구성을 다루고자 합니다.
이번 항목에서는
streamlit 구성 실습을 진행해 보도록 하겠습니다.
상세 내역은 아래와 같습니다.
감사합니다.
> 아래
________________
목차
4. streamlit 21
4.1. Prepare 21
4.1.1. Install Python Packages 21
4.2. Setting 21
4.2.1. main.py - LANGSERVE_ENDPOINT 21
4.3. Run 22
4.3.1. Run - streamlit 22
4.3. Check - Browser 23
________________
4. streamlit
4.1. Prepare
4.1.1. Install Python Packages
* Install
* pip install unstructured[pdf]
* 30분 소요
pip install streamlit
pip install langchain
pip install unstructured
pip install unstructured[pdf]
(env) [lds@llm llm]$ pwd
/home/lds/Works/llm
(env) [lds@llm llm]$
(env) [lds@llm llm]$ pip install streamlit
Collecting streamlit
Downloading streamlit-1.34.0-py2.py3-none-any.whl (8.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.5/8.5 MB 2.7 MB/s eta 0:00:00
Collecting altair<6,>=4.0
Using cached altair-5.3.0-py3-none-any.whl (857 kB)
…
Installing collected packages: pytz, watchdog, tzdata, tornado, toolz, toml, smmap, six, rpds-py, pyarrow, protobuf, pillow, cachetools, blinker, referencing, python-dateutil, pydeck, gitdb, pandas, jsonschema-specifications, gitpython, jsonschema, altair, streamlit
Successfully installed altair-5.3.0 blinker-1.8.1 cachetools-5.3.3 gitdb-4.0.11 gitpython-3.1.43 jsonschema-4.22.0 jsonschema-specifications-2023.12.1 pandas-2.2.2 pillow-10.3.0 protobuf-4.25.3 pyarrow-16.0.0 pydeck-0.9.0 python-dateutil-2.9.0.post0 pytz-2024.1 referencing-0.35.1 rpds-py-0.18.0 six-1.16.0 smmap-5.0.1 streamlit-1.34.0 toml-0.10.2 toolz-0.12.1 tornado-6.4 tzdata-2024.1 watchdog-4.0.0
[notice] A new release of pip available: 22.3.1 -> 24.0
[notice] To update, run: pip install --upgrade pip
(env) [lds@llm llm]$ pip install langchain
…
(env) [lds@llm llm]$ pip install unstructured
…
(env) [lds@llm llm]$ pip install unstructured[pdf]
4.2. Setting
4.2.1. main.py - LANGSERVE_ENDPOINT
* langserve 접근 정보를 확인하여 수정적용
* cd example
vi main.py
…
# 2) LocalHost 접속: 끝에 붙는 N4XyA 는 각자 다르니
# http://localhost:8000/llm/playground 에서 python SDK 에서 확인!
# LANGSERVE_ENDPOINT = "http://localhost:8000/llm/c/N4XyA"
LANGSERVE_ENDPOINT = "http://192.168.50.130:8000/llm/c/N4XyA"
…
(env) [lds@llm example]$ pwd
/home/lds/Works/llm/example
(env) [lds@llm example]$ vi main.py
(env) [lds@llm example]$
*
4.3. Run
4.3.1. Run - streamlit
streamlit run main.py
[lds@llm ~]$ cd /home/lds/Works/llm/example
[lds@llm example]$ pwd
/home/lds/Works/llm/example
[lds@llm example]$ source ../env/bin/activate
(env) [lds@llm example]$
(env) [lds@llm example]$ pwd
/home/lds/Works/llm/example
(env) [lds@llm example]$
(env) [lds@llm example]$ streamlit run main.py
Collecting usage statistics. To deactivate, set browser.gatherUsageStats to false.
You can now view your Streamlit app in your browser.
Network URL: http://192.168.50.130:8501
External URL: http://1.235.192.217:8501
4.3. Check - Browser
* Browser
* http://192.168.50.130:8501
'프로그램 활용 > 인공지능(AI)' 카테고리의 다른 글
| OLLAMA + LLAMA3 + RAG + Vector Database (Local, Open Source, Free) (1) | 2024.12.05 |
|---|---|
| Build Your RAG with Verba+Ollama and Talk to Your Own Data (0) | 2024.12.05 |
| Ollama와 LangChain으로 RAG 구현하기 (with Python) (2) | 2024.12.05 |
| LLaMA3 을 이용한 RAG 구축 + Ollama 사용법 정리 (1) | 2024.12.05 |
| AI Ollama 설치와 운영 : 로컬 환경에서 대규모 언어 모델을 쉽게 실행하기 (0) | 2024.12.05 |
