Version: 1.0
Release Date: Sep 2025
Copyright: © 2025 Advantech Corporation. All rights reserved.
LLM MLC-LLM AI Agent RAG on Qualcomm® Adreno™ delivers an optimized, on-device AI inference solution designed for privacy-preserving, low-latency deployments at the edge. It leverages MLC-compiled LLMs and OpenCL acceleration to efficiently execute Meta Llama 3.2 models 3B directly on Qualcomm® QCS6490™ platforms. The container integrates a FastAPI-based middleware exposing an OpenAI-compatible completions API and the MLC TVM runtime for flexible compute distribution on GPU (OpenCL). Together with OpenWebUI for interactive chat experiences, it forms a complete local inference stack supporting Retrieval-Augmented Generation (RAG), and custom LLM workflows. This container demonstrates how MLC LLM enables edge-native, high-performance generative AI deployments on Qualcomm® hardware.
| Feature | Description |
|---|---|
| MLC TVM Runtime | Optimized compiler + runtime stack for running quantized models with GPU/HTP acceleration. |
| OpenCL Support | Runs MLC LLM models on Adreno GPU |
| QNN & SNPE Acceleration | Added support for execution of quantized .bin models on Hexagon DSP v68. |
| OpenWebUI | Clean, browser-based interface for interactive chat. |
| OpenAI-Compatible API | REST API for /chat/completions, works with OpenWebUI, LangChain, etc. |
| Streaming Output | Real-time token-by-token streaming in both CLI and API. |
| RAG Ready Environment | Can be extended to build Retrieval-Augmented Generation (RAG) and Tool-Augmented Agents on-device using LangChain. |
| LangChain Integration | Multi-turn memory with ConversationChain support |
| Offline Deployment | Fully offline after container setup and model copy. |
| Flexible Parameters | Supports temperature, top_p, repetition_penalty, frequency_penalty, presence_penalty, max_num_tokens |
| Component | Specification |
|---|---|
| Device | Advantech AOM-2721 |
| SoC | Qualcomm® QCS6490™ (soc_id-35) |
| GPU | Adreno™ 643 (OpenCL backend supported) |
| DSP/HTP | Hexagon™ 770 v68 with tensor accelerator |
| Memory | 8GB LPDDR5 |
This container is intended for QCOM Robotics Reference Distro with ROS, version 1.3-ver.1.1 OS running on QCS6490 device.
| Environment | OS |
|---|---|
| Device Host | QCOM Robotics Reference Distro with ROS 1.3-ver.1.1 |
| Container | Ubuntu 22.04 LTS |
| Component | Version | Description |
|---|---|---|
| MLC LLM Runtime | 0.1.dev0 | Provides TVM-compiled LLM runtime for GPU/HTP |
| Apache TVM | 0.14+ | Compiler stack used by MLC to generate optimized kernels |
| QNN SDK | 2.32.0 | Qualcomm Neural Network runtime for quantized models |
| SNPE | 2.32.0 | Snapdragon Neural Processing Engine |
The following software components/packages are provided further as a part of this image:
| Component | Version | Description |
|---|---|---|
| Python | 3.10.12 | For FastAPI + backend scripts |
| LangChain | 0.2.17 | Installed via PIP, framework to build LLM applications |
| FastAPI | 0.116.1 | REST API server for OpenAI-compatible endpoints |
| OpenWebUI | 0.6.5 | Lightweight frontend for chat |
| Uvicorn | Latest | ASGI server for FastAPI |
| FAISS | 1.8.0.post1 | Vector store backend for enabling RAG with efficient similarity search |
| RAG Code Sample | NA | Sample code that shows RAG capability development |
| Sentence-T5-Base | NA | Pulls sentence-t5-base embedding model from HF |
| Attribute | Details |
|---|---|
| Supported Format | |
| File Type | Text-based documents only (scanned or image-based PDFs are not supported). Table data within supported PDFs can also be read and processed |
| Recommended File Size | While files up to 30 MB (approximately ~1000 pages, ~250,000 words) have been tested, performance may degrade with larger or more complex documents. |
| Unsupported Formats | Scanned/image-only PDFs, OCR-intensive documents, Word documents, CSV or text files, and encrypted PDFs |
| Upload Method | PDF upload via the UI is currently not supported. Please place files directly in the langchain-rag-service/pdf directory |
| Multi-file Support | Multiple PDFs can be ingested simultaneously. However, it is recommended to avoid documents with overlapping, redundant, or irrelevant content, as this may reduce retrieval accuracy and lead to inconsistent responses |
| Language Support | Currently supports English-language documents only |
| Model | Format | Notes |
|---|---|---|
| Meta Llama 3.2 1B | .bin |
Converted with MLC TVM |
| Meta Llama 3.2 3B | .bin |
Converted with MLC TVM |
Note: This is a non-exhaustive list of supported models that have been tested. There would be more models compatible with MLC TVM on QCS6490, and users are encouraged to explore further.
| Runtime | Format |
|---|---|
| MLC Runtime | .bin |
| QNN | .bin |
| SNPE | .dlc |
LLM-MLC-LLM-AI-Agent-RAG-on-Qualcomm-Adreno/
├── .env # Environment configuration
├── Model_Conversion_Guide.md # Instructions for converting models with MLC-TVM
├── efficient-prompting-for-compact-models.md # Craft better prompts for small and quantized language models
├── README.md # Overview and quick start steps
├── build.sh # Build script
├── docker-compose.yml # Docker Compose setup
├── start_service.sh # Script to start MLC LLM API Service
├── wise-bench.sh # Script to verify acceleration and software stack inside container
├── windows-git-setup.md # Steps to fix LF/CRLF issues on windows while copying to device
├── data # Contains subfolders for assets like images, gifs etc.
└── langchain-rag-service/ # Core LangChain RAG API service
├── pdfs/ # Folder to store pdf documents
| └── EdgeSync.pdf # Sample PDF i.e. EdgeSync.pdf, contains basic info about EdgeSync
├── app.py # Main LangChain-FastAPI app
├── llm_loader.py # LLM loader (Ollama, DeepSeek, etc.)
├── rag_utils.py # RAG helper functions like load pdf, split, etc.
├── requirements.txt # Python dependencies
├── schema.py # Request schema helper
├── utils.py # Utility functions helper
└── start_services.sh # Startup script
build.sh launches the following containers:
-
llm-mlc-llm-ai-agent-rag-on-qualcomm-adreno: Runs MLC runtime for hardware-accelerated inference using OpenCL.
-
openweb-ui-service: Web UI for chat interactions.
This container delivers a Retrieval-Augmented Generation (RAG)-ready deployment of Meta Llama 3.2 3B on Qualcomm® QCS6490™ devices using the MLC runtime. Built on the Apache TVM compiler stack, it provides a fully optimized, hardware-accelerated environment supporting OpenCL (GPU) backend for efficient on-device inference. The container includes a FastAPI-based orchestration layer exposing an OpenAI-compatible API, and can be easily extended with LangChain for RAG and agent workflows. Designed for offline, low-latency operation, it enables privacy-preserving and real-time retrieval-based reasoning at the edge.
| Feature | Description |
|---|---|
| Local Inference Engine | The mlc_cli_chat tool is provided to run text-to-text inference on the compiled Llama 3.2 3B model. It accepts a user prompt and streams the generated response. |
| Middleware Logic Engine | FastAPI-based LangChain server handles agent logic, tools, memory, and RAG pipelines. |
| RAG-Enabled | Supports Retrieval-Augmented Generation using vector stores and document loaders. |
| Agent & Tool Support | Easily define and run LangChain agents with tool integration (e.g., search, calculator). |
| OpenAI API Compatibility | FastAPI backend exposes an OpenAI-compatible /chat/completions endpoint; works seamlessly with OpenWebUI. |
| Streaming Output Support | Supports real-time token-by-token streaming for chat UIs and API responses. |
| Edge Optimized | Runs MLC LLM models efficiently on Adreno GPU (OpenCL) |
| Customizable Behavior | Configure runtime parameters (e.g., temperature, top_k, top_p, seed, max-num-tokens) in model config JSON or via API. |
| Prompt Engineering | Supports structured prompts with system, user, and assistant roles. |
| Offline-First | Works fully offline after model conversion and deployment; no cloud dependency. |
| Developer Friendly | Simple CLI (mlc_cli_chat) and Dockerized setup for quick local experimentation. |
| Easy Integration | Backend-ready for FastAPI, OpenWebUI, and custom applications. |
| AI Dev Environment | Provides a full hardware-accelerated containerized environment for on-device LLM development and testing. |
OpenWebUI serves as a lightweight, responsive frontend for interacting with LLMs deployed locally on the Qualcomm®
QCS6490™ device. In this setup, it connects directly to the FastAPI wrapper for running the mlc_llm chat interface,
which exposes an OpenAI-compatible endpoint powered by the Meta Llama 3.2 3B model. Containerizing OpenWebUI ensures a
modular, browser-accessible deployment that delivers a seamless real-time chat experience without cloud dependency,
fully optimized for on-device edge inference.
| Feature | Description |
|---|---|
| User-Friendly Interface | Sleek, chat-style UI for real-time interaction. |
| OpenAI-Compatible Backend | Works with MLC, OpenAI, and similar APIs with minimal setup. |
| Container-Ready Design | Lightweight and optimized for edge or cloud deployments. |
| Streaming Support | Enables real-time response streaming for interactive UX. |
| Authentication & Access Control | Basic user management for secure access. |
| Offline Operation | Runs fully offline with local backends like Ollama. |
| Module | Link | Description |
|---|---|---|
| Quick Start | README | Overview of the container image |
| Model Conversion | README | Instructions for converting models with MLC-TVM |
| Prompt Guidelines | README | Guidelines to craft better prompts for small and quantized language models |
| Windows Git Line Ending Setup | README | Steps to configure Git on Windows to handle LF/CRLF line endings correctly |
| SCP Troubleshooting Guide | README | Guidelines for troubleshooting issues related to SCP file transfers |
| Item | Value |
|---|---|
| Source | MLC + TVM compiled .bin model (Meta Llama 3.2 3B) |
| Architecture | Llama 3.2 3B |
| Quantization | q4f16_0 (4-bit integers with FP16 scaling, using layout variant 0) |
| Parameters | ~3B |
| Model size | ~2 GB |
| Context length | 131072 |
- Ensure documents are topically consistent and logically structured to improve semantic retrieval quality.
- Remove irrelevant sections such as watermarks, footers, or repeated headers before uploading.
- Prefer documents with clean metadata and minimal formatting clutter for better parsing and chunking.
- While table content is supported, avoid heavily stylized layouts like multi-column text or embedded visual elements.
- Avoid mixing multiple unrelated domains or topics in the same set of files, as this can confuse context-aware retrieval.
- Increase swap size if available RAM is less.
- Ask focused, document-specific prompts (e.g., "What are the features of T_CONFIG?") rather than broad or generic questions. This ensures the system retrieves answers from the uploaded PDF rather than falling back on the model’s general knowledge.
- When querying, reference document structure or terminology explicitly; this helps improve the precision of results.
- If your query returns information from an unintended document, refine your prompt to include specific terms, section names, or context unique to the desired source. You may also consider temporarily removing unrelated files for isolation.
- Restart services after every addition/deletion/change in the
pdffolder. - You can also customize score thresholding in retriever config to filter irrelevant content via the environment
variable
SCORE_THRESHOLDas per the need. - Keep persistent vector DB (e.g., FAISS saved index) to avoid re-indexing on container restart
- Use appropriate (size/precision) embedding models as per the suitability of the use case.
Note: Before starting this LLM container, ensure that no other LLM container is currently running or using the port specified in the .env file (e.g., 8000); otherwise, stop and remove the active container first. Only one LLM container should run at a time to prevent port conflicts.
Note for Windows Users:
If you are using Linux, no changes are needed — LF line endings are used by default.
If you are on Windows, please follow the steps in Windows Git Line Ending Setup before cloning to ensure scripts and configuration files work correctly on Device.
git clone https://github.com/Advantech-EdgeSync-Containers/LLM-MLC-LLM-AI-Agent-RAG-on-Qualcomm-Adreno.git
cd LLM-MLC-LLM-AI-Agent-RAG-on-Qualcomm-AdrenoIf you cloned the repo on a separate development machine, use scp to transfer only the relevant folder (refer to SCP Troubleshooting Guide if any issues faced):
# From your development machine (Ubuntu or Windows PowerShell if SCP is installed)
scp -r .\LLM-MLC-RAG-on-Qualcomm-Adreno\ <username>@<qcs6490-ip>:/home/<username>/Replace:
<username>– Login username on the QCS6490 board (e.g.,root)<qcs6490-ip>– IP address of the QCS6490 board (e.g.,192.168.1.42)
This will copy the folder to /home/<username>/LLM-MLC-RAG-on-Qualcomm-Adreno/.
Then SSH into the Qualcomm® device:
ssh <username>@<qcs6490-ip>
cd ~/LLM-MLC-RAG-on-Qualcomm-AdrenoBefore starting services, upload your PDF documents (or use the default one) to the designated directory:
# Place your PDFs in the following directory
./langchain-rag-service/pdfs/
Mandatory Pre-requisite: Before executing the build.sh script, ensure that the Meta Llama 3.2 3B model and its associated runtime library are available in the /model directory of your QCS6490 device.
This step is mandatory for successful container deployment and inference.
You have two options to ensure the model files are correctly placed on the target device:
- Convert the Model Using MLC-TVM
- Copy a Pre-Converted Model
If the model is not yet converted for the MLC framework:
- Follow the detailed steps in the Model_Conversion_Guide to generate the MLC-compatible model package.
- Once conversion is complete, transfer the generated files to your QCS6490 target under the project folder.
If you already have the converted model (or use this link to download pre-converted meta llama 3.2 3B model - Download Model):
-
Create the model directory on the target device (
QCS6490).# On the target device (QCS6490) mkdir -p /home/root/LLM-MLC-LLM-AI-Agent-RAG-on-Qualcomm-Adreno/model -
Transfer both the model folder and the
.sofile from your development machine usingscp:# From your development machine scp -r Llama3.2_3B_model_params root@<target_ip>:/home/root/LLM-MLC-LLM-AI-Agent-RAG-on-Qualcomm-Adreno/model/ scp llama3.2-3b-instruct-q4f16_0-adreno-iot.so root@<target_ip>:/home/root/LLM-MLC-LLM-AI-Agent-RAG-on-Qualcomm-Adreno/model/
# Make the build script executable
chmod +x build.sh
# Launch the container
./build.shchmod +x wise-bench.sh
./wise-bench.sh
Wise-bench logs are saved in the wise-bench.log file under /workspace
These example prompts demonstrate how users can query technical documents related to EdgeSync (i.e., EdgeSync.pdf).
The RAG container will retrieve relevant context and generate a meaningful response from the source document. Users can
extend this RAG example container for their own documents and modify their prompts accordingly.
| S.No | Prompt |
|---|---|
| 1 | Please summarize in detail about the EdgeSync API. |
| 2 | What are the pillars of EdgeSync? |
| 3 | Please describe EdgeSync containers. |
| 4 | What can developers do with EdgeSync APIs? |
| 5 | What developers can do with EdgeSync containers? |
# Make the start service script executable
chmod +x start_services.sh
# Launch the start services
./start_services.shAllow some time for the OpenWebUI and LLM MLC RAG on Qualcomm® Adreno™ container to
settle and become healthy. Since this will also download the embedding model & vectorize the PDF(s), please allow some time (depending on internet speed & PDF size) to get all services started successfully. Wait until uvicorn starts serving, and confirm
via uvicorn.log.
Exit from the container and run following command to check the status of the containers:
docker psAllow some time for containers to become healthy.
Access OpenWebUI via any browser using the URL given below. Create an account and perform login:
http://localhost_or_QCS6490_IP:3000
The MLC runtime supports fine-tuning of runtime parameters such
as temperature, top-k, top-p, repetition_penalty, frequency_penalty, presence_penalty, and maximum token
length.
These parameters can be configured in the mlc_chat_config.json file or passed dynamically through the API request.
By customizing these values, you can optimize the model for a wide range of use cases—from deterministic outputs for testing and reproducibility to creative, diverse generations for conversational AI.
Below is an example section from the mlc_chat_config.json file with tuned values for better quality and reduced
repetition:
},
"vocab_size": 128256,
"context_window_size": 131072,
...
"temperature": 0.8,
"top_p": 0.9,
"repetition_penalty": 1.15,
"frequency_penalty": 0.7,
"presence_penalty": 0.5
"tokenizer_files": [
"tokenizer.json",
"tokenizer_config.json"
],
...This README provides essential prompt guidelines to help you get accurate and reliable outputs from small and quantized language models.
The MLC LLM engine provides an option to enable event logging for requests, which can be useful for debugging and
monitoring. You can enable tracing with the --enable-tracing argument for more advanced logging user can enable this.
Currently we are logging api service logs in below files:
| Log File | Description |
|---|---|
| uvicorn.pid | Provides process-id for the currently running uvicorn service |
| uvicorn.log | Provides uvicorn service logs |
Here are quick commands/instructions to troubleshoot issues with running a model using mlc llm ,**mlc_cli_chat
**, FastAPI, uvicorn and OpenWebUI:
-
View uvicorn service logs within the container
tail -f uvicorn.log
-
Make sure that you are in
mlc-venvenvironment if not then use below command to activate themlc-venv.conda activate mlc-venv
-
Verify that the model and all its dependencies are available for
mlc llm. Ensure the model is loaded correctly for execution using themlc. Run the following command inside the Docker container.mlc_llm serve --help
-
If MLC Model is not running then check the Environment Variables
echo $MLC_MODEL_PATH echo $MODEL_LIB
These should match with the following as per the converted model files path:
MLC_MODEL_PATH=/workspace/model/Llama3.2_3B_model_params MODEL_LIB=/workspace/model/llama-3.2-3b-instruct-q4f16_0-adreno-iot.so
-
Verify that the model directory contains all required files in the correct folder structure — such as the MLC Chat Config file, tokenizer configuration file, and the tokenizer JSON file. Below is a reference example of the files typically found inside the model folder.
(mlc-venv) root@qcs6490aom2721a1:/workspace/model Llama3.2_3B_model_params# ls mlc-chat-config.json params_shard_1.bin params_shard_12.bin params_shard_15.bin params_shard_18.bin params_shard_20.bin params_shard_4.bin params_shard_7.bin tokenizer.json ndarray-cache.json params_shard_10.bin params_shard_13.bin params_shard_16.bin params_shard_19.bin params_shard_21.bin params_shard_5.bin params_shard_8.bin tokenizer_config.json params_shard_0.bin params_shard_11.bin params_shard_14.bin params_shard_17.bin params_shard_2.bin params_shard_3.bin params_shard_6.bin params_shard_9.bin
-
If OpenWebUI chat is not returning a response, use the following
curlcommand to test the API directly. If the response shows an error message (e.g., model file or lib missing or any other error), take the necessary action based on the error.curl -X POST \ -H "Content-Type: application/json" \ -d '{ "model": "/workspace/model/Llama3.2_3B_model_params", "messages": [ {"role": "user", "content": "What is the capital of France?"} ], "stream": false }' \ http://192.168.29.121:8000/v1/chat/completions
-
Kill & restart services within container (check pid manually via
ps -eafor use pid stored inuvicorn.pid)kill $(cat uvicorn.pid) ./start_services.sh
Confirm there is no service running using:
ps -eaf
Here's a simple CLI example for running inference using mlc_cli_chat. This command invokes the MLC LLM engine to
execute the Meta Llama 3.2 3B model using a specified model, library file and device (OpenCL) , followed by a
user-defined prompt.
Inside the container /workspace/mlc-llm/build/apps/mlc_cli_chat/
./mlc_cli_chat --model /workspace/model/Llama3.2_3B_model_params/ --model-lib /workspace/model/llama3.2-3b-instruct-q4f16_0-adreno-iot.so --device opencl --with-prompt "What is the capital of France"sample output:
this command will invoke a mlc cli chat application using OpenCL for GPU backend and provide user and chat
interface for inference.
For more details user can run below command:
./mlc_cli_chat --helpsample output:
user can also check the model inference stats using the /stats in the mlc cli chat terminal
- Model Placement: Ensure models are fully loaded into the DSP/HTP memory or GPU VRAM for optimal inference performance.
- Batch Inference: Use batch inference when running multiple requests to improve throughput efficiency.
- Dynamic Memory Offloading: Offload unused models from DSP/HTP/GPU memory when not needed to free up resources for active workloads.
- Quantization Preference: Prefer quantized models (e.g., INT8, Q4F16) to balance speed, memory usage, and accuracy.
- Context Length Tuning: Reduce the maximum context length when possible to minimize memory usage without impacting task quality.
- Token Management: override
max_tokensto avoid unnecessarily long generations that increase latency and memory consumption. - Model Size Guidance: For best performance on QCS6490, use models with ≤3B parameters.
If you’ve launched the FastAPI backend inside the container (./start_services.sh), you can call it with curl:
Inference Request:
curl -X POST \
-H "Content-Type: application/json" \
-d '{
"model": "/workspace/model/Llama3.2_3B_model_params",
"messages": [
{"role": "user", "content": "What is the capital of France?"}
],
"stream": false
}' \
http://192.168.29.121:8000/v1/chat/completionsHere stream mode could be changed as true/false as per the needs.
Response:
{"id":"chatcmpl-c36344de9e30405f814bb6bc71cf4b6b","choices":[{"finish_reason":"stop","index":0,"message":{"content":"The capital of France is Paris.","role":"assistant","name":null,"tool_calls":null,"tool_call_id":null},"logprobs":null}],"created":1758198358,"model":"/workspace/model/Llama3.2_3B_model_params","system_fingerprint":"","object":"chat.completion","usage":{"prompt_tokens":32,"completion_tokens":8,"total_tokens":40,"extra":null}}Sample Screenshot:
- Use asynchronous chains and streaming response handlers to reduce latency in FastAPI endpoints.
- For RAG pipelines, use efficient vector stores (e.g., FAISS with cosine or inner product) and pre-filter data when possible.
- Avoid long chain dependencies; break workflows into smaller composable components.
- Cache prompt templates and tool results when applicable to reduce unnecessary recomputation
- For agent-based flows, limit tool calls per loop to avoid runaway execution or high memory usage.
- Log intermediate steps (using LangChain’s callbacks) for better debugging and observability
- Use models with ≥3B parameters (e.g., Llama 3.2 3B or larger) for agent development to ensure better reasoning depth and tool usage reliability.
-
OpenWebUI Dependencies
On the first startup, OpenWebUI installs certain dependencies. These are persisted in the associated Docker volume, so allow some time for this one-time setup to complete before use. -
Model Compilation
Models must be explicitly converted for execution. Always verify the quantization format and device compatibility before running a model. -
Model Size Restrictions
Models larger than 3B parameters may not run efficiently on QCS6490 due to memory constraints. -
Context Length Limitations
Very long context lengths can exceed memory limits, leading to errors or performance degradation. Adjustmax_tokensand context size accordingly. -
Docker Storage Constraints
Running inside Docker containers can quickly consume disk space due to model weights, logs, and cache. Ensure sufficient storage is available on the device. -
Streaming Support
While streaming improves responsiveness, it can cause higher memory pressure if multiple clients are connected simultaneously. -
Start Time: As the embedding model is downloaded by the container during first-time startup, it may take some time depending on the internet speed. Allow the container service to settle and start using it once logs related to the successful start of the application appear in the log file. The embedding model is just downloaded one time.
-
Domain-Specific Prompts: The container handles PDF document-specific prompts very well. If the user intends the same for other general/domain-specific prompts, it is recommended to use models with higher parameters.
Leverage the container image to build interesting use cases like
-
Legal Document Assistant: Query contracts, case law, or internal legal memos without exposing sensitive legal data to the cloud.
-
Internal SOP Assistant: Build a smart assistant for internal Standard Operating Procedures (SOPs) to help employees follow the correct steps across various department operations.
-
Medical Protocol Access (Offline): Offer doctors and staff instant, voice-accessible retrieval from medical guidelines, drug data, and SOPs, even in low-connectivity zones
-
Compliance and Audit Q&A: Run offline LLMs trained on local policy or compliance data to assist with audits or generate summaries of regulatory alignment—ensuring data never leaves the premises.
-
Safety Manual Conversational Agents: Deploy LLMs to provide instant answers from on-site safety manuals or procedures, reducing downtime and improving adherence to protocols.
-
Technician Support Bots: Field service engineers can interact with the bot to troubleshoot equipment based on past repair logs, parts catalogs, and service manuals.
-
Smart Edge Controllers: LLMs can translate human intent (e.g., “reduce line 2 speed by 10%”) into control commands for industrial PLCs or middleware using AI agents.
-
Conversational Retrieval (RAG): Extend the container capabilities for developing use cases around RAGs. The container already provides a working sample.
-
Tool-Enabled Agents: Create intelligent agents that use calculators, APIs, or search tools as part of their reasoning process—LangChain handles the logic and LLM interface.
Copyright © 2025 Advantech Corporation. All rights reserved.