Enhancing the security of LLM applications with LangChain models and NVIDIA NeMo Guardrails

According to NVIDIA’s tech blog, developers who want to deploy Large Language Model (LLM) applications more securely and quickly now have a robust solution with LangChain Templates and NVIDIA NeMo Guardrails.

Benefits of integrating NeMo guardrails with LangChain models

LangChain templates offer developers a new way to create, share, manage, download and customize LLM-based agents and chains. These templates enable the rapid creation of production-ready applications by leveraging FastAPI for seamless API development in Python. NVIDIA NeMo Guardrails can be integrated into these templates to provide content moderation, enhanced security, and LLM response evaluation.

As generative AI continues to evolve, integrating guardrails ensures that LLMs used in enterprise applications remain accurate, secure, and contextually relevant. The NeMo Guardrails platform provides programmable rules and runtime integration to check user inputs before engaging with LLM and to validate the final LLM output.

Setting up the use case

To demonstrate the integration, the blog post explores a Retrieval-Augmented Generation (RAG) use case using an existing LangChain model. The process involves downloading the template, modifying it to fit your specific use case, and then deploying the application with guardrails added to ensure security and accuracy.

LLM guardrails help minimize hallucinations and keep data safe by implementing self-monitoring input and output binaries that mask sensitive data or recast user inputs. For example, dialog binaries can affect how LLMs respond, and retrieval binaries can mask sensitive data in RAG applications.

Download and customize the LangChain model

To get started, developers need to install the LangChain CLI and the NVIDIA AI Foundation Endpoints LangChain package. The template can be downloaded and customized by creating a new application project:

pip install -U langchain-cli
pip install -U langchain_nvidia_aiplay
langchain app nvidia_rag_guardrails --package nvidia-rag-canonical

The downloaded model sets up an ingestion pipeline into a Milvus vector database. In this example, the dataset contains sensitive information related to Social Security benefits, making guardrail integration crucial for secure responses.

Integration of NeMo guardrails

To integrate NeMo Guardrails, developers need to create a directory named guard rail and configure the necessary files as config.yml, disallowed.co, general.coAND prompts.yml. These configurations define guardrail flows that control the chatbot’s behavior and ensure that it adheres to predefined rules.

For example, a disallowed flow might prevent the chatbot from responding to misinformation, while a general flow might define acceptable topics. Self-checks are also implemented for user inputs and LLM outputs to prevent cybersecurity attacks such as prompt injection.

Activation and use of the model

To enable guardrails, developers must include the configurations in the file config.yml files and configure the server for API access. The following code snippets show how to integrate guardrails and configure the server:

from nvidia_guardrails_with_RAG import chain_with_guardrails as nvidia_guardrails_with_RAG_chain
add_routes(app, nvidia_guardrails_with_RAG_chain, path="/nvidia-guardrails-with-RAG")

from nvidia_guardrails_with_RAG import ingest as nvidia_guardrails_ingest
add_routes(app, nvidia_guardrails_ingest, path="/nvidia-rag-ingest")

Developers can then launch the LangServe instance with the command:

langchain serve

An example of a secure LLM interaction might look like this:

"Question": "How many Americans receive Social Security Benefits?" 
"Answer": "According to the Social Security Administration, about 65 million Americans receive Social Security benefits."

Conclusion

This integration of NeMo Guardrails with LangChain Templates demonstrates a robust approach to building more secure LLM applications. By adding security measures and ensuring accurate responses, developers can create reliable and secure AI applications.

Image source: Shutterstock

. . .

Tags

Fuente