Gen AI Privacy: Storing PII efficiently in Vector DB using FPE

In our previous blog Gen AI Data Privacy, we demonstrated the practical applications of Langchain Document Loader. In this installment, we delve into the seamless integration of this tool with Vectordb, a popular database solution. Additionally, we will explore its compatibility with blockchain technology, ensuring secure and private access to Language Model (LLM) responses.

Technology Stack

• Langchain (Optional)

• CIPH3R FPE Tokenizer/Detokenizer for GenAI

• CIPH3R API

• OpenAI API (Choose Any)

• VectorDB (Choose Any)

Allow me to introduce Retrieval-Augmented Generation (RAG), a sophisticated design framework designed to seamlessly incorporate a company’s proprietary and confidential data alongside Language Model (LLM) capabilities. RAG operates by enriching LLM’s knowledge repository through a fusion of retrieval and response generation techniques. By integrating public, external data sources with internal proprietary information, RAG facilitates the creation of secure and confidential responses while enhancing the depth and accuracy of LLM-generated answers.

Exploring the integration of RAG with CIPH3R solution unveils a robust approach to encrypting Personally Identifiable Information (PII) into Format Preserving Encryption (FPE) format. This enables the utilization of tokenized data, which can be appropriately detokenized as needed. Within privacy settings, RAG emerges as an indispensable architectural framework for orchestrating private data management. By seamlessly amalgamating diverse data sources, RAG ensures the attainment of desired outputs while upholding stringent privacy standards.

In a question-answering scenario, RAG excels at retrieving essential details from pertinent sources. It adeptly crafts responses by accessing encrypted Personally Identifiable Information (PII) data stored in Vectordb, leveraging a synergistic combination of CIPH3R tokenizer and CIPH3R API components on unstructured data sources. This strategic approach ensures the seamless integration of disparate sources while maintaining data security and privacy protocols.

This process enriches Language Models (LLMs) with supplementary information, enabling the provision of context-sensitive responses that surpass those generated by conventional public models.

Connect with us, and we’ll provide a demo illustrating the seamless integration and utilization of CIPH3R to safeguard a company’s private data through the RAG design in conjunction with VectorDB.