When generating vector embeddings outside the database, what is the most suitable option for storing the embeddings for later use?

When vector embeddings are generated outside the database, the storage choice must balance efficiency, scalability, and usability for similarity search. A CSV file (A) is simple and human-readable but inefficient for large-scale vector operations due to text parsing overhead and lack of indexing support. A binary FVEC file (B) offers a compact format for vectors, reducing storage size and improving read performance, but separating relational data into a CSV complicates integration and querying, making it suboptimal for unified workflows. Storing embeddings as BLOBs in a relational database (C) integrates well with structured data and supports SQL access, but it lacks the specialized indexing (e.g., HNSW, IVF) and query optimizations that dedicated vector databases provide. A dedicated vector database (D), such as Milvus or Pinecone (or Oracle 23ai’s vector capabilities if internal), is purpose-built for high-dimensional vectors, offering efficient storage, advanced indexing, and fast approximate nearest neighbor (ANN) searches. For external generation scenarios, where embeddings are not immediately integrated into Oracle 23ai, a dedicated vector database is the most suitable due to its performance and scalability advantages. Oracle’s AI Vector Search documentation indirectly supports this by emphasizing optimized vector storage for search efficiency, though it focuses on in-database solutions.
Reference:Oracle Database 23ai AI Vector Search Guide, Chapter on Vector Storage and Indexing.