V2EX = way to explore
V2EX 是一个关于分享和探索的地方
现在注册
已注册用户请  登录
V2EX 提问指南
baoyinhe
V2EX  ›  问与答

现在存储向量数据库都用什么,哪种是未来的趋势?

  •  
  •   baoyinhe · 2023-05-06 12:42:25 +08:00 · 1468 次点击
    这是一个创建于 376 天前的主题,其中的信息可能已经有所发展或是发生改变。
    vivisidea
        1
    vivisidea  
       2023-05-06 14:38:43 +08:00   ❤️ 1
    刚好在隔壁看一篇关于用 openai embedding 来做检索的文章,里面有提到这个

    向量数据库选项包括:

    Pinecone ,完全托管的向量数据库
    Weaviate ,开源向量搜索引擎
    Redis ,向量数据库
    Qdrant ,向量搜索引擎
    Milvus ,用于可扩展相似性搜索的向量数据库
    Chroma ,开源的嵌入向量存储

    source
    https://zhuanlan.zhihu.com/p/619233637
    baoyinhe
        2
    baoyinhe  
    OP
       2023-05-06 16:46:38 +08:00
    @vivisidea #1 有用
    lchynn
        3
    lchynn  
       2023-05-06 17:00:16 +08:00
    好奇,redis 内存数据库来做向量数据存储,那么内存如果存到后面不够了,oom 了咋办? 生产业务允许矢量化的数据丢失么? 对应到 embedding 的 gpt 例子, 好比你要查找一堆文本有关相似性的业务数据都没了?
    vivisidea
        4
    vivisidea  
       2023-05-06 17:17:03 +08:00
    @lchynn 我没仔细看 redis 检索的实现原理,但在 kv 应用场景,一般可以控制 key 的驱逐策略,比如可以用 noeviction 策略,拒绝新的写入,或者 LRU 把最老的 evict 掉

    https://redis.io/docs/reference/eviction/

    The exact behavior Redis follows when the maxmemory limit is reached is configured using the maxmemory-policy configuration directive.

    The following policies are available:

    noeviction: New values aren’t saved when memory limit is reached. When a database uses replication, this applies to the primary database
    allkeys-lru: Keeps most recently used keys; removes least recently used (LRU) keys
    allkeys-lfu: Keeps frequently used keys; removes least frequently used (LFU) keys
    volatile-lru: Removes least recently used keys with the expire field set to true.
    volatile-lfu: Removes least frequently used keys with the expire field set to true.
    allkeys-random: Randomly removes keys to make space for the new data added.
    volatile-random: Randomly removes keys with expire field set to true.
    volatile-ttl: Removes keys with expire field set to true and the shortest remaining time-to-live (TTL) value.
    关于   ·   帮助文档   ·   博客   ·   API   ·   FAQ   ·   实用小工具   ·   1100 人在线   最高记录 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 30ms · UTC 21:59 · PVG 05:59 · LAX 14:59 · JFK 17:59
    Developed with CodeLauncher
    ♥ Do have faith in what you're doing.