"tests/git@developer.sourcefind.cn:chenpangpang/diffusers.git" did not exist on "4e59bcc680ba4f68bc8d45249db5fe7a078413db"
Commit 47d514b4 authored by zhanggzh's avatar zhanggzh
Browse files

Update README.md

parent 6157f0c0
# Faiss # <div align="center"><strong>MMCV</strong></div>
## 简介
Faiss is a library for efficient similarity search and clustering of dense vectors. It contains algorithms that search in sets of vectors of any size, up to ones that possibly do not fit in RAM. It also contains supporting code for evaluation and parameter tuning. Faiss is written in C++ with complete wrappers for Python/numpy. Some of the most useful algorithms are implemented on the GPU. It is developed by [Facebook AI Research](https://research.fb.com/category/facebook-ai-research-fair/). Faiss 是一个用于高效相似性搜索和密集向量聚类的库。它提供了在各种规模的向量集合中进行搜索的算法,包括那些可能无法完全适应 RAM 的大型集合。除了搜索功能,Faiss 还包含用于评估和参数调优的支持代码
## 安装
## News 组件支持
+ Python 3.7、3.8、3.9、3.10
See [CHANGELOG.md](CHANGELOG.md) for detailed information about latest features.
### 1、使用pip方式安装
## Introduction mmcv whl包下载目录:[http://10.6.10.68:8000/debug/faiss/dtk-24.04.1/](http://10.6.10.68:8000/debug/faiss/dtk-24.04.1/),选择对应的pytorch版本和python版本下载对应faiss的whl包
```shell
Faiss contains several methods for similarity search. It assumes that the instances are represented as vectors and are identified by an integer, and that the vectors can be compared with L2 (Euclidean) distances or dot products. Vectors that are similar to a query vector are those that have the lowest L2 distance or the highest dot product with the query vector. It also supports cosine similarity, since this is a dot product on normalized vectors. pip install faiss* (下载的mmcv的whl包)
Most of the methods, like those based on binary vectors and compact quantization codes, solely use a compressed representation of the vectors and do not require to keep the original vectors. This generally comes at the cost of a less precise search but these methods can scale to billions of vectors in main memory on a single server.
The GPU implementation can accept input from either CPU or GPU memory. On a server with GPUs, the GPU indexes can be used a drop-in replacement for the CPU indexes (e.g., replace `IndexFlatL2` with `GpuIndexFlatL2`) and copies to/from GPU memory are handled automatically. Results will be faster however if both input and output remain resident on the GPU. Both single and multi-GPU usage is supported.
## Building
The library is mostly implemented in C++, with optional GPU support provided via CUDA, and an optional Python interface. The CPU version requires a BLAS library. It compiles with a Makefile and can be packaged in a docker image. See [INSTALL.md](INSTALL.md) for details.
## How Faiss works
Faiss is built around an index type that stores a set of vectors, and provides a function to search in them with L2 and/or dot product vector comparison. Some index types are simple baselines, such as exact search. Most of the available indexing structures correspond to various trade-offs with respect to
- search time
- search quality
- memory used per index vector
- training time
- need for external data for unsupervised training
The optional GPU implementation provides what is likely (as of March 2017) the fastest exact and approximate (compressed-domain) nearest neighbor search implementation for high-dimensional vectors, fastest Lloyd's k-means, and fastest small k-selection algorithm known. [The implementation is detailed here](https://arxiv.org/abs/1702.08734).
## Full documentation of Faiss
The following are entry points for documentation:
- the full documentation, including a [tutorial](https://github.com/facebookresearch/faiss/wiki/Getting-started), a [FAQ](https://github.com/facebookresearch/faiss/wiki/FAQ) and a [troubleshooting section](https://github.com/facebookresearch/faiss/wiki/Troubleshooting) can be found on the [wiki page](http://github.com/facebookresearch/faiss/wiki)
- the [doxygen documentation](https://facebookresearch.github.io/faiss) gives per-class information
- to reproduce results from our research papers, [Polysemous codes](https://arxiv.org/abs/1609.01882) and [Billion-scale similarity search with GPUs](https://arxiv.org/abs/1702.08734), refer to the [benchmarks README](benchs/README.md). For [
Link and code: Fast indexing with graphs and compact regression codes](https://arxiv.org/abs/1804.09996), see the [link_and_code README](benchs/link_and_code)
## Authors
The main authors of Faiss are:
- [Hervé Jégou](https://github.com/jegou) initiated the Faiss project and wrote its first implementation
- [Matthijs Douze](https://github.com/mdouze) implemented most of the CPU Faiss
- [Jeff Johnson](https://github.com/wickedfoo) implemented all of the GPU Faiss
- [Lucas Hosseini](https://github.com/beauby) implemented the binary indexes
## Reference
Reference to cite when you use Faiss in a research paper:
``` ```
@article{JDH17, ### 2、使用源码编译方式安装
title={Billion-scale similarity search with GPUs}, 基于光源pytorch基础镜像环境:镜像下载地址:[https://sourcefind.cn/#/image/dcu/pytorch](https://sourcefind.cn/#/image/dcu/pytorch),根据pytorch、python、dtk及系统下载对应的镜像版本。
author={Johnson, Jeff and Douze, Matthijs and J{\'e}gou, Herv{\'e}}, #### 源码编译安装
journal={arXiv preprint arXiv:1702.08734}, - 代码下载
year={2017} ```shell
} git clone http://10.6.10.68/dcutoolkit/deeplearing/faiss.git # 根据编译需要切换分支
``` ```
- 提供2种源码编译方式(进入faiss目录):
## Join the Faiss community ```
1. 编译whl包并安装
For public discussion of Faiss or for questions, there is a Facebook group at https://www.facebook.com/groups/faissusers/ pip3 install wheel
pip3 install pytest
We monitor the [issues page](http://github.com/facebookresearch/faiss/issues) of the repository. mkdir -p build
You can report bugs, ask questions, etc. cd build
source /opt/dtk/env.sh
## License export BUILD_DIR=`pwd`
export LD_LIBRARY_PATH=${BUILD_DIR}/faiss/gpu/test/:${BUILD_DIR}/faiss/:${BUILD_DIR}/lib/:$LD_LIBRARY_PATH
Faiss is MIT-licensed. CXX=hipcc CC=hipcc cmake -DFAISS_ENABLE_GPU=ON -DBUILD_SHARED_LIBS=ON -DFAISS_OPT_LEVEL=avx -DBUILD_TESTING=ON -DCMAKE_BUILD_TYPE=RELEASE -DCMAKE_MODULE_PATH="${ROCM_PATH}/hip/cmake" -DCMAKE_INSTALL_PREFIX=${BUILD_DIR}/install -DMKL_LIBRARIES=/opt/intel/compilers_and_libraries_2020.4.304/linux/mkl/lib/intel64_lin/libmkl_rt.so ..
make
cd faiss/python
python3 setup.py bdist_wheel
cd dist
pip3 install faiss-* --no-deps --force
验证安装:
python3
Python 3.10.12 (main, May 26 2024, 00:14:02) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import faiss
>>> faiss.__version__
'1.7.2'
>>> faiss.__dcu_version__
'1.7.2+das.dtk24041'
>>>
2. 源码编译安装
mkdir -p build
cd build
source /opt/dtk/env.sh
export BUILD_DIR=`pwd`
export LD_LIBRARY_PATH=${BUILD_DIR}/faiss/gpu/test/:${BUILD_DIR}/faiss/:${BUILD_DIR}/lib/:$LD_LIBRARY_PATH
CXX=hipcc CC=hipcc cmake -DFAISS_ENABLE_GPU=ON -DBUILD_SHARED_LIBS=ON -DFAISS_OPT_LEVEL=avx -DBUILD_TESTING=ON -DCMAKE_BUILD_TYPE=RELEASE -DCMAKE_MODULE_PATH="${ROCM_PATH}/hip/cmake" -DCMAKE_INSTALL_PREFIX=${BUILD_DIR}/install -DMKL_LIBRARIES=/opt/intel/compilers_and_libraries_2020.4.304/linux/mkl/lib/intel64_lin/libmkl_rt.so ..
make && make install
```
## 单测
```
cd faiss/tests
pytest -vs
cd faiss/build
ctest
```
## Known Issue
-
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment