Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Applying evaluation framework #89

Open
wants to merge 17 commits into
base: main
Choose a base branch
from
Open

Applying evaluation framework #89

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
93951bc
Some updates adding multilingual-e5-large embedding model
bukosabino May 24, 2024
7c1064f
Some updates adding multilingual-e5-large embedding model
bukosabino May 28, 2024
090b01a
Some updates adding multilingual-e5-large embedding model
bukosabino May 28, 2024
3420af9
Some updates adding multilingual-e5-large embedding model
bukosabino May 28, 2024
8f29f4f
Some updates adding multilingual-e5-large embedding model
bukosabino May 28, 2024
cafece2
Fixing bug with timeit
bukosabino Jun 7, 2024
794411c
Update to use multilingual-e5-large in all the collections
bukosabino Jun 7, 2024
7e8f526
Adding the evaluation notebooks
bukosabino Jun 11, 2024
803ef7a
Merge branch 'main' into develop-adding-multilingual-e5-large
bukosabino Jul 12, 2024
596f248
update to run evaluation
bukosabino Jul 12, 2024
055d0ac
adding bge-m3 sas metric
bukosabino Jul 12, 2024
6232168
update qdrant internal doc
bukosabino Jul 12, 2024
741e5a1
Merge branch 'main' into develop-adding-multilingual-e5-large
bukosabino Jul 12, 2024
df10a57
minor stuff
bukosabino Jul 12, 2024
bb20743
Update with SAS metric for bge-m3
bukosabino Jul 17, 2024
f5ae641
Update with SAS metric for bge-m3
bukosabino Jul 17, 2024
f46f11c
Update with SAS metric for bge-m3
bukosabino Jul 17, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions config/config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ chunk_overlap: 100

admin_email: bukosabino@gmail.com

embeddings_model_name: dariolopez/roberta-base-bne-finetuned-msmarco-qa-es-mnrl-mn
embeddings_model_size: 768
embeddings_model_name: intfloat/multilingual-e5-large
embeddings_model_size: 1024

vector_store: 'qdrant' # {'qdrant', 'pinecone', 'supabase'}
top_k_results: 10
Expand All @@ -27,11 +27,17 @@ prompt_system_context: |
collections:
- justicio
- boe
- boe-multilingual-e5-large
- bocm
- bocm-multilingual-e5-large
- bopz
- bopz-multilingual-e5-large
- bopv
- bopv-multilingual-e5-large
- boja
- boja-multilingual-e5-large
- boa
- boa-multilingual-e5-large

# Openai
llm_model_name: 'gpt-3.5-turbo-0125' # 'gpt-3.5-turbo-1106', 'gpt-4-1106-preview'
Expand Down
14 changes: 14 additions & 0 deletions doc/qdrant/queries.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,20 @@ GET collections
// Get collection info
GET collections/justicio

// Create a new collection
// https://qdrant.tech/documentation/tutorials/bulk-upload/
PUT /collections/{collection_name}
{
"vectors": {
"size": 1024,
"distance": "Cosine"
},
"shard_number": 2,
"optimizers_config": {
"indexing_threshold": 0
}
}

// List points in a collection, using filter by metadata
POST collections/justicio/points/scroll
{
Expand Down
Loading