Releases: explodinggradients/ragas
Releases · explodinggradients/ragas
v0.1.2
What's Changed
- docs: arize/ ragas notebook by @axiomofjoy in #594
- fix: prompt adaptation type checks by @shahules786 in #615
- fix: node indexing in testset generator by @shahules786 in #617
- Update ragas-arize.ipynb by @PubliusAu in #618
- docs: getting started by @jjmachan in #589
- Added testset generation for bedrock by @VpkPrasanna in #626
- Made embeddings and LLMs dependent on metric in evaluate function by @peanutshawny in #628
- docs: bring your own llms and embeddings by @jjmachan in #631
- docs: fix import in the example code: ContextRelevance -> ContextRelevancy by @izikeros in #646
- Fix typo: Teset -> Testset by @joy13975 in #638
- Update ragas-arize.ipynb by @PubliusAu in #637
- renaming ground_truths to ground_truth. by @alexjyong in #640
- fix: added types so you can pass langchain llms and embeddings too by @jjmachan in #652
New Contributors
- @axiomofjoy made their first contribution in #594
- @PubliusAu made their first contribution in #618
- @VpkPrasanna made their first contribution in #626
- @peanutshawny made their first contribution in #628
- @izikeros made their first contribution in #646
- @joy13975 made their first contribution in #638
- @alexjyong made their first contribution in #640
Full Changelog: v0.1.1...v0.1.2
v0.1.1
What's Changed
- Fixing typos, vertextai to vertex ai by @mrchtr in #572
- fix typo by @Kyeongpil in #570
- fix assertions and quality in prompt adaptation by @shahules786 in #574
- fix: ensure dict type by @shahules786 in #578
- style: remove additional comments in prompt.py by @leehanchung in #577
- Add stepkurniawan community contribution by @stepkurniawan in #585
- Parse JSON string in adapt() by @mitmul in #584
- fix: replace string formating by dict by @shahules786 in #588
- [patch] Reduce NaN Occurrences by Simple Prompt Modification for JSON Output for context_precision by @i-w-a in #581
- ci: migrate to latest llama-index version by @shahules786 in #603
- Fix: evolution flows for test generation by @shahules786 in #602
- fix: llamaIndex latest version by @jjmachan in #604
- Chore: add language to event tracking by @shahules786 in #592
- feat(llms.json_load): Recursively load json lists by @pberger514 in #593
New Contributors
- @mrchtr made their first contribution in #572
- @Kyeongpil made their first contribution in #570
- @leehanchung made their first contribution in #577
- @stepkurniawan made their first contribution in #585
- @mitmul made their first contribution in #584
- @i-w-a made their first contribution in #581
- @pberger514 made their first contribution in #593
Full Changelog: v0.1.0...v0.1.1
v0.1.0
What's Changed
- add initial embeddings.encode_kwargs by @lucasiscovici in #371
- feat(testset_generator): support error catching in generation process by @yuukidach in #368
- added class for FastEmbed by @tinomaxthayil in #379
- Updated NLI Verdicts to Binary Format by @lucasiscovici in #384
- added docs for ragas embeddings by @tinomaxthayil in #386
- fix context in _answer_correctness.py by @lucasiscovici in #385
- docs: AI copilot implementation by @nickscamara in #294
- fix: faithfulness for evasive/reject answers by @shahules786 in #392
- Added prompt class by @tinomaxthayil in #388
- Context precision with ground truth by @shahules786 in #391
- Added note to answer_relevance by @shahules786 in #399
- Import evaluate function before patching RagasLLM, fixes #301 by @HySoaKa in #377
- Fix: Added
ground_truths
arg type-hint info in the eval function by @MANISH007700 in #404 - Typo and phrasing correction by @TruscaPetre in #408
- Small improvements to documentation custom LLMs by @TruscaPetre in #410
- feat(metrics): skip embedding process when the corresponding weight is zero by @yuukidach in #409
- added prompt objects to metrics by @tinomaxthayil in #401
- feat: Automatic Prompt adaptation [language] by @shahules786 in #407
- Update _faithfulness.py by @Robs1999 in #416
- fix(metrics): avoid dividing by zero in correctness by @yuukidach in #421
- Feature/doc update ragas score by @prufrock349 in #417
- Docs: Prompts and prompt adaptation by @shahules786 in #419
- fix(metrics): wrong usage of prompt by @yuukidach in #422
- fix: context_utilization name override by @jjmachan in #424
- feat/executor: make ragas faster and more robust to run by @jjmachan in #390
- fix: Import PrompValue by @shahules786 in #427
- fix: add uuid to analytics by @shahules786 in #428
- feat: added async version for
json_load
by @jjmachan in #430 - fix: comment error in analytics by @jjmachan in #433
- fix: payload in analytics by @shahules786 in #434
- fix: tests for analytics by @jjmachan in #435
- Docs: tracing and logging by @shahules786 in #436
- feat: global prompt adaptation by @shahules786 in #432
- chore: remove mac artifacts from store by @jjmachan in #438
- fix: vary temperature based on strictness by @shahules786 in #437
- docs: custom llms by @shahules786 in #442
- feat: improving testset generation by @jjmachan in #381
- Fix: context_recall AttributeError by @0ENZO in #453
- Update faithfulness.md by @francesco-defalco in #451
- Fixing typo in faithfulness by @jamesbraza in #450
- fix: context_recall example by @shahules786 in #458
- fix: answer relevancy parsing by @shahules786 in #461
- feat: MVP for the new TestsetGenerator - SimpleEvolution by @jjmachan in #464
- fix: clean up embeddings for ragas and add docs for azure embeddings by @jjmachan in #477
- fix: polishing
raise_exception
andmax_workers
by @jjmachan in #486 - Convert prompts in testset generation by @shahules786 in #484
- feat(testset generation): implement new
TestsetGenerator
by @jjmachan in #487 - doc: add community page by @shahules786 in #488
- Fix ValueError: diag requires an array of at least two dimensions #474 by @herman159 in #475
- doc: modify prompt adaptation by @shahules786 in #490
- docs: theming community by @jjmachan in #491
- fix: CP example by @shahules786 in #489
- feat(testset): ported simple, multi_context and reasoning evolutions by @jjmachan in #496
- fix: cleanup old testset generator by @jjmachan in #500
- feat: added tracking of minimal stuff in testset generation by @jjmachan in #509
- feat: added generate_from_langchain function by @jjmachan in #511
- docs: updated testset generation by @shahules786 in #510
- fix: answer_correctness embedding by @jjmachan in #513
- added docs for customisation by @tinomaxthayil in #514
- fix: added rate_limter to langchain wrapper by @jjmachan in #517
- Update testset_generation.md by @Uzarel in #516
- fix: reshape matrix by @shahules786 in #519
- fix (metrics): changed ground_truths -> ground_truth by @jjmachan in #521
- docs: fix spelling by @shahules786 in #522
- fix: removed old prompt by @shahules786 in #523
- feat(testset): added conditional evolution by @jjmachan in #524
- fix: derive answer from relevant context by @shahules786 in #525
- feat(executor): remove the need to have both sync and async versions of the function by @jjmachan in #527
- feat: seed questions using keyphrases by @shahules786 in #526
- docs: updated contributors by @jjmachan in #532
- feat: configure retries and timeouts with evaluations and testset generators by @jjmachan in #534
- Fix case for few shots of context precision metric by @HuskyDanny in #541
- Updated imports, added dotenv by @aevedis in #539
- feat(executor): run executor in different thread to keep things clean by @jjmachan in #544
- add normalization factor to answer similarity score calculation by @enestydarealmc in #538
- Feat: prompt adaptation for test data generation by @shahules786 in #530
- fix:handle non dict output by @shahules786 in #547
- docs: adaptation for test set generator by @shahules786 in #548
- Fix: ensure quality of adapted prompts by @shahules786 in #546
- fix: raise warning for invalid JSON by @shahules786 in #550
- docs: adding code highlighting by @jjmachan in #552
- fix: wrap langchain embeddings automatically by @jjmachan in #553
- Added documentation page: Using Ragas with Athina for production performance metrics by @shivsak in #558
- fix: proper error message and fixed hf embedding error by @jjmachan in #561
- fix: answer_correctness doesn't reset stuff properly by @jjmachan in #562
- docs: fix issue template by @jjmachan in #565
- docs: fix issue template by @jjmachan in #566
- fix: handle non dict cases by @shahules786 in #564
- fix wrong type anno...
v0.1.0rc1
What's Changed
- add initial embeddings.encode_kwargs by @lucasiscovici in #371
- feat(testset_generator): support error catching in generation process by @yuukidach in #368
- added class for FastEmbed by @tinomaxthayil in #379
- Updated NLI Verdicts to Binary Format by @lucasiscovici in #384
- added docs for ragas embeddings by @tinomaxthayil in #386
- fix context in _answer_correctness.py by @lucasiscovici in #385
- docs: AI copilot implementation by @nickscamara in #294
- fix: faithfulness for evasive/reject answers by @shahules786 in #392
- Added prompt class by @tinomaxthayil in #388
- Context precision with ground truth by @shahules786 in #391
- Added note to answer_relevance by @shahules786 in #399
- Import evaluate function before patching RagasLLM, fixes #301 by @HySoaKa in #377
- Fix: Added
ground_truths
arg type-hint info in the eval function by @MANISH007700 in #404 - Typo and phrasing correction by @TruscaPetre in #408
- Small improvements to documentation custom LLMs by @TruscaPetre in #410
- feat(metrics): skip embedding process when the corresponding weight is zero by @yuukidach in #409
- added prompt objects to metrics by @tinomaxthayil in #401
- feat: Automatic Prompt adaptation [language] by @shahules786 in #407
- Update _faithfulness.py by @Robs1999 in #416
- fix(metrics): avoid dividing by zero in correctness by @yuukidach in #421
- Feature/doc update ragas score by @prufrock349 in #417
- Docs: Prompts and prompt adaptation by @shahules786 in #419
- fix(metrics): wrong usage of prompt by @yuukidach in #422
- fix: context_utilization name override by @jjmachan in #424
- feat/executor: make ragas faster and more robust to run by @jjmachan in #390
- fix: Import PrompValue by @shahules786 in #427
- fix: add uuid to analytics by @shahules786 in #428
- feat: added async version for
json_load
by @jjmachan in #430 - fix: comment error in analytics by @jjmachan in #433
- fix: payload in analytics by @shahules786 in #434
- fix: tests for analytics by @jjmachan in #435
- Docs: tracing and logging by @shahules786 in #436
- feat: global prompt adaptation by @shahules786 in #432
- chore: remove mac artifacts from store by @jjmachan in #438
- fix: vary temperature based on strictness by @shahules786 in #437
- docs: custom llms by @shahules786 in #442
- feat: improving testset generation by @jjmachan in #381
- Fix: context_recall AttributeError by @0ENZO in #453
- Update faithfulness.md by @francesco-defalco in #451
- Fixing typo in faithfulness by @jamesbraza in #450
- fix: context_recall example by @shahules786 in #458
- fix: answer relevancy parsing by @shahules786 in #461
- feat: MVP for the new TestsetGenerator - SimpleEvolution by @jjmachan in #464
- fix: clean up embeddings for ragas and add docs for azure embeddings by @jjmachan in #477
- fix: polishing
raise_exception
andmax_workers
by @jjmachan in #486 - Convert prompts in testset generation by @shahules786 in #484
- feat(testset generation): implement new
TestsetGenerator
by @jjmachan in #487 - doc: add community page by @shahules786 in #488
- Fix ValueError: diag requires an array of at least two dimensions #474 by @herman159 in #475
- doc: modify prompt adaptation by @shahules786 in #490
- docs: theming community by @jjmachan in #491
- fix: CP example by @shahules786 in #489
- feat(testset): ported simple, multi_context and reasoning evolutions by @jjmachan in #496
- fix: cleanup old testset generator by @jjmachan in #500
- feat: added tracking of minimal stuff in testset generation by @jjmachan in #509
- feat: added generate_from_langchain function by @jjmachan in #511
- docs: updated testset generation by @shahules786 in #510
- fix: answer_correctness embedding by @jjmachan in #513
New Contributors
- @lucasiscovici made their first contribution in #371
- @nickscamara made their first contribution in #294
- @HySoaKa made their first contribution in #377
- @TruscaPetre made their first contribution in #408
- @Robs1999 made their first contribution in #416
- @prufrock349 made their first contribution in #417
- @0ENZO made their first contribution in #453
- @francesco-defalco made their first contribution in #451
- @jamesbraza made their first contribution in #450
- @herman159 made their first contribution in #475
Full Changelog: v0.0.22...v0.1.0rc1
v0.0.22
What's Changed
- docs: comparing llms using ragas evaluations by @tinomaxthayil in #314
- convert context precision to json format by @shahules786 in #322
- fix: faithfulness.md by @kaiyikang in #328
- fix: structure faithfulness output by @shahules786 in #333
- fix: Column remap fix for Dataset evaluation by @tinomaxthayil in #331
- Fix OpenAIEmbeddings reference error in answer_relevancy by @ajinkyavbhandare in #329
- Fix: minor improvements across metrics by @shahules786 in #337
- fix wrong number by @Yongtae723 in #338
- fix: faithfulness for unrelated answering by @shahules786 in #340
- fix: load response as json by @shahules786 in #341
- docs: remove answer_relevancy from llm-docs by @tinomaxthayil in #343
- fix: bedrock llm error when evaluating rag qa
validate_api_key
by @arm-diaz in #350 - Added integration with Amazon API Gateway + bug fix in _answer_relevance.py by @brunopistone in #327
- feat: improve quality of answer correctness by @shahules786 in #339
- docs: add zeno visualization integration by @cabreraalex in #359
- fix(metrics): avoid getting none in context_recall by @yuukidach in #355
- fix: improved answer relevancy by @shahules786 in #346
- fix(types): fix pyright type issues with latest pyright version by @jjmachan in #366
- Retry for failed json loading by @shahules786 in #364
- fix: handle edge cases in prompt processing by @shahules786 in #374
New Contributors
- @kaiyikang made their first contribution in #328
- @ajinkyavbhandare made their first contribution in #329
- @brunopistone made their first contribution in #327
- @cabreraalex made their first contribution in #359
- @yuukidach made their first contribution in #355
Full Changelog: v0.0.21...v0.0.22
v0.0.21
What's Changed
- docs: Import AnswerSimilarity by @nelsonauner in #296
- FIx: testset_generator.py by @ferdinandl007 in #291
- Case-insensitive and whitespace handling for context_precision. by @Pauldevillers in #297
- Fix/tiny refactor by @Yongtae723 in #305
- fix: broken link to metrics by @younes-io in #311
- Convert context_recall to json format by @shahules786 in #312
- fix: add retry logic for OpenAI and Azure OpenAI by @jjmachan in #315
- fix: openai env var load after init and before score also by @jjmachan in #316
- import llama_index gracefully, solves #318 by @ceferisbarov in #319
- Make testgenerator output compatible with evaluate by @tinomaxthayil in #302
New Contributors
- @nelsonauner made their first contribution in #296
- @ferdinandl007 made their first contribution in #291
- @Pauldevillers made their first contribution in #297
- @younes-io made their first contribution in #311
- @ceferisbarov made their first contribution in #319
Full Changelog: v0.0.20...v0.0.21
v0.0.20
What's Changed
- docs: update how LLMs are customized in Ragas metrics by @jjmachan in #219
- fix: Add RagasLLM for customized test set generation by @chankeith96 in #242
- docs: add import statement in testset generation by @jjmachan in #249
- expose metrics prompts by renaming metric modules by @grauvictor in #243
- fix: added support for google vertexAI by @jjmachan in #252
- Renamed module names and made changes in init to reflect the change by @tinomaxthayil in #244
- added hint in synthetcic data generation by @tinomaxthayil in #259
- feat: RagasEmbeddings by @shahules786 in #232
- Fix: type hints by @shahules786 in #263
- Update issue templates by @shahules786 in #265
- Update evaluation.md typo by @whitewum in #275
- removed ragas score from result object by @tinomaxthayil in #276
- Custom embeddings from HuggingFace (sentence transformer) do not pass the assertion in src/ragas/metrics/_answer_similarity.py by @alebondarenko in #277
- fix: set temperate to near zero value by @shahules786 in #267
- feat: add native support for OpenAI and AzureOpenAI by @jjmachan in #261
- feat: make ragas pip more lighter by @jjmachan in #283
- docs: amazon bedrock (#241) by @arm-diaz in #246
- docs: added vertext ai support by @jjmachan in #284
- chore: pin OpenAI to >1 by @jjmachan in #285
New Contributors
- @chankeith96 made their first contribution in #242
- @grauvictor made their first contribution in #243
- @tinomaxthayil made their first contribution in #244
- @whitewum made their first contribution in #275
- @alebondarenko made their first contribution in #277
Full Changelog: v0.0.19...v0.0.20
v0.0.19
What's Changed
- docs: fix typo by @shahules786 in #214
- feat: context average precision by @shahules786 in #204
- fix: avoid division by zero by @shahules786 in #218
- fix: typecast in TestsetGeneration fails for python3.8 by @jjmachan in #215
- fix: make context_relevancy available by @jjmachan in #229
- feat: add support for llms on AWS Bedrock (#226) by @arm-diaz in #227
- tests: for metrics by @jjmachan in #233
- docs: add context_precision by @shahules786 in #236
- docs: add langfuse notebook by @jjmachan in #216
- Fix typos in langfuse.ipynb by @marcklingen in #237
New Contributors
- @arm-diaz made their first contribution in #227
- @marcklingen made their first contribution in #237
Full Changelog: v0.0.18...v0.0.19
v0.0.18
What's Changed
- fix: use llm from baseclass for metrics by @jjmachan in #189
- bugfix: context_precision by @shahules786 in #194
- Fix typos in Eval doc by @mmaysami in #198
- testset generation : Improvements by @shahules786 in #195
- feat: langchain documents support for TestsetGenerator by @jjmachan in #201
- docs: document loaders support by @shahules786 in #202
- feat: support for multiple n in llms by @jjmachan in #197
- feat: llamaIndex llm support by @jjmachan in #205
- Refactor testset_generator.py for newbies to understand easily by @Yongtae723 in #207
- fix: remove limit on pydantic 2.0 by @jjmachan in #209
New Contributors
Full Changelog: v0.0.17...v0.0.18
v0.0.17
What's Changed
- Testset generation: add gaurdrails to chunk filtering by @shahules786 in #166
- Docs testset generation by @shahules786 in #165
- don't warn when
context_precision
is correctly called by @monstertruck in #168 - fixed chunk validation by @shahules786 in #169
- doc: init docs.ragas.io by @jjmachan in #170
- docs: Add references to ragas by @jjmachan in #171
- Docs: added contents by @shahules786 in #175
- Answer correctness metrics by @shahules786 in #161
- added imports to doc by @shahules786 in #178
- docs: added github and discord links by @jjmachan in #179
- Fix typos by @pitmonticone in #180
- docs: add refferences by @jjmachan in #177
- docs: support notebooks by @jjmachan in #182
- added emojis to docs by @shahules786 in #183
- Docs improvements by @shahules786 in #186
- testset generation: bug fixes by @shahules786 in #185
New Contributors
- @monstertruck made their first contribution in #168
- @pitmonticone made their first contribution in #180
Full Changelog: v0.0.16...v0.0.17