v0.1.12

What's Changed

• [Fix bug] The adaptation for rewrite_invalid_question_prompt was not saved by @psydok in #1071
• [Fix bug] utils.py by @jrredondo in #1083
• Llama Index integration: optional RunConfig for evaluate() by @antono in #1085
• Create a custom directory loader for RAGAS by @vaishakhRaveendran in #1122
• Fix "sentence" -> "claim" the context recall metric docs by @artmatsak in #1126
• fix: temperature parameter in generate_text not ignored. by @HerrIvan in #887

New Contributors

• @psydok made their first contribution in #1071
• @jrredondo made their first contribution in #1083
• @antono made their first contribution in #1085
• @vaishakhRaveendran made their first contribution in #1122
• @artmatsak made their first contribution in #1126
• @HerrIvan made their first contribution in #887

Full Changelog: v0.1.11...v0.1.12

v0.1.11

What's Changed

• Testgeneration v3 by @shahules786 in #1024
• fix: import errors and make experimental runnable by @jjmachan in #1078
• chore: added analytics to count the num of testsets generated with experimental by @jjmachan in #1081
• Fix mutable dataclass defaults by @yasyf in #1082
• fix(executor): simplify executor by removing threading by @jjmachan in #1093
• chore: added ragas as a dependency by @jjmachan in #1095
• test: added more tests for executor by @jjmachan in #1110
• chore: remove context_relevancy from library by @jjmachan in #1111
• Update index.md by @IdaJess in #1117
• Improve Clarity and Structure of Faithfulness Explanation in Documentation by @aybruhm in #1114
• fix: remove is_async argument since everything is async now by @jjmachan in #1116

New Contributors

• @yasyf made their first contribution in #1082
• @IdaJess made their first contribution in #1117
• @aybruhm made their first contribution in #1114

Full Changelog: v0.1.10...v0.1.11

v0.1.10

What's Changed

• Update testset_generation.md - fix typo by @GTimothee in #1008
• bugfix: evaluation thread timeout by @NIL-zhuang in #858
• Update index.md by @shahules786 in #1023
• Add initial version of summarization metric by @sky-2002 in #990
• feat: added experimental package in ragas by @jjmachan in #1056
• Docs for summarization metric by @sky-2002 in #1026
• adding get_available_metrics method by @immortal3 in #1025
• chore: Fix typo in index.md by @anfindsen in #1058
• fix(testset): for generating testset with new docs by @Manav916 in #1001
• Fix counter in Runner and Executor in case of any exception by @baptiste-pasquier in #995
• Chore: Fix RuntimeError: ... got Future <..> attached to a different loop by @abetatos in #979
• fix: langchain EvaluatorChain OpenAI validation error by @danielfornarini in #1051
• chore: removed async time tests from benchmarks by @jjmachan in #1061
• Feature/updating prompt by @louky123 in #1050

New Contributors

• @GTimothee made their first contribution in #1008
• @NIL-zhuang made their first contribution in #858
• @immortal3 made their first contribution in #1025
• @anfindsen made their first contribution in #1058
• @Manav916 made their first contribution in #1001
• @baptiste-pasquier made their first contribution in #995
• @abetatos made their first contribution in #979
• @danielfornarini made their first contribution in #1051
• @louky123 made their first contribution in #1050

Full Changelog: v0.1.9...v0.1.10

v0.1.9

Feat/llamaindex: adding llamaindex (#999)

fixes: #557 


its been long 🙂

v0.1.8

What's Changed

• Devanshu's Blog Added in Doc's community folder. by @DevanshuBrahmbhatt in #869
• Update data_preparation doc link in evaluation.md by @ragul-kachiappan in #874
• docs: langchain.document_loaders is deprecated by @bioerrorlog in #881
• chores: change cal link by @shahules786 in #892
• chore: added issue manager by @jjmachan in #907
• fix: Feat/issue closer by @jjmachan in #909
• fix(docs): update document regarding langfuse usage by @jjmachan in #904
• docs: clean up integrations by @jjmachan in #919
• doc: Updated vertexai python notebook to use langchain wrapper approach for vertex ai by @shawnmittal in #915
• Add new RAG evaluation articles by @Yongtae723 in #932
• Fix/ci: fix the broken CI by @jjmachan in #933
• reproducibility PR1 : introduce ensembler by @shahules786 in #924
• Cleaned installation step in README by @andreped in #921
• reproducibility PR2: add new statement creation #930 by @shahules786 in #941
• Update generator.py - Redundant code by @omkar-334 in #942
• docs: improve vertex ai docs by @jjmachan in #951
• docs: small fixes in vertex ai docs by @jjmachan in #952
• Update discord invite link by @shahules786 in #953
• fix: sentence segmenter for multi languages by @shahules786 in #946
• answer-correctness : fix edge cases by @shahules786 in #970
• Fix ensembler : handle edge cases by @shahules786 in #972
• feat: adding new defaults to openai Model and embeddings by @jjmachan in #973
• Improve grammar and punctuation in critique prompts: addresses #971 by @ruankie in #974
• Fix testset generator issue on context selection by @Jovillios in #937
• feat: Ragas CI/CD by @jjmachan in #976
• fix: patch type issue in evolution parsing by @shahules786 in #980

New Contributors

• @ragul-kachiappan made their first contribution in #874
• @bioerrorlog made their first contribution in #881
• @shawnmittal made their first contribution in #915
• @andreped made their first contribution in #921
• @omkar-334 made their first contribution in #942
• @ruankie made their first contribution in #974
• @Jovillios made their first contribution in #937

Full Changelog: v0.1.7...v0.1.8

v0.1.7

What's Changed

• minor: fix llm expansion in docs by @glenacota in #847
• Update llama_index imports by @sestinj in #849
• feat: Change parsers and add custom critic model by @shahules786 in #838
• fix: change documentation to reflect accurate method name by @dbtreasure in #837
• Update answer_relevance.md by @carllapierre in #832
• fix ci: some linting issues by @jjmachan in #852

New Contributors

• @glenacota made their first contribution in #847
• @sestinj made their first contribution in #849
• @dbtreasure made their first contribution in #837
• @carllapierre made their first contribution in #832

Full Changelog: v0.1.6...v0.1.7

v0.1.6

What's Changed

• Fix: json format inconsistency in context recall (prompt for statemen… by @jokokojote in #776
• fix: Explanaitions typo on index.md by @dcamejo1 in #801
• feat(integrations): add support for langchain and langsmith by @jjmachan in #795
• Docs: fix confusion regarding getting started by @shahules786 in #815
• docs: add openlayer integration by @shahules786 in #827
• fix: Eventloop unclosed by @yunfwe in #796
• More robust JSON prompting and parsing by @mrtj in #807
• fix(integration): langchain integration and CI fixes by @jjmachan in #830
• Added more examples to the translation prompts by @baswenneker in #826
• fix: use aparse in all metrics by @shahules786 in #831

New Contributors

• @dcamejo1 made their first contribution in #801
• @yunfwe made their first contribution in #796
• @mrtj made their first contribution in #807
• @baswenneker made their first contribution in #826

Full Changelog: v0.1.5...v0.1.6

v0.1.5

What's Changed

• fix: handle string score in node filter by @shahules786 in #755
• Add ipython notebook demonstrating the ragas-haystack integration. by @annthurium in #754
• Update Bring Your Own LLM Docs Link by @pmbaumgartner in #765
• fix: edge case in answer_correctness by @shahules786 in #778
• Added metadata field to datarows by @rnbokade in #782
• fix: is_async not passed for llm generation in context recall by @jokokojote in #777
• fix: Nan in metrics/test set gen by @shahules786 in #786
• Fix adapt issue for keyphrase extractor by @smodlich in #766
• updated context_scoring_prompt and provide extra examples. by @nofrish in #725
• fix: faithfulness by @shahules786 in #787

New Contributors

• @annthurium made their first contribution in #754
• @rnbokade made their first contribution in #782
• @smodlich made their first contribution in #766
• @nofrish made their first contribution in #725

Full Changelog: v0.1.4...v0.1.5

v0.1.4

What's Changed

• docs: Add docs for new entity based metric by @sky-2002 in #674
• Fix typos that cause fatal assertion errors in prompts by @joy13975 in #676
• fix: edge case in faithfulness by @shahules786 in #682
• docs: Fix the precision calculation in context-precision metric doc by @amit-timalsina in #685
• fix: improving reproducability of evals when running everything together by @jjmachan in #671
• added octlane docs by @jjmachan in #695
• Docs: Fix content precision description by @jokokojote in #693
• Docs: Created own section for Calculation (instead of Dropdown) by @GeroVanMi in #696
• Fix: Enable Critique Prompt Customization in AspectCritique by @aaronkim218 in #700
• chore: improve bug/question template by @shahules786 in #702
• docs: clean and smooth by @shahules786 in #703
• Docs: Fix and supplement answer relevancy description by @jokokojote in #705
• fix: remove unwanted quotes from questions by @shahules786 in #708
• Readme Formatting by @gkamradt in #710
• docs: customising embeddings and llms by @shahules786 in #712
• chore: add copyright by @shahules786 in #716
• feat: test set generation improvements by @shahules786 in #719
• Adding Tonic Validate integration notebook by @akamor in #707
• fix: edge case in CR by @shahules786 in #728
• fix(ragas): remove mutable defaults by @mspronesti in #684
• fix: remove experimental filters by @shahules786 in #743
• Correct Faithfulness Verdict Map by @pmbaumgartner in #739
• feat(generator): extend construction to any langchain LLM and Embeddings by @mspronesti in #670
• docs: minor corrections by @shahules786 in #747

New Contributors

• @amit-timalsina made their first contribution in #685
• @jokokojote made their first contribution in #693
• @GeroVanMi made their first contribution in #696
• @aaronkim218 made their first contribution in #700
• @gkamradt made their first contribution in #710
• @akamor made their first contribution in #707
• @mspronesti made their first contribution in #684
• @pmbaumgartner made their first contribution in #739

Full Changelog: v0.1.3...v0.1.4

v0.1.3

What's Changed

• feat: improved node selection and question fixing by @shahules786 in #644
• docs: add explanation to metrics by @shahules786 in #658
• fix: edge cases in metrics by @shahules786 in #661
• Update evolution by @cuent in #666
• feat: Add new "entity-based context recall" metric by @sky-2002 in #667
• fix: filter and context selection by @shahules786 in #672
• Add support for optional max concurrency by @joy13975 in #643

New Contributors

• @cuent made their first contribution in #666
• @sky-2002 made their first contribution in #667

Full Changelog: v0.1.2...v0.1.3