[Doc, Core] Rename Metron to Etalon (#10)

* Rename metron to etalon

* remove logo, need to change

.github/ISSUE_TEMPLATE/100-documentation.yml

@@ -1,5 +1,5 @@
 name: 📚 Documentation
-description: Report an issue related to https://project-metron.readthedocs.io/
+description: Report an issue related to https://project-etalon.readthedocs.io/
 title: "[Doc]: "
 labels: ["documentation"]
 
@@ -8,7 +8,7 @@ body:
   attributes:
     label: 📚 The doc issue
     description: >
-      A clear and concise description of what content in https://project-metron.readthedocs.io/ is an issue.
+      A clear and concise description of what content in https://project-etalon.readthedocs.io/ is an issue.
   validations:
     required: true
 - type: textarea

.github/ISSUE_TEMPLATE/200-installation.yml

@@ -7,7 +7,7 @@ body:
 - type: markdown
   attributes:
     value: >
-      #### Before submitting an issue, please make sure the issue hasn't been already addressed by searching through [the existing and past issues](https://github.com/project-metron/metron/issues?q=is%3Aissue+sort%3Acreated-desc+).
+      #### Before submitting an issue, please make sure the issue hasn't been already addressed by searching through [the existing and past issues](https://github.com/project-etalon/etalon/issues?q=is%3Aissue+sort%3Acreated-desc+).
 - type: textarea
   attributes:
     label: Your current environment
@@ -17,12 +17,12 @@ body:
     required: true
 - type: textarea
   attributes:
-    label: How you are installing Metron
+    label: How you are installing etalon
     description: |
       Paste the full command you are trying to execute.
     value: |
       ```sh
-      pip install -vvv Metron
+      pip install -vvv etalon
       ```
 - type: markdown
   attributes:

.github/ISSUE_TEMPLATE/300-usage.yml

@@ -1,13 +1,13 @@
 name: 💻 Usage
-description: Raise an issue here if you don't know how to use Metron.
+description: Raise an issue here if you don't know how to use etalon.
 title: "[Usage]: "
 labels: ["usage"]
 
 body:
 - type: markdown
   attributes:
     value: >
-      #### Before submitting an issue, please make sure the issue hasn't been already addressed by searching through [the existing and past issues](https://github.com/project-metron/metron/issues?q=is%3Aissue+sort%3Acreated-desc+).
+      #### Before submitting an issue, please make sure the issue hasn't been already addressed by searching through [the existing and past issues](https://github.com/project-etalon/etalon/issues?q=is%3Aissue+sort%3Acreated-desc+).
 - type: textarea
   attributes:
     label: Your current environment
@@ -17,11 +17,11 @@ body:
     required: true
 - type: textarea
   attributes:
-    label: How would you like to use Metron
+    label: How would you like to use etalon
     description: |
-      A detailed description of how you want to use Metron.
+      A detailed description of how you want to use etalon.
     value: |
-      I want to run inference of a [specific inference engine](put link here). I don't know how to integrate it with Metron.
+      I want to run inference of a [specific inference engine](put link here). I don't know how to integrate it with etalon.
 - type: markdown
   attributes:
     value: >

.github/ISSUE_TEMPLATE/400-bug report.yml

@@ -7,7 +7,7 @@ body:
 - type: markdown
   attributes:
     value: >
-      #### Before submitting an issue, please make sure the issue hasn't been already addressed by searching through [the existing and past issues](https://github.com/project-metron/metron/issues?q=is%3Aissue+sort%3Acreated-desc+).
+      #### Before submitting an issue, please make sure the issue hasn't been already addressed by searching through [the existing and past issues](https://github.com/project-etalon/etalon/issues?q=is%3Aissue+sort%3Acreated-desc+).
 - type: textarea
   attributes:
     label: Your current environment

.github/ISSUE_TEMPLATE/500-feature request.yml

@@ -1,13 +1,13 @@
 name: 🚀 Feature request
-description: Submit a proposal/request for a new Metron feature
+description: Submit a proposal/request for a new etalon feature
 title: "[Feature]: "
 labels: ["feature request"]
 
 body:
 - type: markdown
   attributes:
     value: >
-      #### Before submitting an issue, please make sure the issue hasn't been already addressed by searching through [the existing and past issues](https://github.com/project-metron/metron/issues?q=is%3Aissue+sort%3Acreated-desc+).
+      #### Before submitting an issue, please make sure the issue hasn't been already addressed by searching through [the existing and past issues](https://github.com/project-etalon/etalon/issues?q=is%3Aissue+sort%3Acreated-desc+).
 - type: textarea
   attributes:
     label: 🚀 The feature, motivation and pitch

.github/ISSUE_TEMPLATE/600-RFC.yml

@@ -7,7 +7,7 @@ body:
 - type: markdown
   attributes:
     value: >
-      #### Please take a look at previous [RFCs](https://github.com/project-metron/metron/issues?q=label%3ARFC+sort%3Aupdated-desc) for reference.
+      #### Please take a look at previous [RFCs](https://github.com/project-etalon/etalon/issues?q=label%3ARFC+sort%3Aupdated-desc) for reference.
 - type: textarea
   attributes:
     label: Motivation.

.github/ISSUE_TEMPLATE/800-misc discussion.yml

@@ -7,12 +7,12 @@ body:
 - type: markdown
   attributes:
     value: >
-      #### Before submitting an issue, please make sure the issue hasn't been already addressed by searching through [the existing and past issues](https://github.com/project-metron/metron/issues?q=is%3Aissue+sort%3Acreated-desc+).
+      #### Before submitting an issue, please make sure the issue hasn't been already addressed by searching through [the existing and past issues](https://github.com/project-etalon/etalon/issues?q=is%3Aissue+sort%3Acreated-desc+).
 - type: textarea
   attributes:
-    label: Anything you want to discuss about Metron.
+    label: Anything you want to discuss about etalon.
     description: >
-      Anything you want to discuss about Metron.
+      Anything you want to discuss about etalon.
   validations:
     required: true
 - type: markdown

.github/PULL_REQUEST_TEMPLATE.md

@@ -10,7 +10,7 @@ FIX #xxxx (*link existing issues this PR will resolve*)
 <!-- inside this <details> section, markdown rendering does not work, so we use raw html here. -->
 <summary><b> PR Checklist (Click to Expand) </b></summary>
 
-<p>Thank you for your contribution to Metron! Before submitting the pull request, please ensure the PR meets the following criteria. This helps Metron maintain the code quality and improve the efficiency of the review process.</p>
+<p>Thank you for your contribution to etalon! Before submitting the pull request, please ensure the PR meets the following criteria. This helps etalon maintain the code quality and improve the efficiency of the review process.</p>
 
 <h3>PR Title and Classification</h3>
 <p>Only specific types of PRs will be reviewed. The PR title is prefixed appropriately to indicate the type of change. Please use one of the following:</p>
@@ -32,15 +32,15 @@ FIX #xxxx (*link existing issues this PR will resolve*)
 <ul>
     <li>Pass all linter checks. Please use <code>make format</code></a> to format your code.</li>
     <li>The code need to be well-documented to ensure future contributors can easily understand the code.</li>
-    <li>Please add documentation to <code>docs/source/</code> if the PR modifies the user-facing behaviors of Metron. It helps user understand and utilize the new features or changes.</li>
+    <li>Please add documentation to <code>docs/source/</code> if the PR modifies the user-facing behaviors of etalon. It helps user understand and utilize the new features or changes.</li>
 </ul>
 
 <h3>Notes for Large Changes</h3>
 <p>Please keep the changes as concise as possible. For major architectural changes (>500 LOC), we would expect a GitHub issue (RFC) discussing the technical design and justification. Otherwise, we will tag it with <code>rfc-required</code> and might not go through the PR.</p>
 
 <h3>Thank You</h3>
 
-<p> Finally, thank you for taking the time to read these guidelines and for your interest in contributing to Metron. Your contributions make Metron a great tool for everyone! </p>
+<p> Finally, thank you for taking the time to read these guidelines and for your interest in contributing to etalon. Your contributions make etalon a great tool for everyone! </p>
 
 
 </details>

Makefile

@@ -2,17 +2,17 @@
 .DEFAULT_GOAL := help
 
 lint/black: ## check style with black
-	black --check metron
+	black --check etalon
 
 lint/isort: ## check style with isort
-	isort --check-only --profile black metron
+	isort --check-only --profile black etalon
 
 lint: lint/black lint/isort ## check style
 
 format/black: ## format code with black
-	black metron
+	black etalon
 
 format/isort: ## format code with isort
-	isort --profile black metron
+	isort --profile black etalon
 
 format: format/isort format/black ## format code

README.md

@@ -1,16 +1,16 @@
-<p align="center">
+<!-- <p align="center">
   <picture>
     <source media="(prefers-color-scheme: dark)" srcset="docs/_static/logo/dark.png">
     <img alt="vLLM" src="docs/_static/logo/light.png" width=50%>
   </picture>
-</p>
+</p> -->
 
 <h3 align="center">
 Tool to benchmark LLM Inference Systems
 </h3>
 
 <p align="center">
-| <a href="https://project-metron.readthedocs.io/en/latest/"><b>Documentation</b></a> | <a href="https://arxiv.org/abs/2407.07000"><b>Paper</b></a> |
+| <a href="https://project-etalon.readthedocs.io/en/latest/"><b>Documentation</b></a> | <a href="https://arxiv.org/abs/2407.07000"><b>Paper</b></a> |
 
 </p>
 
@@ -20,18 +20,18 @@ Tool to benchmark LLM Inference Systems
 
 ### Clone repository
 ```bash
-git clone https://github.com/project-metron/metron.git
+git clone https://github.com/project-etalon/etalon.git
 ```
 
 ### Create conda environment
 ```bash
-conda create -n metron python=3.10
-conda activate metron
+conda create -n etalon python=3.10
+conda activate etalon
 ```
 
-### Install Metron
+### Install etalon
 ```bash
-cd metron
+cd etalon
 pip install -e .
 ```
 
@@ -59,7 +59,7 @@ export OPENAI_API_BASE=https://api.endpoints.anyscale.com/v1
 ```
 #### Running Benchmark
 ```bash
-python -m metron.run_benchmark \
+python -m etalon.run_benchmark \
 --model "meta-llama/Meta-Llama-3-8B-Instruct" \
 --max-num-completed-requests 150 \
 --timeout 600 \
@@ -81,11 +81,11 @@ python -m metron.run_benchmark \
 
 There are many more arguments for running benchmark, run the following to know more:
 ```bash
-python -m metron.run_benchmark -h
+python -m etalon.run_benchmark -h
 ```
 
 ### Running with Open Source Systems
-Metron can be run with any open source LLM inference system. If open source system does not provide OpenAI Compatible APIs, then kindly implement new LLM clients to support new open source system as explained in [here](#implementing-new-llm-clients).
+etalon can be run with any open source LLM inference system. If open source system does not provide OpenAI Compatible APIs, then kindly implement new LLM clients to support new open source system as explained in [here](#implementing-new-llm-clients).
 
 Here we give an example with vLLM.
 
@@ -109,11 +109,11 @@ And then we can run the benchmark as shown [here](#running-benchmark). Be sure t
 The results of the benchmark are saved in the results directory specified by the `--output-dir` argument.
 
 ## Running Prefill Profiler
-To profile prefill times of open source systems and create a prefill time predictor for a given model and open source system, based on input prompt length, we can run `metron.prefill_profiler`.
+To profile prefill times of open source systems and create a prefill time predictor for a given model and open source system, based on input prompt length, we can run `etalon.prefill_profiler`.
 
 Launch any open source system and setup API keys and URL as shown for [vLLM](#running-with-open-source-systems).
 ```bash
-python -m metron.prefill_profiler \
+python -m etalon.prefill_profiler \
 --model "meta-llama/Meta-Llama-3-8B-Instruct" \
 --timeout 600 \
 --fixed-request-generator-decode-tokens 16 \
@@ -123,7 +123,7 @@ python -m metron.prefill_profiler \
 
 To modify range of prompt tokens for which prefill times get profiled, use the flag ``--prefill-lengths`` as follows:
 ```bash
-python -m metron.prefill_profiler \
+python -m etalon.prefill_profiler \
 --model "meta-llama/Meta-Llama-3-8B-Instruct" \
 --timeout 600 \
 --output-dir "prefill_experiments/prefill_profiler_vllm_llama-3-8b" \
@@ -134,15 +134,15 @@ python -m metron.prefill_profiler \
 ## Running Capacity Search
 `Important`: Run prefill profiler for a given model and open source system before running capacity search of `deadline-based` SLO type.
 
-Refer to [readme](metron/capacity_search/README.md) file of `metron/capacity_search` folder to know more about how to run capacity search.
+Refer to [readme](etalon/capacity_search/README.md) file of `etalon/capacity_search` folder to know more about how to run capacity search.
 
 ## Implementing New LLM Clients
 
-To implement a new LLM client, you need to implement the base class `metron.llm_client.BaseLLMClient` and decorate it as a ray actor.
+To implement a new LLM client, you need to implement the base class `etalon.llm_client.BaseLLMClient` and decorate it as a ray actor.
 
 ```python
 
-from metron.llm_client import BaseLLMClient
+from etalon.llm_client import BaseLLMClient
 import ray
 
 
@@ -166,8 +166,8 @@ class CustomLLMClient(BaseLLMClient):
 ## Citation
 If you use our work, please consider citing our paper:
 ```cite
-@misc{agrawal2024metronholisticperformanceevaluation,
-      title={Metron: Holistic Performance Evaluation Framework for LLM Inference Systems}, 
+@misc{agrawal2024etalonholisticperformanceevaluation,
+      title={etalon: Holistic Performance Evaluation Framework for LLM Inference Systems}, 
       author={Amey Agrawal and Anmol Agarwal and Nitin Kedia and Jayashree Mohan and Souvik Kundu and Nipun Kwatra and Ramachandran Ramjee and Alexey Tumanov},
       year={2024},
       eprint={2407.07000},

docs/conf.py

@@ -6,9 +6,9 @@
 # -- Project information -----------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
 
-project = "metron"
+project = "etalon"
 copyright = "2024-onwards Systems for AI Lab, Georgia Institute of Technology"
-author = "Metron Team"
+author = "etalon Team"
 
 # -- General configuration ---------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
@@ -29,12 +29,12 @@
 html_static_path = ["_static"]
 
 # -- Added configurations ----------------------------------------------------
-html_title = "metron"
+html_title = "etalon"
 
 html_context = {
     "display_github": True, # Integrate GitHub
-    "github_user": "project-metron", # Username
-    "github_repo": "metron", # Repo name
+    "github_user": "project-etalon", # Username
+    "github_repo": "etalon", # Repo name
     "github_version": "main", # Version
     "conf_py_path": "/docs/", # Path in the checkout to the docs root
 }

docs/guides/guide.rst

@@ -1,7 +1,7 @@
 Guides
 ======
 
-This section contains additional topics and features that are not covered in the :doc:`../how_to_use` section. These guides are meant to provide a deeper understanding of the library and its features, and help in getting the most out of the ``metron``.
+This section contains additional topics and features that are not covered in the :doc:`../how_to_use` section. These guides are meant to provide a deeper understanding of the library and its features, and help in getting the most out of the ``etalon``.
 
 Check out the following guides to learn more:
 

docs/guides/new_llm_client.rst

@@ -1,11 +1,11 @@
 Implementing New LLM Clients
 ============================
 
-To implement a new LLM client, you need to implement the base class ``metron.llm_client.BaseLLMClient`` and decorate it as a ray actor.
+To implement a new LLM client, you need to implement the base class ``etalon.llm_client.BaseLLMClient`` and decorate it as a ray actor.
 
 .. code-block:: python
 
-    from metron.llm_client import BaseLLMClient
+    from etalon.llm_client import BaseLLMClient
     import ray
 
 

docs/guides/request_generator_providers.rst

@@ -1,12 +1,12 @@
 Configuring Request Generator Providers
 =======================================
 
-``metron`` supports multiple types of request and interval generators. They are of two types:
+``etalon`` supports multiple types of request and interval generators. They are of two types:
 
 1. **Request Interval Generator Provider**: They determine the time interval between consecutive requests.
 2. **Request Length Generator Provider**: They determine the number of prompt and decode tokens for each request.
 
-The following sections describe the different request generator providers available in ``metron``. 
+The following sections describe the different request generator providers available in ``etalon``. 
 
 .. toctree::
     :maxdepth: 2